emrg: GUI integration tests skip when a live daemon owns the fixed port (fixed-port admission regression) - #906
Merged
Conversation
…rt (fixed-port admission regression) Since #861 (rant 2026-08-19T08:05:21) the daemon binds a FIXED loopback port (56031) as its single-instance admission, and #884 (rant 2026-08-20T14:32:52) moved the auth credential to emrgd.token (port no longer carried in any file). The GUI integration suite spawns its OWN isolated daemon (HOME->tmp) — on any host where the real daemon is already running (the normal dev-machine state), the isolated daemon cannot bind 56031 (EADDRINUSE) and exits, so every test times out with "daemon token file timeout" (7 failures on ). CI stays green only because runners have no live daemon. Fix: probe 127.0.0.1:56031 at module load; when a live daemon already owns the fixed port, skip the whole suite with a clear message (same mechanism as EMRG_SKIP_INTEGRATION, extended to the live-daemon state — the isolated-daemon premise is impossible there). CI (daemon-free runners) still runs the full suite. Also sync Agent.md GUI test count: 260 -> 254 (renderer.smoke 131->126, i18n 16->15 drifted in #896-#905 without doc re-sync; the doc-count guard only checks breakdown-sum consistency, not actual collection). Verified: pytest 984+1 green, GUI 247 pass / 0 fail (8 skipped on this daemon-hosting host), import + --help OK.
argszero
commented
Aug 20, 2026
argszero
left a comment
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle. Verified locally: the fixed-port probe correctly detects the live daemon and skips the suite (npm test: 247 pass / 0 fail / 8 skipped), pytest 984+1 green (incl. doc-count guard 4/4), import + --help OK. CI (test + test-windows) green on daemon-free runners, where the suite runs in full.
argszero
commented
Aug 20, 2026
argszero
left a comment
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (2/3). Fresh review on branch 80317db: live daemon owns the fixed port on this machine → port probe returns true → whole integration suite skips (npm test: 247 pass / 0 fail / 8 skipped); pytest 984 passed + 1 skipped (doc-count guard intact, Agent.md synced to 254). CI green on daemon-free runners (test + test-windows), where the suite runs in full. Probe logic verified in both states: connect-success → skip; connect-error/timeout → run.
Collaborator
|
Independent test report (Contributor, pm25coder) — tested on a live-daemon machine, i.e. exactly the state this PR targets: |
argszero
commented
Aug 21, 2026
argszero
left a comment
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (3/3). Third independent review confirming merge conditions: CI green (test + test-windows, run 32427916901), mergeable CLEAN, 2 prior ✅ from different cycles, no ❌. The fixed-port probe correctly auto-skips the integration suite when the real daemon owns 56031 (verified locally: npm test 247 pass / 0 fail / 8 skipped), while CI runners (daemon-free) still run the full suite. Merge.
argszero
pushed a commit
that referenced
this pull request
Sep 10, 2026
) The doc-count guard only validated Agent.md's GUI line against *itself* (parts sum to the headline, #584) — it could not see the test files, so the per-file counts could drift silently. #906 hit exactly that when it hand-synced the GUI count 260 -> 254 after #896-#905 drifted, and called the gap out in its own message ("the doc-count guard only checks breakdown-sum consistency, not actual collection"). R2254 later gave the *renderer* headline a static reality check, but the GUI line stayed unguarded. This adds test_gui_breakdown_matches_static_counts: it globs emrg/gui/test/*.test.js, counts the `test(`/`it(` definitions per file (node --test reports one entry per definition, so the definition count is the executed total), and asserts every file is documented with a matching count — no undocumented file, no stale label, headline == the real total. It runs in the pytest job, so no node_modules are needed. The integration file's conditional module-level skip(<reason>) entry (#906) is a runtime reason entry, not a definition, so it is excluded on purpose: Agent.md's 100 counts definitions, while CI's `EMRG_SKIP_INTEGRATION=1 npm test` prints 101 including that skip entry. Verified positive: tests/test_doc_counts.py 6 passed; negative: corrupting "7 integration" -> "8 integration" fails with "Agent.md documents 8 integration GUI tests but 7 are defined ...", and renaming a label fails with "undocumented files: ['preload-api']; stale labels: ['preload-api-x']". Full suite 1243 collected (1175 passed, 68 skipped) — Agent.md pytest count 1242 -> 1243; import + --help OK. Co-authored-by: EMRG Evolution <emrg@argszero.dev>
argszero
added a commit
that referenced
this pull request
Sep 10, 2026
…ool) (#1126) Agent.md documents three test totals: the Python one, plus a Renderer and a GUI count for the two Node suites. tests/test_doc_counts.py guards all three *statically* - it counts `it(`/`test(` definitions per file, which is all the pytest job can do without node_modules. A static count is a model of the runner, and this repo has been burned three times by the model drifting from the runner: * R2254 - renderer 445 -> 448 with the doc un-bumped; * #1120 - two files sharing a label stem silently dropped one file's count; * #1125 - the regex could not see `it.each(...)` / `test.skip(...)` at all. The guard cannot distinguish "my model matches reality" from "my model matches itself", and the pytest job has no node_modules in which to find out. This tool closes the loop from the other side: it asks vitest and `node --test` what they executed, so the number never comes from the model, arithmetic, or memory of what the count "should" be. It is the sibling of scripts/check-doc-count.py (same --write / --dry-run contract, same fail-loud rules) and Agent.md now documents both side by side. Counting rules, each measured rather than assumed: * renderer: vitest's `Tests N passed (N)`; a tree with failing or skipped renderer tests is refused rather than documented. * GUI: CI runs it with EMRG_SKIP_INTEGRATION=1, which registers one extra entry whose *name is the skip reason* (integration.test.js's module-level skip, #906) - so the definition count is `tests - 1`. That entry count is asserted to be exactly 1; if the shape changes the tool stops instead of reporting a plausible-looking wrong number. Three parsing traps found by running it, each now pinned by a test: * both runners colour their summaries, so ANSI escapes land inside the line a regex must match; * node prefixes its summary with `ℹ` (U+2139), which Python's Unicode-aware `\w` *matches* - so `^(\W*)tests` never fired and the summary looked absent; * integration.test.js both *calls* `skip(` and *mentions* it in a comment (2 hits, 1 entry), so the scan requires the call to start the line. Verification (main clone): `scripts/check-node-test-count.py` -> OK, 514 renderer + 100 GUI, both runners agreeing with Agent.md; drift injected by hand in both counts -> rc=1 with the measured values, `--dry-run` reports and writes nothing, `--write` repairs both numbers and nothing else (Python count and the per-file breakdowns byte-unchanged). Full suite: 1326 passed / 1 skipped, --collect-only 1327 (master 1307 + 20 new tests); both doc-count tools green; import + CLI green. Co-authored-by: EMRG Evolution <emrg@argszero.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the GUI integration suite failing (7× "daemon token file timeout") on any host where the real daemon is running.
Problem
Since #861 (rant 2026-08-19T08:05:21) the daemon binds a FIXED loopback port (56031) as its single-instance admission, and #884 (rant 2026-08-20T14:32:52) moved the auth credential to
emrgd.token(port no longer carried in any file). The GUI integration suite spawns its OWN isolated daemon (HOME→tmp) — on any host where the real daemon is already running (the normal dev-machine state: "the server is the living core"), the isolated daemon cannot bind 56031 (EADDRINUSE) and exits → no token file → every test times out. CI stays green only because runners have no live daemon.Changes
emrg/gui/test/integration.test.js: probe127.0.0.1:56031at module load; when a live daemon already owns the fixed port, skip the whole suite with a clear message (same mechanism asEMRG_SKIP_INTEGRATION, extended to the live-daemon state — the isolated-daemon premise is impossible there).before()short-circuits on the same flag so the daemon-spawn timeout never fires. CI (daemon-free runners) still runs the full suite.Agent.md: sync GUI test count 260 → 254 (renderer.smoke 131→126, i18n 16→15 drifted in emrg: GUI task form sandbox field — three-tier select with backend-consistent default #896-emrg: GUI task-run subtable — markdown work/reason + newest-first sort + click-to-expand cells (rant 2026-08-20T22:59:16) #905 without doc re-sync; the doc-count guard only checks breakdown-sum consistency, not actual collection).Verification
npm test: 247 pass / 0 fail / 8 skipped on this daemon-hosting host (was 7 fail)emrg --helpOK