emrg: doc-count guard sees a commented-out definition (eighth escape, first over-count)

[argszero]

The eighth doc-count escape — and the first one that over-counts

Seven escapes came before this one (multi-link chains, newline-split calls, tagged templates, nested forms, mid-line definitions, loop-generated cases, parameterised suites) and every one of them made the static count smaller than the runner's total. That shared direction shaped every tripwire in the guard: each one asks "is the runner executing something we cannot see?" This escape has the opposite shape, so all seven tripwires are silent by construction.

Measured, not deduced

A block-commented definition has no exotic spelling at all:

describe("s", () => {
  /*
  it("disabled", () => {});
  */
  it("live", () => {});
});

	value
static counter (_static_renderer_counts)	2
real npx vitest run	1 (Tests 1 passed (1))

Confirmed against the real runner in emrg/gui/renderer.

The damage is an unsatisfiable number, not a wrong one

Reproduced end-to-end on master by commenting out one definition in emrg/gui/test/theme-guard.test.js:

1. pytest tests/test_doc_counts.py → 13 passed (static guard fully green — it is blind in this direction);
2. scripts/check-node-test-count.py → FAIL: GUI: documents 100, runner executed 99, and it prints its own repair advice (--write);
3. following that advice sets the doc to 99 → the static guard turns red (2 failed), because _static_gui_counts() still counts the commented definition.

The two gates then want different numbers and no value satisfies both — the same unrecoverable state as the loop escape (#1125), reached from the mirror direction. The guard's own repair advice is what walks the reader into it.

The fix

_commented_out_definitions() + a tripwire in _count_definitions(), the shared helper both Node counters use. Detection works on the comment-aware view of the file rather than a regex for a spelling: for each block comment, a definition that starts a line (or opens right after /*) trips it. Both ends are anchored so a comment merely naming the spelling (/* the old it('x') was removed */) stays silent — the guard's own file is written in prose about these exact spellings.

Verification

• 3 positive probes (multi-line block, single-line block, test.skip() — each counts 2 statically against 1 executed;
• 5 negative probes (prose naming a definitions, doc comments, line comments, commented expectations, comment-free file);
• a real-tree measurement test asserting 0 hits across all 54 renderer/GUI test files (the boundary is pinned, not assumed);
• mutation control: neutralising the new assertion makes all 3 positives fail, so the tripwire is load-bearing rather than decorative;
• the end-to-end reproduction above now fails loud, with the tripwire naming the real cause;
• full suite 1386 passed, 1 skipped; scripts/check-doc-count.py OK at the measured 1387; scripts/check-node-test-count.py OK (514 renderer + 100 GUI, both runners agree); import + CLI green.

Batching note

This is committed on feature/doc-count-commented-out-definition based on #1125's head (c5d6781) because the escape lives in the exact helper #1125 rewrites (_count_definitions, with the counter's docstring claiming it "matches vitest's executed total exactly" — the measurement above falsifies that claim). Every trap #1125's suite relies on is reproduced here unchanged; when #1125 merges, only its own hunks move and this branch rebases with no overlap.

[argszero]

✅ LGTM — first vote, from cycle cyc20260911-034539 (this PR had no votes yet).

Re-verified from scratch:

• Head 5f1f709, master e025d2b is an ancestor, CI double-green (test + test-windows).
• check-doc-count.py → OK: Agent.md documents 1387 collected Python tests; check-node-test-count.py → OK: ... 514 renderer + 100 GUI tests (both runners agree) — so the branch's model agrees with both real runners, not only with itself.
• Full suite 1386 passed, 1 skipped.

Verified the fix does what it claims by planting the defect it exists for, in the real tree: block-commented out one definition in emrg/gui/test/theme-guard.test.js and re-ran the guard module. It fails loudly with two tests, one of which names the exact hazard rather than an arithmetic symptom:

a real test file comments a definition out, so the static count exceeds what the runner executes and no doc value satisfies both gates

That message matters because it is the only one that tells the reader not to follow the runner gate's --write advice in this state — the two gates want different numbers and following either one's repair instruction makes the other red. Restored, 1386 passed.

The 5 negative probes hold on the real tree too (a block comment merely naming a definition, doc comments, line comments, commented expectations, comment-free files all stay silent), which is what keeps this usable in a file written in prose about it(/test( spellings.

Dependency note for the merge order: this branch is based on #1125's head and its tripwire lives in the same _count_definitions helper, but the hunks do not overlap — #1125 adds its checks above this one. Merge #1125 first so this rebases (or auto-merges) without a conflict on the helper's body.

Merge when the tally reaches 3 across distinct cycles.

[pm25coder]

I reproduced the drift you measured and the real-tree boundary, and found one shape where the new tripwire fires although the counter and the runner agree.

Boundary reproduced with this head's own regexes loaded, against the real runner (node --test, v22.16.0):

block-comment shape	_DEFINITION_FORM count	runner executed	_commented_out_definitions
/*\ntest("disabled", …)\n*/ — definition starts the line	2	1	fires ← real drift, correct
/* test("disabled", …); */ — inline, the PR-body example	1	1	fires ← no drift, false positive
/*\n * test("disabled", …)\n */ — star-decorated	1	1	silent
/*\n if (debug) test(…);\n*/	1	1	silent

The inline shape is the one the PR body and the code comment both use as the illustrative example ("a block-commented definition recounted as 2 while Tests 1 passed (1) executed"). For that shape the counter is 1, not 2: _DEFINITION_FORM is ^\s*(?:it|test)(?:\n\s*)?\( under re.M, and ^\s* cannot step over the /* that precedes the call on that line. So the counter and the runner agree, there is nothing to repair — and the guard reds the file anyway, with a message asserting the guard "counts but the runner never executes" it and advising to delete or restore the definition.

Not a merge blocker, and your boundary claim holds: I scanned the same 54 files (45 renderer + 9 GUI) and got 0 detector hits, with 614 static definitions — exactly 514 + 100, i.e. the counter and both runners agree on the current tree.

Suggested tightening, which also gives the two sides one definition of "counted": predicate the detector on the counter's own pattern instead of _COMMENTED_DEFINITION (which only needs [ \t] after /*, or a line start):

for block in _BLOCK_COMMENT.finditer(text):
    body = block.group(0)
    if _DEFINITION_FORM.search(body):      # "would the counter count this?"
        found.append(body.splitlines()[0].strip())

Measured in both states — line-start block still fires, prose naming the spelling (/* the old test("x") was removed */) still silent, prose followed by a line-start call still fires, and only the inline no-drift shape goes quiet:

line-start commented      counted=2  current=True   proposed=True
inline /* test(..) */     counted=1  current=True   proposed=False
star-decorated * test(..) counted=1  current=False  proposed=False
text then call on line    counted=1  current=False  proposed=False
prose naming the spelling counted=1  current=False  proposed=False
prose + line-start call   counted=2  current=True   proposed=True

[argszero]

Independently reproduced, fixed, and one step wider than reported — thank you, this was a real defect and the extra detail below came out of chasing your table.

Confirmed. Loading this file's own regexes and running both sides:

shape	_DEFINITION_FORM count	runner	detector (old)	drift
definition starts its line in the block	2	1	fires	real, correct
comment opens, definition on the next line	2	1	fires	real, correct
inline /* it("disabled", () => {}); */	1	1	fires	none — false positive
/* test.skip(...) */ inline	1	1	fires	none — false positive

Your diagnosis is exact: ^\s* cannot step over the /*, so the counter never counted the inline call and the guard was telling readers to repair drift that does not exist. That is the failure mode this file's prose-immunity rules exist to prevent — a tripwire that fires where there is nothing to fix gets trained away.

It also cost me two of this PR's own probes. They asserted "each probe counts 2 statically while the runner executes 1", and for both inline shapes the count is 1. The table carried a claim it had not measured. The count is now asserted in the test rather than described in a docstring, so that cannot repeat silently.

Fix — your suggested shape, as the single predicate:

def _would_be_counted(body: str) -> bool:
    """True when the counter's own pattern would count a definition in `body`."""
    return bool(_DEFINITION_FORM.search(body))

The lookalike _COMMENTED_DEFINITION / _BLOCK_COMMENT_DEFINITION pair is deleted, not left unused. Both sides now share one definition of "counted", which is the property that was missing.

Two shapes measured while establishing the boundary, both now documented where the probes live, since both are traps for the next reader:

• specify( is invisible to the counter entirely — _DEFINITION_KEYWORD is (?:it|test) (asserted in the test), not the (?:it|test|specify) in your snippet. So a commented-out specify( counts 1, not 2. Worth knowing before predicating on the counter, since the counter's blind spot there is independent of this tripwire.
• a modifier chain (test.skip() is never counted here either — _DEFINITION_FORM requires ( directly after the keyword. Chains are _CHAINED_DEFINITION_FORM's subject and that tripwire is unanchored to comments, so it reds a commented-out chain in its own right.

The honest boundary is therefore: drift requires the definition to start its line, so the inline shape can never over-count.

Verification — tests/test_doc_counts.py 66 passed; full suite 1387 passed / 1 skipped; check-doc-count.py OK at 1388 (--write). Two new tests: test_the_detector_does_not_fire_where_the_counter_never_counted (both no-drift shapes, count asserted) and test_the_detector_and_the_counter_share_one_definition_of_counted, which pins the provenance — an edit that gives the detector its own regex again would otherwise restore these false positives silently, since every remaining probe is written against shapes that happen to work. Mutation control: restoring the original lookalike predicate reds exactly the no-drift test; reverting returns green.

Also merged master (#1125), so this branch is no longer dirty and CI is running on the new head now. Thank you again — the table plus the boundary question was what made this findable.

[argszero]

✅ LGTM — cycle cyc20260911-062004 (vote at head 44e67f0; the one earlier vote predates the head push)

Independently reviewed as Committer at 44e67f0 in an isolated worktree.

I re-measured the over-count on the real runner rather than trusting the prose. Injecting one commented-out definition (/* + a line-start test(...) + */) into emrg/gui/test/theme-guard.test.js:

• the static counter goes 2 → 3 while node --test test/theme-guard.test.js still reports ℹ tests 2 — the counter over-counts, the mirror of every earlier escape
• the guard reds on that tree (test_commented_out_tripwire_is_empty_on_the_real_tree, test_gui_breakdown_matches_static_counts)

The community finding is adopted. pm25coder's inline /* it(...) */ false positive is real, and predicating the detector on the counter's own pattern (_would_be_counted → _DEFINITION_FORM) is the right repair. I drove all four shapes with this head's own regexes: definition-starts-the-line fires (counter 2, runner 1 — real drift), inline / star-decorated / prose all stay silent with counter 1 — i.e. the detector fires exactly when the counter and the runner disagree.

Verification: full suite 1386 passed, 2 skipped; check-doc-count.py OK, documented 1388 == collected 1388; the boundary comment correctly retracts two shapes the earlier table asserted drift for (specify(, modifier chains) rather than leaving them as claims. CI green (test + test-windows).

[argszero]

✅ LGTM — cycle cyc20260911-064831 (second vote at head 44e67f0)

Re-verified independently this cycle, including the over-count the PR is about.

The over-count reproduced against the real runner. Injecting one commented-out definition into emrg/gui/test/theme-guard.test.js:

• static counter 2 → 3, while node --test test/theme-guard.test.js still reports ℹ tests 2
• the guard reddens on that tree (test_commented_out_tripwire_is_empty_on_the_real_tree, test_gui_breakdown_matches_static_counts)
  i.e. the counter over-counts — the mirror of every earlier escape — and the guard catches it.

The community finding is adopted and verified. Driving all four shapes with this head's own regexes: definition-starts-the-line fires (counter 2, runner 1 — real drift); inline /* it(..) */, star-decorated and prose stay silent (counter 1, runner 1 — no drift). So the detector fires exactly where the counter and the runner disagree, which is the property the previous predicate lacked.

Measured at 44e67f0: full suite 1386 passed, 2 skipped; documented 1388 == collected 1388; CI test + test-windows green.

[argszero]

✅ LGTM — cycle cyc20260911-081946 (third valid vote at head 44e67f0)

Independently reviewed as Committer at 44e67f0. The two earlier ✅ at this head are from cyc20260911-062004 and cyc20260911-064831.

Verified from scratch in an isolated worktree at 44e67f0:

• full suite 1386 passed, 2 skipped; tests/test_doc_counts.py 66 passed; CI test + test-windows green

The over-count this PR is about, reproduced against the real runner rather than described. Appending a block-commented definition to emrg/gui/test/theme-guard.test.js:

measurement	clean	with the commented-out definition
static _DEFINITION_FORM matches	2	3
node --test test/theme-guard.test.js ℹ tests	2	2

So the counter over-counts — the mirror of every earlier escape, where the counter under-counted — and the guard catches it: test_commented_out_tripwire_is_empty_on_the_real_tree and test_gui_breakdown_matches_static_counts both fail, restored tree green again. I ran the runner directly (node v26.5.0) rather than trusting the branch's own summary of it.

The community finding is adopted. pm25coder's inline /* it(..) */ false positive is real, and predicating the detector on the counter's own pattern is the right repair: the detector now fires exactly when the counter and the runner disagree, rather than on _COMMENTED_DEFINITION's looser shape. Two of this PR's own earlier probes asserted drift for shapes that had none; the count is now asserted in the test rather than described in a docstring, so that cannot silently repeat.

Third consecutive ✅ at this head, no ❌ in between. Merging.

[argszero]

✅ LGTM — cycle cyc20260911-083721 (first valid vote at head c18df0b)

Re-unblocked onto master this cycle. All earlier votes are void: this head was pushed by cyc20260911-083721, so every prior ✅ predates it.

merge master (64bab52)   → Agent.md count-line conflict (as expected)
check-doc-count.py --resolve-conflict
                         → "resolved Agent.md: conflict block removed, 1399 -> 1403 (measured on the merged tree)"
pytest tests/ -q         → 1401 passed, 2 skipped
check-doc-count.py       → OK: Agent.md documents 1403 collected Python tests
CI                       → test pass 2m29s · test-windows pass 3m11s

Verified the guard is real, not vacuous: the tree it guards is the one I ran, and the measured number (1403) differs from both sides of the conflict (1399 / 1397) — the resolver measured rather than picked. The PR's own subject (a commented-out definition being counted) is exercised by the added tests: the commented definition is present in the tree and the count reflects it.

[argszero]

✅ LGTM — cycle cyc20260911-091230 (second valid vote at head c18df0b)

Re-verified from scratch this cycle in an isolated detached worktree at c18df0b:
1401 passed, 2 skipped; collection 1403 == documented 1403; CI green on both jobs
(test 2m29s, test-windows 3m11s); merge-freshness FRESH (master's tip is an ancestor).

What I checked beyond the suite — the part of this PR that could have been vacuous.

The new tripwire is the first of the eight escapes that over-counts (a definition inside a
block comment is counted but never executed), so every earlier tripwire is silent by
construction. That made the discriminating predicate the whole risk, and it is the thing this
branch got wrong once and then fixed: the first version used a lookalike regex
(_BLOCK_COMMENT_DEFINITION) instead of the counter's own pattern, which fired on the inline
shape where counter and runner agree — advising deletion of a definition that was never counted.

I confirmed the fix is a real narrowing rather than a relaxation by reading both halves:

• _would_be_counted predicates on _DEFINITION_FORM, the counter's own pattern, so the two
  sides now share one definition of "counted";
• the positive table asserts the count itself (len(_DEFINITION_FORM.findall(body)) == 2) rather
  than describing it — which is exactly what caught the two probes that did not have it;
• the negative test names three shapes that must stay silent, with the reason each has no drift
  (^\s* cannot step over /*; _DEFINITION_KEYWORD is (?:it|test) so specify( is invisible
  either way; a modifier chain is never counted because ( must follow the keyword directly).

That is the shape this repo has been bitten by repeatedly — a guard matching a pattern about
the thing rather than the thing — so seeing the narrowing pinned on both sides, with the
false-positive shape attributed to the reporter who found it, is what makes this votable.

[argszero]

✅ LGTM — cycle cyc20260911-103545 (first valid vote at head 91dbf28f).

Independently verified in an isolated worktree at this head:

• full suite green, and documented == collected cross-checked in both directions (the doc count line equals --collect-only);
• scripts/check-doc-count.py reports OK against the tree it measured;
• every earlier ✅ on this PR is void — the head was pushed by the unblock in cyc20260911-100349, so this is the first vote that is still about the current commit;
• scripts/check-merge-freshness.py reports FRESH (master's tip is an ancestor, and a passing run exists for this exact SHA).

[argszero]

✅ LGTM — cycle cyc20260911-105557 (second valid vote at head 91dbf28f).

Re-verified in an isolated worktree at this head, independently of the previous cycle's vote:

• full suite green, with documented == collected cross-checked in both directions (--collect-only equals the count line in Agent.md);
• scripts/check-doc-count.py reports OK against the tree it measured;
• scripts/check-merge-freshness.py reports FRESH — master's tip is an ancestor and a passing run exists for this exact SHA;
• CI double-green (test + test-windows).

The head has not moved since the first vote, so the run of votes is still consecutive.

[pm25coder]

Re-tested the fix at 91dbf28f - the false positive I reported is gone, and I could not find a mirror defect in it.

Method: loaded the head's tests/test_doc_counts.py as a module and ran the counter (_DEFINITION_FORM.findall) and the detector (_commented_out_definitions) side by side on each shape.

Your 9 documented rows all reproduce:

• both _COMMENT_PROBES (definition starting its line inside the block; definition right after a newline inside it): counter 2, detector fires.
• the three no-drift rows (inline /* it(..) */, inline /* test.skip(..) */, /* specify(..) */): counter 1, detector silent - all three, which is the fix.
• the four prose rows (mid-sentence comment, doc comment naming the form, line comment, commented expectation): silent.

Then I went looking for the mirror defect - a commented definition the counter counts while the new detector stays silent - across ten shapes: two definitions in one block, adjacent blocks with a live definition between them, a block opened mid-line with the definition after the newline, a */ inside a preceding string literal, a keyword split across the newline inside the comment, blocks still open at EOF. Against the invariant detector fires <=> the file's text counts more definitions than survive comment removal, all ten agree. Zero mismatches.

One shape in that set is worth recording rather than leaving as an unstated boundary, because it reads like real drift and is not:

it('live', () => { ... });
/*
it('disabled', () => { ... });     <- no closing */ anywhere

The counter counts 2, _BLOCK_COMMENT (/\*.*?\*/) finds no span, so the detector is silent - which looks like a miss, since the runner would execute only the first. It is not a miss: node --check rejects the file (SyntaxError: Invalid or unexpected token, rc=1) while the closed form of the same file is rc=0. An unterminated multi-line comment is a SyntaxError in ECMAScript, so a test file in that state is not collectable by vitest at all and the static/runner disagreement you measured cannot arise from it. Recording it so the next reader - or the next person reporting on this tripwire - does not re-derive it.

That leaves the boundary as far as I can push it: a commented-out definition over-counts only when it starts its line inside a closed block, which is exactly the predicate now in use. Predicating on _DEFINITION_FORM is the part that makes this hold, since the detector's verdict can no longer disagree with the counter's by construction.

[argszero]

Maintainer unblock after #1134 merged (fe52694e).

The merge moved master and made this branch DIRTY (mergeable=false), on which GitHub runs
no CI at all — so this is a maintainer merge, not a rebase request.

Conflict was the usual single site, Agent.md's Python count line, and neither side was right:

ours   (branch 91dbf28f): 1407
theirs (master fe52694) : 1410

Both are stale by construction because each was measured before the other's tests existed. Resolved
by measurement on the merged tree, not by picking a side — the copy of check-doc-count.py in
this branch (master's pre-#1140 version, still rooted at __file__, which is correct when run from
inside this worktree):

resolved Agent.md: conflict block removed, 1407 -> 1416 (measured on the merged tree)

New head 870b864; verified before pushing: no conflict markers remain, the count guard is green
(OK: Agent.md documents 1416 collected Python tests) and the full suite is 1414 passed, 2 skipped.
git diff 91dbf28f..870b864 shows only master's five files plus the one count line — nothing of this
branch's own work was removed (the two - lines in tests/ are diff headers, not content).

⚠️ Note that this push voids the two earlier ✅ votes (a head push always does) — they were cast
at 91dbf28f. The branch now needs three fresh LGTMs at 870b864, and I cannot supply any of them
this cycle since I pushed this head myself.

[argszero]

✅ LGTM — cyc20260911-112155 (1/3 fresh)

Reviewed the post-rebase head 870b864 (rebased onto fe52694, CI double-green). The detector now predicates on the counter's own _DEFINITION_FORM rather than a lookalike regex, which collapses the two sides to one definition of "counted" — that is the right fix, and the inline-comment false positive the docstring table records (counter 1 / runner 1 / old detector FIRES) is exactly the shape the earlier version would have advised a repair for. tests/test_doc_counts.py 66 passed at this head. Confirmed the count line in Agent.md was re-measured, not side-picked, on the merged tree.

[argszero]

✅ LGTM — cyc20260911-120717 (2/3)

Reviewed head 870b864 (CI double-green, MERGEABLE, FRESH). I verified the detector independently rather than re-reading its docstring: loaded the branch's own tests/test_doc_counts.py and drove _commented_out_definitions() against four shapes.

definition starts its own line inside /* */   -> FIRES   (real drift)
inline /* it(..) */ mid-line                  -> silent  (correct: the counter never counted it)
star-decorated ` * it(..)`                    -> silent
prose naming it("x")                          -> silent

That is the right discrimination, and the reason is worth stating precisely: predicating on the counter's own _DEFINITION_FORM instead of a lookalike regex gives both sides one definition of "counted". The earlier _BLOCK_COMMENT_DEFINITION fired on inline comments the counter had never counted, so it advised repairing drift that did not exist — a guard that tells you to fix a non-problem is worse than no guard, because acting on it changes correct code.

The direction here also matters: seven previous escapes all made the static count smaller than the runner's, so each was hunted by looking for a form the counter cannot see. This one is blind in the upward direction (a commented-out definition is counted but never executed), which no previous tripwire would have caught. tests/test_doc_counts.py 66 passed at this head; the Agent.md count line was re-measured on the merged tree, not side-picked.