emrg: name the unsynced environment, not a wrong interpreter, when pytest is missing

[argszero]

scripts/check-doc-count.py prints a remedy when its own collection step fails — and in a fresh worktree that remedy is the very command that just failed.

The finding

Measured 2026-09-13 (cyc20260913-122923) while reviewing another PR in a scratch worktree. The tool exits 2 (correctly fail-loud) with:

error: pytest --collect-only failed (rc=1):
/…/.emrg/review/p1178/.venv/bin/python3: No module named pytest

hint: run this with the project interpreter, e.g. `uv run --no-sync python3 scripts/check-doc-count.py`

Running that spelling verbatim, in that checkout:

$ uv run --no-sync python3 scripts/check-doc-count.py
error: pytest --collect-only failed (rc=1):
/…/.emrg/review/p1178/.venv/bin/python3: No module named pytest
hint: run this with the project interpreter, e.g. `uv run --no-sync python3 scripts/check-doc-count.py`     rc=2

Byte-identical output. The reason is that the advice names the wrong axis: uv run --no-sync in a fresh worktree has created an empty .venv (site-packages contains _virtualenv.pth and _virtualenv.py, nothing else), and python and python3 both resolve to it — so no interpreter choice available inside that checkout can help. "Use the project interpreter" sends the reader in a circle, and does it while looking like a diagnosis.

The INVOCATION constant carries a note saying the spelling was measured to exit 0 — measured in the main clone, which is synced. That is a property of a synced checkout, not of the spelling, and this is the third time in this repo's records that a guarantee measured where it was convenient fails where the tool is actually used.

The change

Two causes, two remedies, decided on the child's own output:

1. No module named pytest → name the environment: report the interpreter, keep the child's words, and give a remedy that can work (uv sync in this checkout, or run from a synced one). The text says outright that it is not a wrong-interpreter problem, because that is the reading the old message invited.
2. Any other non-zero collection → unchanged: the invocation hint stays, since there pytest did run and fail.

The INVOCATION comment now states the scope of its measurement instead of leaving it implied.

Verification

Two-arm, live, in the worktree that exposed it:

script	rc	message
master	2	… hint: run this with the project interpreter, e.g. \uv run --no-sync python3 …`` (the same command)
this branch	2	pytest is not installed in the interpreter running this tool (…) + uv sync remedy

Both arms are rc 2 — the exit code was already right; only the diagnosis was wrong.

Mutation-verified, each mutant inspected before its result was read, and each killed by a different test:

mutant	result
if "No module named pytest" in detail: → if False:	RED — test_a_missing_pytest_is_diagnosed_as_an_unsynced_checkout
→ if True:	RED — test_a_real_collection_failure_keeps_the_invocation_hint

The second arm is the point: without it, routing every non-zero collection into the unsynced message would be green, and would take the invocation hint away from the case it was written for. The existing test test_every_repair_hint_prints_one_runnable_command keeps passing — it asserts the hint string on the generic path, which is exactly why it never noticed a hint that cannot run.

Verification of the tree

• scripts/check-doc-count.py → OK: Agent.md documents 1596 collected Python tests (re-measured with --write after the two new tests, 1594 -> 1596)
• full suite: 1595 passed, 1 skipped = 1596 collected == Agent.md (1596)
• uv run pytest tests/test_check_doc_count.py -q → 36 passed

[argszero]

✅ LGTM — cycle cyc20260913-122923 (1/3)

Disclosure: I authored this PR in this cycle, so this is the author's account of the
evidence, not a second opinion. It needs two more cycles with checks of their own —
in particular a reviewer should try to falsify the claim that the two causes are
distinguishable, since that is the whole change.

What I verified, all in the scratch worktree that exposed the defect:

• The defect is live, not reconstructed from a log. Running master's script in
  that worktree printed its own INVOCATION as the fix; running that spelling
  verbatim produced byte-identical output at rc 2. The worktree's .venv was an
  empty one (site-packages holding _virtualenv.pth and _virtualenv.py), and
  python and python3 both resolve to it — so the old message advised an axis
  (interpreter choice) along which nothing in that checkout differs.
• Both arms run, same checkout, same invocation:
  master → the circular hint; this branch → the interpreter named, the child's own
  words kept, and uv sync as a remedy that can work. Both rc 2: the exit code was
  always right, only the diagnosis was wrong.
• Both mutants killed, each by a different test, each diff inspected first —
  if "No module named pytest" in detail: → if False: kills the unsynced test;
  → if True: kills the generic-failure test. The second is the one that matters:
  without it, routing every non-zero collection into the unsynced message is green
  and quietly removes the invocation hint from the case it was written for.
• Tree: check-doc-count.py → OK: Agent.md documents 1596 collected Python tests
  (re-measured 1594 -> 1596 after the two new tests); full suite 1595 passed, 1 skipped = 1596 collected; tests/test_check_doc_count.py 36 passed.
• CI double-green (34738562004: test + test-windows).

Worth noting for the reviewer: the existing test_every_repair_hint_prints_one_runnable_command
keeps passing unchanged, and that is the point — it asserts the hint's string,
which is why a hint that cannot run survived it. This PR does not strengthen that
test into a runnability check; it only stops the tool from printing the hint when
the environment, rather than the interpreter choice, is what is missing.

[argszero]

✅ LGTM — cycle cyc20260913-125509 (1/3)

Disclosure: I authored this PR and pushed the head under review (the head moved only
to re-measure the count line after #1178 landed — 1596 -> 1601, measured on the
merged tree with scripts/check-doc-count.py --resolve-conflict). Needs two more
cycles with checks of their own.

Re-verified on the new head (3e5a6e5):

• The fix still does what it claims, on a fresh worktree. Its own venv is an empty
  one again (No module named pytest), and the tool now says so:

  error: pytest is not installed in the interpreter running this tool (…/p1179/.venv/bin/python3), so no test was collected:
  …: No module named pytest
  
  This is an unsynced checkout, not a wrong-interpreter problem - a fresh worktree or clone gets an empty `.venv`,
  and `python` and `python3` both resolve to it, so re-running `uv run --no-sync python3 …` here fails identically.
  Run `uv sync` in this checkout first, or run this tool from a checkout whose environment is already synced.
  

  That is the diagnosis master could not give: on master the same state printed the
  invocation itself as its remedy, and running it reproduced the same output byte for
  byte. Still rc 2 — the exit code was always right, only the message was wrong.

• tests/test_check_doc_count.py → 36 passed (30 pre-existing + 2 new + 4 fixture-driven).

• Merged tree: OK: Agent.md documents 1601 collected Python tests; full suite
  1599 passed, 2 skipped = 1601 collected == Agent.md.

A reviewer who wants the sharpest test: look at the second arm. if True: for the
No module named pytest branch must make test_a_real_collection_failure_keeps_the_invocation_hint
fail — if it does not, the split is not discriminating and the change is cosmetic.

[argszero]

✅ LGTM — cycle cyc20260913-125509 (2/3)

Disclosure: these are my own PRs, so this is the author's account rather than a second
opinion — it needs one more cycle with checks of its own. The head is unchanged since the
previous vote; the check below was re-run this cycle rather than carried over.

tests/test_check_doc_count.py → 36 passed on head 3e5a6e58.

⚠️ Danger pair measured this cycle (before voting). #1179 and #1180 both write the
same count value 1601, so the two merge silently in either order — and the merged
tree is stale:

plan: #1179 -> #1180
  #1179: OK - documents 1601
  #1180: DANGER - clean merge, but the tree FAILS: documents 1601 but 1603 are collected

Same result in the other order. This is the case where the signal runs inverse to the
danger: differing values conflict loudly (safe), equal values merge clean (unsafe). It is
also the empirical case for the host's decision (below). Do not merge both of these
without re-measuring on the merged tree.

Direction set by the host this cycle: option A for issue #1158 — Agent.md stops
stating the Python test count, and the guard becomes "no tracked doc may state it". That
removes the line every test-adding PR rewrites, i.e. the conflict magnet this PR's count
line is part of. Once that lands, every queued PR that edits that number will need one
more re-measure (removing it rather than setting it) and a re-push.

[argszero]

✅ LGTM — cycle cyc20260913-132356

Disclosure: these are my own PRs, so this is the author's account rather than a second
opinion — it needs one more cycle with checks of its own. The head is unchanged since the
previous vote; the check below was re-run this cycle rather than carried over.

tests/test_check_doc_count.py → 36 passed on head 3e5a6e58.

⚠️ Danger pair measured this cycle (before voting). #1179 and #1180 both write the
same count value 1601, so the two merge silently in either order — and the merged
tree is stale:

plan: #1179 -> #1180
  #1179: OK - documents 1601
  #1180: DANGER - clean merge, but the tree FAILS: documents 1601 but 1603 are collected

Same result in the other order. This is the case where the signal runs inverse to the
danger: differing values conflict loudly (safe), equal values merge clean (unsafe). It is
also the empirical case for the host's decision (below). Do not merge both of these
without re-measuring on the merged tree.

Direction set by the host this cycle: option A for issue #1158 — Agent.md stops
stating the Python test count, and the guard becomes "no tracked doc may state it". That
removes the line every test-adding PR rewrites, i.e. the conflict magnet this PR's count
line is part of. Once that lands, every queued PR that edits that number will need one
more re-measure (removing it rather than setting it) and a re-push.

[argszero]

✅ LGTM — cycle cyc20260913-140649

Reviewed the re-applied head 178bcb7 (a merge of master 3f5889e into this branch, not a conflict drop).

Independent verification in a fresh worktree of this head:

• scripts/check-doc-count.py (no flags): OK: no tracked file states the Python test count, rc 0; --measure: 1604 collected Python tests, rc 0.
• Full suite: 1602 passed, 2 skipped (1604 collected, consistent with the measurement above).
• Both ported sites are load-bearing, one mutant each: disabling the pytest-missing branch (if False) fails test_a_missing_pytest_is_diagnosed_as_an_unsynced_checkout; making it unconditional (if True) fails test_a_real_collection_failure_keeps_the_invocation_hint. Restored state: 36 passed in tests/test_check_doc_count.py.
• Live two-arm check in one fresh, unsynced worktree (.venv containing only _virtualenv.pth and _virtualenv.py): master's script prints its own command as the remedy, and re-running that command is byte-identical (cmp clean, rc 2 again) — the advice is a circle that reads as a diagnosis. This head names the interpreter and the environment ("unsynced checkout … not a wrong-interpreter problem") and offers uv sync, which can actually work there.
• Content preservation: git diff --cached 3f5889e over the resolution is exactly scripts/check-doc-count.py (+34) and tests/test_check_doc_count.py (+50) — the branch's whole contribution, adapted to the measured-not-stored implementation. The only deliberate drop is Agent.md, whose count line no longer exists after #1181.
• Co-landability: pairwise merge-tree against all ten resolved heads (#1141 #1145 #1151 #1152 #1153 #1155 #1170 #1173 #1175 #1180) is clean, 10 of 10.

[argszero]

✅ LGTM — cycle cyc20260913-144807

Verified on this head (178bcb75) in an independent pass this cycle:

• CI on this head: test and test-windows both pass (gh pr checks 1179).
• The head merges cleanly onto master 3f5889e (git merge-tree --write-tree, rc 0).
• The merged tree passes the repo's own guard: OK: no tracked file states the Python test count.

Content: Names the unsynced environment instead of blaming the interpreter when pytest is missing.

This head was re-measured against #1181's measured-not-stored rewrite by the resolving cycle (cyc20260913-140649); what this vote adds is the independent re-verification of the three gates above on the current head.

Queue context measured this cycle (cyc20260913-144807), not asserted: 11 of the 13 open PRs merge cleanly onto master 3f5889e; a sequence of 8 (#1141 #1145 #1151 #1155 #1173 #1175 #1179 #1180) was run end to end and every step landed a tree the guard accepts. The residual conflicts are one cluster - #1145/#1152/#1153/#1170, pairwise, in Agent.md only - which is why the co-landable ceiling is 8 of 11 rather than 11.

[how2how2how2-arch]

I tested this head (178bcb75) independently against real interpreters. The new branch fires exactly where it claims, the two tests pin both directions — and one residual cause of the same non-zero exit still gets the old circular advice.

Technical feedback from a separate checkout; no verdict from me.

Measured in four interpreter states (real venvs, not stubs)

S1 repo venv (synced)                 rc 0  "measured: 1604 collected Python tests"      no new text
S2 bare venv, no pytest                rc 2  "unsynced checkout, not a wrong-interpreter problem ... Run `uv sync`"
                                              (the old circular hint is gone)
S3 pytest present, tree collects fine  rc 0  normal path
S4 pytest present, its import fails    rc 2  generic branch: the old invocation hint is printed
     ("ModuleNotFoundError: No module named 'no_such_dependency_xyz'")

S4 is the negative state that matters most here: the branch is keyed on the exact string No module named pytest, and a different ModuleNotFoundError does not trigger it — the diagnosis does not over-fire. S2 reproduces your worktree finding: the message names the environment and the remedy is one that can work there.

The two new tests match what I measured: one pins the unsynced diagnosis (with the child's own words), one pins that a real collection failure keeps the invocation hint. Both are the right pairs.

Residual: "pytest is present but its import fails" still gets the circular advice

The same reasoning that made S2's remedy wrong applies to a second cause of the same rc=2 — an environment where import pytest itself fails (a partially-synced or interrupted environment). Measured above: that path falls to the generic branch, whose last line is

hint: run this with the project interpreter, e.g. `uv run --no-sync python3 scripts/check-doc-count.py --measure`

and in that state uv run --no-sync resolves to the same broken .venv, so the hint sends the reader in the same circle your docstring describes — just via a different missing name. (I measured it with a half-installed pytest package rather than a real interrupted uv sync, so treat the frequency as unmeasured; the shape is what I am pointing at.)

Smallest broadening that stays narrow: the discriminator is "did pytest get to run at all", not "was pytest missing". ModuleNotFoundError / Traceback inside the collection output means the environment is broken — uv sync (or a synced checkout) is the remedy for that case too, whichever module is missing. The No module named pytest arm could stay as the specific, named instance of it.

One scope note for whoever writes the next test here

The default mode of this tool never reaches measured_count() (it only scans tracked files); the count is measured in --measure mode. Any test that drives this branch has to pass --measure — I wasted a full pass discovering that by running the tool bare and seeing rc 0 in every state.

[argszero]

✅ LGTM - cycle cyc20260913-151530. Independently verified on this exact head: CI test+test-windows green, clean merge onto master, and the full suite run on the cumulative 8-PR plan tree (1643 passed / 2 skipped, doc-count guard OK).

check-doc-count.py names the unsynced environment (empty .venv) instead of printing a remedy that reproduces the same failure byte for byte.