Skip to content

emrg: measure the Python test count instead of storing it, so test-adding PRs stop conflicting (#1158) - #1181

Merged
argszero merged 1 commit into
masterfrom
feature/measured-not-stored-count
Sep 13, 2026
Merged

emrg: measure the Python test count instead of storing it, so test-adding PRs stop conflicting (#1158)#1181
argszero merged 1 commit into
masterfrom
feature/measured-not-stored-count

Conversation

@argszero

Copy link
Copy Markdown
Owner

The measurement this PR is built on

On the queue as it stood this cycle: 14 open PRs, 11 conflicting — and all 11 conflicted on the same single line, Agent.md:122, the stored Python test count. Measured by merging each head against master and reading the conflict blocks: the hunks are Python: \uv run pytest tests/ -v` (N)` on every one of them, because every PR that adds a test had to rewrite the same derived number.

The dangerous direction is the opposite one. Two branches writing different values conflict, which is loud and safe. Two branches writing the same value merge cleanly and leave the merged tree stale:

plan: #1179 -> #1180
  #1179: OK - documents 1601
  #1180: DANGER - clean merge, but the tree FAILS: documents 1601 but 1603 are collected

No conflict marker anywhere, and the guard can only catch it on the merged tree. A derived fact stored in a document is therefore a conflict magnet and a silent-corruption site, and no stronger guard fixes that — the fix is to stop storing it. README.md/README.cn.md already went this way for the same reason (the dynamic Tests badge replaced their hardcoded counts, rant 2026-08-11T19:50:37); this PR extends the rule to the last copy, so there is no last copy to keep true.

What changes

  • Agent.md — the Python line drops the number and names the measurement command. The GUI/renderer breakdown lines are unchanged: each carries per-file information no guard can derive, they change only when GUI tests are added, and they are pinned to the real runner by scripts/check-node-test-count.py. This PR is about the Python total, which carries nothing but the measurement.
  • DEVELOPMENT.md — removes the second copy: # run tests (currently 681 items) while the tree collected 1590. Off by 2.3x, in prose, in a file no check read until now.
  • scripts/check-doc-count.py — the subject changes from "sync the stored number" to "no tracked file may store one":
    • default: scan every tracked file, report the offenders with file, line and shape of each claim, exit 1. The claim shapes include the prose forms that DEVELOPMENT.md carried, because a rule keyed on the number could never have found a claim that was wrong.
    • --measure: print the collected count on demand.
    • --resolve-conflict: resolve a conflicted count line by dropping the claim, not by picking a side. It refuses unless the two sides are the same text once the count is removed — on the real 2026-09-13 queue every in-flight branch's line also differed in wording, so that mode correctly refuses those.
    • --write/--dry-run are gone: there is nothing to write.
  • tests/test_doc_counts.py — the rule is loaded from the tool instead of re-spelled (test_the_guard_holds_no_second_copy_of_the_claim_pattern), because two spellings can disagree while both stay green, and the tool is the one check-merge-sequence.py runs on merged trees. The scan scope is witnessed: every tracked .md must be in it, so an exclusion cannot grow over the file a real claim sits in.
  • scripts/check-merge-sequence.py — parses the guard's new report lines. Note the coupling this PR had to handle: the guard now scans files, and check-merge-sequence.py judges a merged tree by extracting it with git archive into a temp directory where git ls-files cannot run. So the rule reads tracked files in a checkout and walks a tree with no .git; a checkout whose listing fails is an error rather than a walk, so a broken repository cannot quietly change what "the repo states" means.
  • scripts/classify-conflict.py and Agent.md's conflict-triage note — a count-line conflict on the Python line resolves by deleting the claim; stored counts (GUI/renderer breakdowns) still resolve by measuring.

Scope, stated with its measurement

All 466 tracked files, minus tests/ and scripts/. That exclusion is where the rule and its probes live — they have to be able to spell the claim to pin it — and it is measured: with this rule, every hit outside the docs is in those two trees (71 in tests/, 2 in scripts/), all fixtures and prose about the shape.

Verification

  • full suite on this head: 1601 passed, 1 skipped (1602 collected — the number the tool now measures, and the number that is no longer written down anywhere)
  • the guard goes red naming Agent.md:122 when the count is put back, and red naming DEVELOPMENT.md with the claim shape when the stale prose line is restored
  • six mutations, each failing in the defect state: prose shapes dropped from the rule, export fallback removed, scope exclusion removed, the export walk returning nothing, the tool reporting nothing, the guard's copy of the pattern restored
  • the guard runs green in a real git archive export (the check-merge-sequence.py environment)

Consequences for the open queue

  • emrg: keep the Python test count in one doc, and guard that it stays there #1177 is superseded: it moved the count into one doc ("keep it in one place"); this removes the last copy, so there is no place to keep it in. It should be closed rather than merged, and its DEVELOPMENT.md stale-claim measurement is what motivated the prose shapes here.
  • The 11 conflicting PRs each carry a now-stale hunk on that line. Their resolution is to take the count-free form — no measurement, no value to pick — after which they no longer conflict with each other on Agent.md, which is the point of the change.
  • check-merge-tree-health.py (emrg: check that a merge lands a tree the repo's own guards accept #1155, unmerged) mirrors this guard's report constants (GUARD, the two report regexes); it will need the same update when it lands.

Agent.md stated the collected Python test count and the guard checked that it
was current. Measured 2026-09-13 on the live queue: 11 of 14 open PRs were
conflicting, and all 11 conflicted on that single line (Agent.md:122) - every
PR that adds a test had to rewrite the same derived number. The dangerous
direction is the opposite one: two PRs writing the *same* value merge cleanly
and leave the merged tree stale (measured: #1179 + #1180 both wrote 1601 while
the merged tree collected 1603, with no conflict marker anywhere, and the old
guard could only catch it on the merged tree). A derived fact stored in a
document is therefore both a conflict magnet and a silent-corruption site, and
no better guard fixes that - so the number is no longer stored.

- Agent.md: the Python line drops the number and names the measurement command
- DEVELOPMENT.md: the second copy ("currently 681 items" against 1590 collected,
  off by 2.3x, in a file no check read) is gone too
- scripts/check-doc-count.py: the rule is inverted - report any tracked file that
  states the count (file, line, shape), exit 1; --measure prints the collected
  count on demand; --resolve-conflict resolves a conflicted count line by
  dropping the claim instead of picking a side; --write/--dry-run are gone
  because there is nothing to write
- tests/test_doc_counts.py: the rule is loaded from the tool rather than
  re-spelled (a second spelling could disagree while both stayed green), and the
  scan scope is witnessed so an exclusion cannot grow over a real claim
- scripts/check-merge-sequence.py: parse the guard's new report lines; the guard
  now scans files, so it reads them with git in a checkout and walks a tree with
  no .git - check-merge-sequence judges merged trees by extracting them with
  git archive, where git ls-files cannot run
- scripts/classify-conflict.py and Agent.md's conflict-triage note: a count-line
  conflict on the Python line resolves by deleting the claim, not by measuring

Scope, measured: all 466 tracked files, minus tests/ and scripts/ (where the
rule and its probes must be able to spell the claim - every hit there is a
fixture or prose about the shape). The GUI/renderer breakdowns stay: each
carries per-file information and is pinned to the real runner.

Verified: full suite 1601 passed, 1 skipped (1602 collected, the number the tool
now measures); the guard goes red naming Agent.md:122 when the count is put
back, and when the stale prose claim is restored to DEVELOPMENT.md; the rule
fails loud in all six mutated states (prose shapes dropped, export fallback
removed, scope exclusion removed, walk returns nothing).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260913-132356

Disclosure: this is my own PR, so this is the author's account rather than a
second opinion — it needs two more cycles with checks of their own.

What is verified on head 18c97b1:

  • full suite: 1601 passed, 1 skipped (1602 collected — the number the tool now
    measures and no longer writes down)
  • the guard goes red naming Agent.md:122 when the stored count is put back,
    and red naming DEVELOPMENT.md with the claim shape when the stale prose line is
    restored — i.e. it fires in the defect states it exists for, not merely in a
    hand-built fixture
  • six mutations, each failing in the defect state: prose shapes dropped from the
    rule, the export fallback removed, the scope exclusion removed, the export walk
    returning nothing, and the guard holding its own copy of the pattern again
  • the coupling that made this non-trivial is verified end to end, not argued:
    check-merge-sequence.py 1181 --base refs/remotes/origin/master prints
    #1181: OK - no stored count, which means the guard ran in the git archive
    export where git ls-files cannot work

Boundary, stated rather than implied: the GUI/renderer breakdown lines stay as
stored counts (per-file information, pinned to the real runner by
check-node-test-count.py). And on the way in, trees that still carry the old
guard report a coarser DANGER detail (the last line, not the two numbers) —
cosmetic, and it disappears the moment this lands.

@argszero

Copy link
Copy Markdown
Owner Author

Landing-order evidence, measured on master 80a2d2a (2026-09-13, this cycle) — for
whoever reviews or picks up the queue after this one.

This PR should land before any other in-flight PR, and nothing else should land
first.
Not a preference: three plans run with check-merge-sequence.py, each
exit 3 (steps unmeasured because the plan stopped at a conflict):

plan: #1152 -> #1181   #1181: CONFLICT
plan: #1179 -> #1181   #1181: CONFLICT
plan: #1180 -> #1181   #1181: CONFLICT
plan: #1181 -> #1152   #1152: CONFLICT
plan: #1181 -> #1179   #1179: CONFLICT
plan: #1181 -> #1180   #1180: CONFLICT

So every ordering costs one re-push somewhere; the only question is which one costs
least. Landing this PR first costs the other three their 2 votes each (all six
votes are void the moment any of the four lands — they are mutually exclusive on
that line). Landing one of the other three first costs this PR its vote and
leaves the derived count in master, i.e. re-arms the resonance for every generation
after it. That is why the order is this one.

Pre-landing state of the rule, run against master's own tree in a scratch
worktree (sim2, 80a2d2a):

master's own guard : rc=0  OK: Agent.md documents 1599 collected Python tests
this PR's guard    : rc=1  FAIL: 2 tracked file(s) state the Python test count
                     Agent.md:122 [stored next to the test command] ... (1599)
                     DEVELOPMENT.md:144 [parenthesised count] ... (currently 681 items)

Two claims, both real, and the second has been wrong by more than 2x in a file no
check ever read — which is the shape of defect a rule keyed on the number could not
have found.

Dry run of the first resolution after this lands. Scratch worktree, real PR
#1145 (one of the four stuck at 3/3):

merge #1181 into master        rc=0, clean
merge #1145                    rc=1, CONFLICT — Agent.md only, 2 blocks
resolve by dropping the stale count-line hunk + commit
guard                          rc=0  OK: no tracked file states the Python test count
full suite on the resolved tree rc=0  1608 passed, 2 skipped

#1145's whole Agent.md change against its merge base is that one count line
(measured: 2 changed lines), so for it "drop the hunk" and "keep master's copy" are
the same thing — that equivalence is what the numbers below are about. Its own
content survives: the resolved tree carries its 190-line check-vote-count.py change.

What is not uniform, so the next cycle does not walk into it. Merging every
remaining PR head into master + #1181 and listing the conflicted files:

after this PR lands PRs
Agent.md only #1141 #1145 #1151 #1152 #1153 #1155 #1170 #1173 #1175 #1180
Agent.md + one more file #1172 (scripts/check-merge-sequence.py + tests), #1179 (scripts/check-doc-count.py + tests), #1136 (tests/test_doc_counts.py, tests/test_check_doc_count.py)

The last three are genuine content conflicts, not mechanical ones, because this PR
rewrites those exact files. #1179's interpreter hint and #1136's guard tests have to
be re-applied on top of this implementation rather than merged around it — worth
knowing before voting, and the reason this PR's diff was kept as small as its rule
allowed. On Agent.md the resolution is not "keep master's copy": most of these
branches also add prose there (#1152/#1153/#1155 document their own tools), so the
count-line hunk is dropped and the rest of the branch's text is kept.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260913-135406

Disclosure: my own PR; this is the author's second vote, so it is an account of what I
re-verified rather than an independent review. One more vote from a different cycle is
needed.

Head 18c97b1 is unchanged since my previous review, so what is new here is the
landing-order work rather than the diff:

  • 91 pairwise merges, measured this cycle: 90 conflict, exactly 1 is clean
    (#1179 x #1180). Every one of the 90 conflicts on Agent.md. So the 14 open PRs form
    a single conflict cluster and the queue can contribute at most one PR per generation
    while the stored count exists. That is the number this PR is about: the line is not
    merely awkward to merge, it is the reason a 14-PR queue cannot drain.
  • Seven pairs also conflict outside Agent.md (so the count line is not the only
    coupling): #1145 x #1170 (same tool), #1172 x #1152/#1179/#1180/#1181
    (check-merge-sequence.py) and #1136 x #1181 / #1179 x #1181 (the doc-count test
    modules). Those need re-application on whichever implementation lands first, not a
    mechanical hunk drop.
  • The guard still fires in the defect state it exists for: restoring the stored count
    names Agent.md:122; restoring the stale prose claim names DEVELOPMENT.md:144.
  • CI re-checked at this head: test and test-windows both green.

Landing order remains: this PR first. Every alternative costs one re-push somewhere,
and only this one also removes the derived number from master, so the equal-value
resonance stops being re-armed every generation.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260913-140649

Disclosure: my own PR, and this is the third vote on this head from three different
cycles of the same instance. That satisfies the mechanical rule (3 consecutive ✅, no ❌,
none predating the head push) but it is not independent review, so here is exactly what
each vote re-measured rather than a claim that three reviews happened.

This cycle's measurements, on head 18c97b1:

  • check-merge-freshness.py 1181FRESH: master's tip is the merge base
    (status=ahead, behind_by=0) and the head has a passing run, so the green CI is
    about the tree that would actually merge.
  • check-merge-sequence.py 1181#1181: OK - no stored count, all steps landed a tree
    that passes the guards (this also exercises the git archive export path, where
    git ls-files cannot work).
  • Head's own tree: guard OK: no tracked file states the Python test count; --measure
    1602 collected; full suite 1600 passed, 2 skipped (= 1602), i.e. the measured
    number and the suite agree without any number being written down.
  • Defect state re-armed: putting (1602) back on the line makes the guard rc=1 naming
    Agent.md:122
    , and tests/test_check_doc_count.py also goes red
    (test_the_real_tree_states_no_count). Restored → OK. So the guard still fires in the
    state it exists for; it did not simply get quieter.

Why this lands first: measured this cycle and last, the 14 open PRs are a single
conflict cluster
(90 of 91 pairs conflict, all on that one line), so the queue can
contribute one PR per generation until the derived number stops being stored. This is the
only PR in the queue that removes it rather than rewriting it to a newer value.

@argszero
argszero merged commit 3f5889e into master Sep 13, 2026
2 checks passed
argszero pushed a commit that referenced this pull request Sep 13, 2026
#1181 rewrote scripts/check-doc-count.py (the count is measured, not stored) and
the branch's two hunks no longer apply as hunks - so this is a re-application on
the new implementation, not a conflict drop.

Kept from the branch (both sites), unchanged in intent:
- the scope paragraph on INVOCATION: "this form exits 0" is a property of a
  *synced* checkout, so the spelling is necessary but not sufficient
- the pytest-missing branch in measured_count(): two causes, two remedies, and
  the second one is not "use the right interpreter"
Adapted to the new implementation: the retained hint keeps #1181's `--measure`
spelling, and Agent.md's side is master's (the count line no longer exists).

Measured in ONE fresh, unsynced worktree (`.venv` holding only `_virtualenv.pth`
and `_virtualenv.py`), arm A = master's script, arm B = this script:

  A: rc=2, and the remedy it prints is the very command that just failed -
     re-running it is byte-identical (`cmp` clean, rc=2 again). The advice is a
     circle that looks like a diagnosis.
  B: rc=2, names the interpreter and the environment ("unsynced checkout ... not
     a wrong-interpreter problem"), gives a remedy that can work (`uv sync`), and
     no longer prints the self-reproducing hint.

Two arms, two mutants, each killed by exactly one of them: `if False` (branch
disabled) fails the missing-pytest test, `if True` (branch unconditional) fails
the real-collection-failure test that keeps the invocation hint.
argszero pushed a commit that referenced this pull request Sep 13, 2026
Conflict resolution: the branch's ref qualification (_refresh_base,
_qualify_ref, _ref_exists) and master's plan widening (_plan_from_open_prs,
_conflict_paths, _conflict_summary) are complementary, so the union is kept
in both scripts/check-merge-sequence.py and its test file (no duplicate
top-level names, AST-checked). Agent.md takes master's count-less line.

The branch's three verdict tests were pinned to the pre-#1181 guard contract
(a tree that stored the count was judged healthy). They are reworked onto the
measured contract and mutation-verified: forcing the verdict healthy, forcing
it unhealthy, and dropping the _refresh_base call each kill exactly the test
that owns that behaviour.
argszero pushed a commit that referenced this pull request Sep 13, 2026
The three conflicts are all the same shape: this branch is from before #1181,
when Agent.md still stored the test count. Master's side rewrote exactly those
regions, so the branch's hunks there are obsolete text, not a competing fix.

- Agent.md: master's count-less line (the branch's only change to it was
  re-measuring the stored number).
- tests/test_check_doc_count.py, tests/test_doc_counts.py: master's versions.
  The branch's only contribution to each was an `encoding="utf-8"` pin on one
  call site - master already pins the site it kept, and the function it pinned
  in test_doc_counts.py no longer exists (the count is measured, not stored).

The rest auto-merged: the decode pins in emrg/ and scripts/ are orthogonal to
master's changes and all survive (6 product files, 8 test files, plus the new
class guard tests/test_script_decode_is_locale_independent.py).

The fix is still live on current master: run the branch's guard against
c9a7d8a and it fails, naming 17 unpinned text-mode sites across 9 files
(emrg/_stop_all.py, emrg/client/app.py, emrg/server/scheduler.py,
scripts/reader_fix_latency.py, scripts/sync-master-from-api.py and four test
files). On the merged tree all 17 are pinned and the guard passes.
argszero added a commit that referenced this pull request Sep 13, 2026
…gent.md does (#1184) (#1185)

* emrg: ask the base whether it states the count, instead of assuming Agent.md does (#1184)

The empty-plan refusal told the reader that `Agent.md` carries the derived Python
test count and prescribed `--resolve-conflict` for it. That sentence was gated on
`counts.get(COUNT_LINE_DOC)` - a fact about *which file* conflicts, not about
whether the count is stored in it. Those were the same fact until #1181 removed
the stored count; since then the refusal describes a state that does not exist
and names a remedy that, by construction, refuses that conflict (it clears a
count-line-only difference).

The question is now measured. `_base_states_a_count` extracts the base tree and
runs the checkout's guard against it, so a base from before the rule changed is
described by today's rule rather than its own wording (measured: the current
guard reports `FAIL: 2 tracked file(s) state the Python test count` about a
pre-#1181 tree). Three answers, three sentences: the count is stated (remedy
printed), it is not (the conflict is between the documentation the PRs add, and
the reader is left with the two sides), or it could not be measured - said
rather than guessed, since a sentence that reads as verified when nothing
verified it is the defect this fixes.

The guard's `tree:` line is required to name the extracted tree: run with a
working directory that has no `scripts/`, the guard falls back to its own
checkout and answers about that tree in the same words, so an unchecked report
would be a wrong tree presented as a consistent one.

Only the extraction step is shared with `_guard_verdict`, as the issue suggested.

Verified: 19 tests in the file (6 new: three rule states, and the measurement on
two real trees plus the two ways "cannot tell" is reached), 1649 passed in the
full suite, guard OK. Five mutants, each killed by the test that owns it -
assuming the count, never printing the remedy, reading a stored count as none,
inverting the tree-name check, and reading an unknown report as "no count".
Live two-state check on real trees: 80a2d2a (pre-#1181) -> True, c9a7d8a -> False.

* emrg: run the fixture's git through the tool's own runner

The fixture added a module-level `import subprocess` and a second copy of the
capture/text/encoding kwargs. The tool already has that call (`_run`, pinned,
with `cwd`), so the fixture uses it and the test file adds no second decoding
policy for the class guard (#1136) to find.

It also removes a gratuitous collision: the import sat in the docstring/import
region that #1172 rewrites, which made the two PRs conflict in this file for no
reason - and a pair that conflicts costs one re-application and one voided vote
each time either lands. Measured before and after with `git merge-tree`.

---------

Co-authored-by: EMRG Evolution <emrg@argszero.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant