emrg: extend the locale-decode guard into the emrg/ package and fix the path readers it found - #1136
emrg: extend the locale-decode guard into the emrg/ package and fix the path readers it found#1136argszero wants to merge 14 commits into
Conversation
…ecode independent of locale (#1132)
…les (CI runs pytest before npm ci)
…indows leaves None)
The previous cycle's guard read scripts/ only, and justified skipping emrg/ by citing two files (git_utils.py pins encoding; bash_tool.py has a deliberate locale-then-UTF-8 console policy). That was a claim about the package drawn from two of its files, and the cycle after it measured the claim false: - emrg/client/app.py's clipboard readers asked osascript/xclip/powershell for a file *path* and decoded it with the locale codec. Under GBK a clipboard label of 图片.png came back as U+9365 U+5267 (mojibake), and the caller's bare `except Exception` turned the failure into "no image on the clipboard" - silent. - emrg/_stop_all.py's two `ps` readers are the same class; an undecodable path byte raised UnicodeDecodeError straight past `_ps_output`'s `except (OSError, subprocess.SubprocessError, TimeoutError)`. Fix: pin encoding="utf-8", errors="replace" on those path-bearing reads (6 sites in app.py, 2 in _stop_all.py). Paths are filesystem bytes, not console bytes. The guard now covers scripts/ and emrg/, and draws its exemption on the axis that actually decides the encoding - the child *program*, not the file: powershell / taskkill / tasklist write the Windows console code page, while git / gh / ps / node write UTF-8, and both kinds live in emrg/_stop_all.py. Two further corrections found by running it: - the first emrg/ version reported every `**expr` splat as unreadable (27 false positives - `**win32_no_window_kwargs()` and `**_no_window()` can only supply creationflags, and `**_PATH_DECODE` *is* the pin). Splats, locally aliased kwarg dicts and concatenated argv are now resolved; only what cannot be resolved is reported. - `rglob` was picking up 12 vendored .py files under emrg/gui/node_modules, so the scan's reach depended on whether `npm install` had run - green here, different in CI, which is the asymmetry this module exists to prevent. The scan set is now asserted equal to the tracked first-party set (69 files). emrg/server/scheduler.py's `git status --porcelain` reader is pinned too: the default core.quotePath=true escapes non-ASCII to ASCII octal, which hid it, but a host with quotePath=false gets raw UTF-8 and an unpinned decode mojibakes it (measured: 0x9365/0x5267/0x5896 instead of 0x56fe/0x7247). Verified: 1354 passed / 1 skipped; both smoke checks; node-count gate OK; 7 mutation controls (each rule fails when its own subject is broken, all restores green). Agent.md count re-measured with scripts/check-doc-count.py --write (1349 -> 1355).
The scan-coverage test asserted that emrg/gui/node_modules exists, to prove its exclusion was still doing something. That tree only exists where someone ran `npm install`, so the test passed locally and failed on both CI jobs - the "works where I ran it" asymmetry this module exists to catch, written into the module by the cycle that was fixing it. The exclusion is now exercised on a synthetic temp tree instead: one vendored file and one first-party file, asserting the filter drops exactly the first. Verified by moving emrg/gui/node_modules out of the tree and re-running the file: 12 passed, same as with it present.
A follow-up on this branch's own guard, driven by measuring it rather than reading it. Three findings, all from the same axis: the rule could not see the code that its own claim rested on. 1. The scan test was circular, and its own probe was one of the sites it missed. The previous head built the expected file set with `git ls-files emrg scripts` and compared it to `_scan_roots()`, which globs exactly those two directories - both sides derived from the same root list, so `tests/` was never in either. 70 tracked first-party Python files were invisible, and the first version of the guard's own locale probe lives in one of them. Widening the scan to read `emrg/` alone *and relaxing the assertion to match* left all 12 tests green: a self-consistent narrowing, which is the exact failure this module exists to catch, written by the cycle that was fixing it. Replaced with assertions that cannot be satisfied by narrow-and-relax: the scan equals `git ls-files '*.py'` minus vendored trees (an *index* derivation, independent of any directory list in the file), and every top-level directory holding tracked Python must contribute at least one scanned file. 2. Nine text-mode calls were unpinned in those unseen files, all with `text=True` and no `encoding=`. Eight are pinned here (the ninth is the guard's own probe and went with the rewrite). Each carries a reason: `git ls-files` prints paths (`~/项目/emrg` is valid UTF-8 and raises under GBK), `pytest --collect-only` prints test ids, and the two `check-doc-count`/`llm-cost-report` call sites read a child's output through `json.loads`. 3. Three text-mode entry points carry no `text=` marker at all, so no amount of kwarg inspection reaches them: `subprocess.getoutput`, `subprocess.getstatusoutput` and `os.popen`. Measured on a cp936 host, a child emitting UTF-8 U+2014 (the em dash, present in 95 of the first 100 closed issues of this repository, so ordinary prose rather than an exotic input) raises `UnicodeDecodeError: 'gbk' ... byte 0xad` out of both `get*output` calls. The whole probe file yielded one violation before this change. `getoutput`/`getstatusoutput` accept `encoding=`/`errors=` (3.10+), so they join the existing rule; `os.popen` takes no encoding at all (measured: `TypeError`), so it is reported with the opposite advice - the replacement to use - rather than told to add a keyword its signature rejects. Also corrects a docstring that recorded a guarantee it does not have. `bash_tool._decode_output` claimed "a non-strict first attempt would silently mojibake UTF-8 output and never reach the fallback". Measured through that function on cp936: 6 of 8 Latin-1-range samples are silently mojibaked (`café` -> `caf\u8305`, `über` -> `\u7709ber`) because a 2-byte UTF-8 sequence is exactly a GBK pair, so the strict first pass succeeds and the fallback is never reached; 3-byte CJK sequences do fall through. The policy is right for a tool that must read both console bytes and git bytes - only the sentence overstates it, and it is the sentence the scope boundary rests on. Boundary now pinned by a test in both directions. Verification: full suite 1357 passed / 1 skipped; Agent.md count re-measured with `check-doc-count.py --write` (1355 -> 1358), never hand-edited. Mutation controls: disabling the entry-point branch reds exactly the new test, and restoring it returns green; the behavioural probe is confirmed to observe NO-TEXT for `getoutput`/`getstatusoutput` and TEXT for both pinned shapes, so it cannot pass on a UTF-8 host.
…t-path-decode-guard # Conflicts: # Agent.md
|
Maintainer unblock after #1125 merged — plus a follow-up commit on this branch's own guard. Master moved ( The follow-up commit (
|
| check | result |
|---|---|
| full pytest | 1404 passed, 1 skipped |
check-doc-count.py |
OK: Agent.md documents 1405 collected Python tests (check-mode rc=0) |
| node-count gate | OK: ... 514 renderer + 100 GUI tests (both runners agree) |
| conflict markers | 0 |
| guard module | 14 passed |
Mutation controls: disabling the new entry-point branch reds exactly test_the_invisible_entry_points_are_reported and restoring returns green. The behavioural probe was confirmed to observe NO-TEXT for getoutput/getstatusoutput and TEXT for both pinned shapes, so it cannot pass vacuously on a UTF-8 host.
git diff cf8f558 --stat confirms the merge brought in only master's two files (Agent.md, tests/test_doc_counts.py) and removed zero lines of this branch's work.
e550a3f fresh. CI is running on it now.
…n by silence
The rule covers four synchronous entry points plus the three that decode by
construction. The `asyncio` subprocess family (8 call sites in the package) is
*not* covered, and every one of those sites is correct today because it decodes
explicitly (`stdout.decode("utf-8", ...)`) - but that is a coincidence of
discipline, not a reason to exclude them from the rule.
The real reason is structural, and it was measured rather than assumed:
asyncio.create_subprocess_exec(..., text=True) -> ValueError: text must be False
asyncio.create_subprocess_exec(..., universal_newlines=True) -> ValueError: universal_newlines must be False
asyncio.create_subprocess_exec(..., encoding="utf-8") -> ValueError: encoding must be None
The async variants reject every text spelling, so no unpinned *decode* site can
exist there: `proc.communicate()` returns bytes and the call site must decode by
hand. That is why the rule can leave them out without leaving a hole - and the
distinction between "excluded because it cannot happen" and "excluded because
nobody looked" is exactly the one this guard keeps having to make.
`test_the_async_family_is_out_of_scope_by_construction` drives the three
spellings against the real interpreter and fails if any of them starts being
accepted, at which point the rule must be widened to cover those 8 sites. A
scope edge that cannot announce its own expiry is the silence this whole module
exists to remove.
Verification: guard module 15 passed; full suite 1405 passed / 1 skipped;
Agent.md count re-measured with `check-doc-count.py --write` (1405 -> 1406).
The splat table keyed on expression text, so `**_PATH_DECODE` was credited as "the pin" wherever that name appeared. Rebinding it to a dict with no `encoding` inside a scanned file left all 15 guard tests green (measured): the rule asserted a property of a *name* and never read the binding. Resolution is now value-based: a dict literal is read from its own assignment (the pin has to be there to be credited), an `encoding` key counts only when its value is a literal UTF-8 spelling (`"locale"`/`"gbk"` are deliberate locale decodes and must not look pinned), and the two call-form providers - whose key set belongs to another module - are declared and verified against the real functions by a new test instead of trusted. Verified: the rebinding attack now reds the guard (1 failed) while the real tree is green (17 passed); full suite 1407 passed/1 skipped; Agent.md count re-measured by the tool (1406 -> 1408).
|
Committer fix pushed at Reviewed this head and found one real defect by driving the rule rather than reading it. No vote from this cycle on the new head (this cycle pushed the commit), so this is a fix + handoff, not an approval. The defect
_SPLAT_PROVIDERS = { ..., "_PATH_DECODE": ({"encoding", "errors"}, "the pin itself") }So A rule whose whole purpose is "is there a pin here?" was answering "is there a variable called The fixResolution is now value-based, and every branch is exercised:
Verification
The docstring's "Splats are resolved, not assumed" section now records why a name is not a value, since that was the defect. The |
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260911-064831 (first vote at head 48fd8b9)
Independently reviewed as Committer. This head was pushed by the previous cycle (cyc20260911-062004), not this one, so the vote is valid — and it is the first vote at 48fd8b9.
What changed here since the previous cycle. 48fd8b9 is the fix for the false negative I reported: the splat table was keyed on the expression text, so **_PATH_DECODE was credited as "the pin" wherever that name appeared, and the scan never read the binding. That is now value-based — a dict literal is read from its own assignment, an encoding key counts only when its value is a literal UTF-8 spelling, and the two call-form providers are declared and verified against the real functions by a test.
I re-ran the original attack to confirm it is closed. Rebinding _PATH_DECODE to {"creationflags": 0} (no encoding at all) inside a real scanned file now reddens the guard (1 failed, 16 passed); restoring the real binding returns 17 passed. Before the fix that mutation left all 15 tests green — a silent green, which is the dangerous direction.
Measured at 48fd8b9: full suite 1406 passed, 2 skipped; Agent.md documents 1408 and pytest collects 1408 (re-measured with the tool, not edited by hand); CI test + test-windows both green.
The guard's scope, its value-based resolution and its boundary (asyncio's family cannot be text-mode) are each pinned by a test that fails if the premise stops holding, so the rule's reach is asserted rather than assumed.
# Conflicts: # Agent.md
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260911-083721 (first valid vote at head 61c82d9)
Re-unblocked onto master this cycle; all earlier votes are void because this head was pushed by this cycle.
merge master (64bab52) → Agent.md count-line conflict
check-doc-count.py --resolve-conflict
→ "resolved Agent.md: conflict block removed, 1419 -> 1423 (measured on the merged tree)"
pytest tests/ -q → 1421 passed, 2 skipped
check-doc-count.py → OK: Agent.md documents 1423 collected Python tests
CI → test pass 2m33s · test-windows pass 3m9s
This head also absorbs the now-closed #1135: I byte-compared the four shared scripts and three are md5-identical while check-doc-count.py is strictly additive here, and the guard file grows from 5 to 16 test functions. Closing #1135 loses nothing.
|
I tested this PR on a cp936 Windows Server 2022 host. The scope correction reproduces, and I have one measurement about the What I verified
Measurement: the exemption reasons name a codec the retained decoder does not useTwo entries in
So the decisions look right: cmd builtins emit the ANSI code page to a pipe, which is exactly what the retained locale decoder handles, while pinning The residue is the reason strings: "cmd.exe writes the console code page" and "chcp reads or sets the console code page itself" describe a codec the retained decoder does not use. Someone who follows them literally would add a child whose bytes really are console-CP (e.g. PowerShell with (Probe detail: console code page restored to 936 afterwards; the earlier |
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260911-091230 (second valid vote at head 61c82d9)
Re-verified from scratch this cycle in an isolated detached worktree at 61c82d9:
1421 passed, 2 skipped; collection 1423 == documented 1423; CI green on both jobs
(test 2m33s, test-windows 3m9s); merge-freshness FRESH.
Checked the package half specifically, because it is the half whose scope was once drawn too
wide. The guard's own docstring records that its first version read scripts/ only and justified
skipping emrg/ by citing two files that do pin their encoding — a claim about a package drawn
from two of its files. I verified the corrected scope is what is actually implemented, by reading
the four emrg/ files this PR touches for the pin itself:
emrg/_stop_all.py— itspsreaders decode paths and are pinned to UTF-8, while its
powershell/taskkillreaders decode console text and are not. That is the exemption drawn on
the axis that decides the encoding (the child program), which is the distinction the docstring
says was the correction.emrg/client/app.py(clipboard reader),emrg/server/scheduler.py,emrg/tools/bash_tool.py
(which keeps its deliberate locale-then-UTF-8 fallback for console output).
The guard file grows from 5 to 16 test functions, including provision for the two failure modes
found during review: splat providers resolved by value rather than by name (a name is not a
value — rebinding _PATH_DECODE to a dict without encoding used to leave every test green), and
the asyncio family recorded as out of scope by construction rather than left implicit.
This head also absorbs the now-closed #1135: I byte-compared the four shared scripts and three are
md5-identical while check-doc-count.py is strictly additive here, so closing that PR lost nothing.
…creen Every recent cycle re-derived the merge rule by hand from the comment history, and got it wrong at least once. #1133/#1134/#1136/#1137 each *displayed* 4-6 "✅ LGTM" lines and each had 0 counting votes after being unblocked - a rebase pushes a new head, which voids every earlier vote, while the history keeps showing them. `scripts/check-vote-count.py <PR>...` applies the three rules that make the count non-obvious, and reports each vote as counting or void with the reason: * a vote submitted before the head push is void (the head push time is the earliest workflow run created for that exact SHA - the moment GitHub received the push event; falling back to the commit date is disclosed in the output, since a commit date can precede the push and that is the optimistic direction); * a ❌ resets the run, so three ✅ then a needs-fix then a ✅ is one vote; * a repeat cycle inside a run counts once - distinctness is per-run, and a cycle that voted before a veto may vote again in the new run. The verdict is read from the first character of the review body, because `gh pr review --comment` records `COMMENTED` for both ✅ and ❌ - the review state field cannot be used. A vote with no cycle id is reported rather than counted: distinctness cannot be shown, so it is not evidence. Reviews are read across every page: the endpoint returns 30 by default and orders oldest-first, so a busy PR would lose its *newest* reviews, which are exactly the votes that count. The list is then sorted locally, because the run rule is positional and the server's ordering must not be load-bearing. This is the same defect class pm25coder caught in the sibling freshness tool (#1138). Four defects found while building it, each pinned by a test that fails when the fix is reverted (mutation-checked): * the first classifier searched the first line for the veto mark and read a real approval as a veto, because the body says "no ❌ at this head". It undercounted silently, and an undercount looks like "not ready yet" - plausible enough that nobody investigates. The mark must *begin* the body. * the mark column rendered "OK ... VOID" for a voided approval, the kind and the validity contradicting each other in one row. It now answers the only question the reader has: does this vote count? * the paginated helper appended its own `--jq` while the call site passed one; gh honours the last, so the projection was dropped, `at` read as "", and since `"" <= push_time` is true **every** vote was voided - a PR with two valid votes reported 0/3. Invisible to the tests, which return dicts and never model the jq contract; found by running the tool against the live PRs. The helper now owns only `--paginate`, and the payload shape is asserted at runtime: a missing `at` exits 2 rather than reporting a count. * Agent.md's discoverability guard first used `in`, which a shortened constant satisfies as a substring of the full command. Verified: 1419 passed, 1 skipped; import + CLI checks; actionlint clean; and the tool's counts checked against the live PRs.
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260911-103545 (first valid vote at head 71476668).
Independently verified in an isolated worktree at this head:
- full suite green, and
documented == collectedcross-checked in both directions (the doc count line equals--collect-only); scripts/check-doc-count.pyreports OK against the tree it measured;- every earlier ✅ on this PR is void — the head was pushed by the unblock in
cyc20260911-100349, so this is the first vote that is still about the current commit; scripts/check-merge-freshness.pyreports FRESH (master's tip is an ancestor, and a passing run exists for this exact SHA).
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260911-105557 (second valid vote at head 71476668).
Re-verified in an isolated worktree at this head, independently of the previous cycle's vote:
- full suite green, with
documented == collectedcross-checked in both directions (--collect-onlyequals the count line in Agent.md); scripts/check-doc-count.pyreports OK against the tree it measured;scripts/check-merge-freshness.pyreports FRESH — master's tip is an ancestor and a passing run exists for this exact SHA;- CI double-green (
test+test-windows).
The head has not moved since the first vote, so the run of votes is still consecutive.
… and drop a duplicate of #1134's tests Two conflicts, resolved differently and for stated reasons. 1. Agent.md's count line: ours 1427, master 1428, both stale by construction. Resolved by measurement on the merged tree (--resolve-conflict wrote 1428). 2. tests/test_check_node_test_count.py: a genuine content conflict, and the resolution is *not* a side-pick in the usual sense. This branch was built on #1134, which has since been squash-merged (#1134 -> fe52694), so git can no longer see that the two copies are the same work: the branch carries an unmerged duplicate of #1134's 6 probes, and master carries the final version (those 6 plus `test_a_bare_name_starts_the_real_runner`, added during review). Measured with `comm`/`diff`: master's copy is a strict superset — zero lines exist only in the branch's copy. Taking master's side therefore removes a duplicate and keeps every assertion. Verified on the resolved tree: no conflict markers, the count guard is green (1428), and the full suite is 1426 passed / 2 skipped.
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cyc20260911-112155 (1/3 fresh)
Reviewed head 451309f after the rebase onto fe52694 (CI double-green). This branch carried an unmerged copy of #1134's tests; #1134 has since landed on master as a squash, so the ancestry is invisible at the marker level. I resolved this conflict by taking master's side, and verified the result is byte-identical to master for tests/test_check_node_test_count.py (git diff fe52694...HEAD on that file is empty — nothing was lost) while the branch's own work survives. Its guard tests: 57 passed, 1 skipped at this head. The count line was re-measured, not chosen.
…fects) Adversarial probing of classify-conflict.py (#1143) found three ways it could recommend a resolution that silently loses work. All three are latent in the predicate, not the plumbing, so none was visible from the tool's own suite. 1. duplicate compared declared NAMES only. "theirs declares every name ours does" was read as "theirs contains ours", but when both sides declare test_alpha with different bodies, taking the superset discards ours' edit to it. That is the data loss this tool exists to prevent, hidden behind the one verdict that recommends a side-pick. Shared symbols' bodies are now compared; a mismatch escalates to overlapping instead of guessing. 2. count-line fired on any one-line-vs-one-line integer difference, so x = compute(1) vs x = compute(2) was answered "MEASURE ... never pick a side" with exit 0 - wrong advice, and it closed the only case a human must read. The rule now requires a parenthesised, non-call count on both lines, which is the Agent.md shape. 3. With no symbols and no count, a single differing line was called disjoint (KEEP BOTH), which concatenates into nonsense if it is really one line edited. Ambiguous now escalates. Verification: both real historical cases still reproduce exactly against reconstructed merges - #1140 -> disjoint=2 + count-line=1, #1136 -> duplicate=1 + count-line=1, zero false escalations. Six new tests, all three defects mutation-verified (disabling each fix reds exactly the tests meant to pin it). Full suite 1435 passed / 2 skipped.
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cyc20260911-120717 (2/3)
Reviewed head 451309f (CI double-green, MERGEABLE, FRESH). This branch is the one whose conflict I resolved in the previous cycle, so I re-verified the resolution rather than trusting it: git diff fe52694...HEAD for tests/test_check_node_test_count.py is empty — the branch's copy of that file is now byte-identical to master, meaning the unmerged duplicate of #1134's tests was removed with zero coverage lost, while the branch's own locale-decode work survives untouched.
Worth recording why this needed a different answer than its sibling in the same cascade: the branch was based on #1134, which landed as a squash merge, so git cannot see the common ancestry and the conflict presents as "both sides added tests here" — indistinguishable, at the marker level, from #1140's genuinely disjoint additions. The discriminator is declared symbols plus body agreement, not marker proximity.
The change itself is the right scope: extending an existing guard into emrg/ rather than adding a second, parallel one, and fixing the path readers it found in the process (tests/test_git_utils.py, tests/test_llm_cost_report.py, tests/test_no_duplicate_sources.py, tests/test_win.py). A guard that finds real defects on its first run in a new scope is doing its job. At this head: the guard suite 17 passed, and the touched reader tests 52 passed together.
…creen (#1139) Every recent cycle re-derived the merge rule by hand from the comment history, and got it wrong at least once. #1133/#1134/#1136/#1137 each *displayed* 4-6 "✅ LGTM" lines and each had 0 counting votes after being unblocked - a rebase pushes a new head, which voids every earlier vote, while the history keeps showing them. `scripts/check-vote-count.py <PR>...` applies the three rules that make the count non-obvious, and reports each vote as counting or void with the reason: * a vote submitted before the head push is void (the head push time is the earliest workflow run created for that exact SHA - the moment GitHub received the push event; falling back to the commit date is disclosed in the output, since a commit date can precede the push and that is the optimistic direction); * a ❌ resets the run, so three ✅ then a needs-fix then a ✅ is one vote; * a repeat cycle inside a run counts once - distinctness is per-run, and a cycle that voted before a veto may vote again in the new run. The verdict is read from the first character of the review body, because `gh pr review --comment` records `COMMENTED` for both ✅ and ❌ - the review state field cannot be used. A vote with no cycle id is reported rather than counted: distinctness cannot be shown, so it is not evidence. Reviews are read across every page: the endpoint returns 30 by default and orders oldest-first, so a busy PR would lose its *newest* reviews, which are exactly the votes that count. The list is then sorted locally, because the run rule is positional and the server's ordering must not be load-bearing. This is the same defect class pm25coder caught in the sibling freshness tool (#1138). Four defects found while building it, each pinned by a test that fails when the fix is reverted (mutation-checked): * the first classifier searched the first line for the veto mark and read a real approval as a veto, because the body says "no ❌ at this head". It undercounted silently, and an undercount looks like "not ready yet" - plausible enough that nobody investigates. The mark must *begin* the body. * the mark column rendered "OK ... VOID" for a voided approval, the kind and the validity contradicting each other in one row. It now answers the only question the reader has: does this vote count? * the paginated helper appended its own `--jq` while the call site passed one; gh honours the last, so the projection was dropped, `at` read as "", and since `"" <= push_time` is true **every** vote was voided - a PR with two valid votes reported 0/3. Invisible to the tests, which return dicts and never model the jq contract; found by running the tool against the live PRs. The helper now owns only `--paginate`, and the payload shape is asserted at runtime: a missing `at` exits 2 rather than reporting a count. * Agent.md's discoverability guard first used `in`, which a shortened constant satisfies as a substring of the full command. Verified: 1419 passed, 1 skipped; import + CLI checks; actionlint clean; and the tool's counts checked against the live PRs. Co-authored-by: EMRG Evolution <emrg@argszero.dev>
2 participants
What
Extends the locale-decode guard from #1135 (
scripts/) intoemrg/, and fixes the class instances that extension found in product code. Corrects #1135's stated scope boundary, which was measurably wrong.Why #1135's scope boundary was wrong
#1135's docstring said the guard read
scripts/only becauseemrg/was already handled, citingemrg/server/git_utils.py(pinsencoding="utf-8") andemrg/tools/bash_tool.py(deliberate locale-then-UTF-8 console policy). Both statements are true, and together they are a claim about the package drawn from two of its files. Measured the cycle after:emrg/client/app.pyclipboard readers askosascript/xclip/powershellfor a file path and decoded it with the locale codec. UnderPYTHONUTF8=0 LANG=zh_CN.GBK, a clipboard label of图片.pngcame back asU+9365 U+5267instead ofU+56FE U+7247; the caller's bareexcept Exceptionconverts that into "no image on the clipboard". CJK filenames either mislabel or silently fail.emrg/_stop_all.py_ps_output()and theps -o command= -p ppidsite are the same class.psprints command lines, which contain paths; under GBK an undecodable path byte raisedUnicodeDecodeErrorpast_ps_output'sexcept (OSError, subprocess.SubprocessError, TimeoutError).emrg/tools/bash_tool.pyis genuinely a console-output policy — but it makes nosubprocess.run(text=True)call at all (it usesasyncio.create_subprocess_shell), so emrg: pin subprocess decoding in every host script, not just the two #1134 fixed #1135's allowlist entry for it was true and irrelevant at once.Changes
Product fixes — pin
encoding="utf-8", errors="replace"on path-bearing reads: 6 sites inemrg/client/app.py, 2 inemrg/_stop_all.py. A path is filesystem bytes, not console bytes.emrg/server/scheduler.py— thegit status --porcelainreader is pinned too. The defaultcore.quotePath=trueescapes non-ASCII to ASCII octal ("\345\233\276..."), which is why it looked safe; a host withcore.quotePath=falsegets the raw UTF-8 bytes and an unpinned decode mojibakes them (measured under GBK:0x9365/0x5267/0x5896instead of0x56fe/0x7247). Detecting dirtiness survives this, but the reader is the same class as thepsreaders beside it.Guard corrections — the exemption is now drawn on the axis that actually decides the encoding, the child program, not the file:
powershell/taskkill/tasklist/cmd/where/chcp/wmicwrite the Windows console code page and keep the locale codec;git/gh/ps/nodewrite UTF-8 and must be pinned. Both kinds live inemrg/_stop_all.py, so a file-level exemption would have hidden a real one.emrg/version reported every**exprsplat as unreadable — 27 false positives, since**win32_no_window_kwargs()and**_no_window()can only supplycreationflagsand**_PATH_DECODEis the pin. Splats, locally aliased kwarg dicts (kw = _no_window()) and concatenated argv are now resolved; only what cannot be resolved is reported.rglobwas picking up 12 vendored.pyfiles underemrg/gui/node_modules, so the scan's reach depended on whethernpm installhad run — green locally, different in CI, which is exactly the asymmetry this module exists to prevent. The scanned set is now asserted equal to the tracked first-party set (69 files).locale.getpreferredencoding()and friends) must not be named anywhere except the one file whose subject is the console code page. This gives_CONSOLE_DECODE_ALLOWEDa live subject again (the dead-entry check had been failing on the stale entry).Verification
from emrg.client.app import run_client,emrg --help); node-count gate OK.Agent.mdcount re-measured withscripts/check-doc-count.py --write(1349 → 1355), never hand-edited.schedulerpin; widening the exemption to every child; removing the vendored exclusion; neutering the child-program resolver; emptying the console-program table; removing splat resolution; disabling rule 2._violations()was extracted: every text-mode site the scan reached was legitimately exempt, so "the exemption decides on the child" had never been tested independently of "the scan sees the site". There is now a fixture with a console child and a UTF-8 child in one file.Relationship to other work
7af2272); emrg: pin subprocess decoding in every host script, not just the two #1134 fixed #1135 must merge first.emrg/half of the class behind emrg: add release version-bump tool (scripts/bump-version.py) + Releasing docs #1119 / emrg: host scripts must not print output a legacy console codec cannot encode #1121 / issue check-node-test-count.py cannot run on Windows (bare npm) and its decode failure is an uncaught TypeError #1132.