From 82290c575db3c70a7d03135f6808a83a3b19c043 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 22:17:32 +0000 Subject: [PATCH 1/3] fix: caret-parity bug in audit tool; correct future-dated backlog entries tools/audit_console_messages.py's redirect/separator detection only checked one caret back, so cmd.exe's real parity rule (^^& is a literal caret followed by an ACTIVE separator, not an escaped one) was inverted -- a doubled-caret separator was wrongly treated as escaped, letting a later chained command's redirect wrongly drop the call :log record before it. Replaced the single-lookbehind regexes with an explicit backward caret-count scan; added ^^&/^^| regression cases plus a single-caret contrast case. Also: fixed four 2026-08-01 dates that should read 2026-07-31 (today), and removed docs/open-questions.md's already-resolved cascade-consent paragraph, which contradicted the file's own "only currently-open items" scope. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01A4iE1BRSkUETwz237XeuTW --- CLAUDE.md | 6 +-- docs/open-questions.md | 11 +----- tests/test_audit_console_messages.py | 28 ++++++++++++++ tools/audit_console_messages.py | 57 ++++++++++++++++++++-------- 4 files changed, 73 insertions(+), 29 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1a128182..a549d8a7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -485,7 +485,7 @@ Item numbers are informal labels for cross-referencing within a session or PR, n guaranteed-unique ID scheme -- pick anything that looks free in the list below when filing a new item; do not cross-check it against `docs/agent-closed-backlog.md`'s history first, and do not renumber an item if it later turns out to coincidentally repeat an older, already-closed item's -number. (Owner decision 2026-08-01, see Known Findings below: the earlier renumber-on-collision +number. (Owner decision 2026-07-31, see Known Findings below: the earlier renumber-on-collision convention was more rigor than a plain-text backlog needs -- a real uniqueness guarantee belongs in an actual issue tracker, e.g. GitHub Issues, not a hand-maintained numbering scheme here.) Once an item is fully resolved it is removed from here entirely and archived (keeping its @@ -695,7 +695,7 @@ start at 1 and has gaps. - **19. The `cache` CI lane's corruption recovery is a one-way trap: once a restored cache is flagged corrupted, nothing in that lane ever produces a fresh, valid cache again -- found - 2026-08-01 while investigating a maintainer report that the lane "never works," always logging + 2026-07-31 while investigating a maintainer report that the lane "never works," always logging `Cache corrupted, skipping fast-path tests (HP_CACHE_CORRUPTED=1)`, confirmed against the current `.github/workflows/batch-check.yml` source, not just the symptom report.** Traced the full mechanism: the cache key is `win-...-conda-${{ hashFiles('run_setup.bat') }}-` @@ -936,7 +936,7 @@ of a second or third pin actually needing it. ## Known Findings (diagnosed, no action warranted) -- **Backlog item numbering: renumber-on-collision convention dropped, 2026-08-01 owner decision.** +- **Backlog item numbering: renumber-on-collision convention dropped, 2026-07-31 owner decision.** A prior pass found that Active Backlog items 9 and 11 (filed 2026-07-29) each collided with an older, already-closed item of the same number, and -- following a convention this file used to document -- renumbered them to 17 and 16 when archiving (see `docs/agent-closed-backlog.md`'s diff --git a/docs/open-questions.md b/docs/open-questions.md index fc7f4b68..b0e0e1f3 100644 --- a/docs/open-questions.md +++ b/docs/open-questions.md @@ -11,7 +11,7 @@ changelog-style sections are for. **1. Is the `cache` CI lane's self-perpetuating-corruption bug (CLAUDE.md Active Backlog item 19) worth fixing, given the lane is explicitly non-gating/informational by design?** Diagnosed -2026-08-01: once a restored cache is flagged corrupted, the lane has no code path that ever +2026-07-31: once a restored cache is flagged corrupted, the lane has no code path that ever produces a fresh valid cache again (the one step that could do a fresh install is skipped whenever corruption is detected, and the save step is gated on the same flag) -- so it likely never recovers on its own, consistent with the maintainer's own report that it "never works." @@ -24,12 +24,3 @@ deliberately excluded from PR-merge gating either way, is it worth the multi-cyc effort, or should it stay as documented, low-priority backlog debt? No strong reason not to attempt it in a dedicated future loop either way -- flagging so the priority call is explicit rather than assumed. - ---- - -The cascade-consent design question (was item 1: whether the cascade -decision should remain an interruptive consent prompt at all, given `HP_CASCADE_CANDIDATE`'s own -mixed reliability) is resolved -- kept exactly as shipped, no code change. See CLAUDE.md's -Known Findings entry for the decision and `docs/agent-interconnect.md`'s "Cascade signal -reliability" subsection (under "Post-execution checkpoint") for the full truth-table analysis -that informed it. diff --git a/tests/test_audit_console_messages.py b/tests/test_audit_console_messages.py index e88aa929..fa6116e7 100644 --- a/tests/test_audit_console_messages.py +++ b/tests/test_audit_console_messages.py @@ -100,6 +100,34 @@ def test_extract_records_keeps_call_log_with_redirect_on_later_chained_command(t assert records == [(1, '[INFO] visible')] +def test_extract_records_keeps_call_log_with_doubled_caret_separator(tmp_path): + # cmd.exe caret parity: '^^&' is a literal '^' followed by an ACTIVE '&' (the first caret + # escapes the second, leaving the '&' unescaped) -- so this is a real command separator, and + # the later command's redirect must not cause the call :log record to be dropped. + bat = tmp_path / 'run_setup.bat' + bat.write_text('call :log "[INFO] visible" ^^& echo hidden > log.txt\n', encoding='ascii') + records = acm.extract_records(bat) + assert records == [(1, '[INFO] visible')] + + +def test_extract_records_keeps_call_log_with_doubled_caret_pipe_separator(tmp_path): + # Same parity rule for '|': '^^|' is a literal '^' followed by an ACTIVE '|'. + bat = tmp_path / 'run_setup.bat' + bat.write_text('call :log "[INFO] visible" ^^| echo hidden > log.txt\n', encoding='ascii') + records = acm.extract_records(bat) + assert records == [(1, '[INFO] visible')] + + +def test_extract_records_skips_call_log_with_single_caret_escaped_ampersand(tmp_path): + # Contrast case: a SINGLE caret ('^&') genuinely escapes the '&' into literal text -- there + # is no real separator here, so the whole line (including its trailing redirect) is one + # command and the call :log record is correctly dropped. + bat = tmp_path / 'run_setup.bat' + bat.write_text('call :log "[INFO] hidden" ^& echo also_hidden > log.txt\n', encoding='ascii') + records = acm.extract_records(bat) + assert records == [] + + def test_extract_records_skips_call_log_with_redirect_in_same_segment(tmp_path): # A redirect immediately after the closing quote, with no intervening command # separator, DOES belong to this call :log -- it must still be skipped. diff --git a/tools/audit_console_messages.py b/tools/audit_console_messages.py index f6942dff..d71ed010 100644 --- a/tools/audit_console_messages.py +++ b/tools/audit_console_messages.py @@ -42,25 +42,50 @@ re.IGNORECASE, ) -# A real redirect operator: one or two literal '>' not preceded by a caret escape (a caret -# protects an INNER command's own redirect, e.g. the `2^>nul` on a nested `powershell -Command` -# call, from being read as this line's own redirect) and not immediately followed by '=' (so a +# A candidate redirect token: one or two literal '>' not immediately followed by '=' (so a # genuine console message containing '>=' -- e.g. "running (>=30 days since last update)" -- -# is never mistaken for a redirect). -REDIRECT_RE = re.compile(r'(?{1,2}(?!=)') +# is never mistaken for a redirect). Caret-escape state is NOT checked by the regex itself -- +# see _is_caret_escaped() below -- a single trailing (?{1,2}(?!=)') -# A real command separator (& / | / && / ||), not caret-escaped. A redirect appearing AFTER one -# of these belongs to a later, separately-chained command -- it must not cause the FIRST -# command's own record (e.g. a `call :log` that already reached the console before the -# separator) to be dropped. -SEGMENT_END_RE = re.compile(r'(? bool: + """True if the character at `pos` is caret-escaped, per cmd.exe's parity rule: a caret + escapes the NEXT character, and a caret itself is escaped by a preceding caret. So an ODD + number of consecutive carets immediately before `pos` means `text[pos]` is escaped (^X); + an EVEN number (including zero) means it is not -- e.g. ^^& is a literal caret followed by + an ACTIVE '&', not an escaped one. A single-character regex lookbehind cannot express this + for an arbitrary run of carets, so it is checked by explicit backward scan instead.""" + carets = 0 + i = pos - 1 + while i >= 0 and text[i] == '^': + carets += 1 + i -= 1 + return carets % 2 == 1 + + +def _first_unescaped(pattern: re.Pattern, text: str): + """Return the first match of `pattern` in `text` whose start position is not + caret-escaped, or None.""" + for m in pattern.finditer(text): + if not _is_caret_escaped(text, m.start()): + return m + return None + + +def _has_unescaped_redirect(text: str) -> bool: + return _first_unescaped(_REDIRECT_TOKEN_RE, text) is not None def _own_command_segment(tail: str) -> str: - """Truncate `tail` at the first real command separator, so a redirect belonging to a - later chained command is never mistaken for one that applies to the command being - checked.""" - m = SEGMENT_END_RE.search(tail) + """Truncate `tail` at the first real (unescaped) command separator, so a redirect + belonging to a later chained command is never mistaken for one that applies to the + command being checked.""" + m = _first_unescaped(_SEGMENT_TOKEN_RE, tail) return tail[:m.start()] if m else tail @@ -95,7 +120,7 @@ def extract_records(bat_path: Path): # is indistinguishable from message content by position alone -- scan the whole raw # line. No current echo message contains a real (non-'>=', non-caret-escaped) '>', # confirmed by a full-file sweep at the time this check was added. - if REDIRECT_RE.search(raw): + if _has_unescaped_redirect(raw): continue records.append((i, normalize(body))) continue @@ -107,7 +132,7 @@ def extract_records(bat_path: Path): # inside the matched group, never scanned here. Truncate at the first real command # separator first: a redirect on a LATER, `&`-chained command (e.g. `call :log "..." # & echo hidden > file`) does not mean call :log itself was redirected. - if REDIRECT_RE.search(_own_command_segment(line[m.end():])): + if _has_unescaped_redirect(_own_command_segment(line[m.end():])): continue records.append((i, normalize(m.group(1)))) return records From 4df04b50b8d45f9b9b09454e2feed69cb7792d65 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 1 Aug 2026 00:06:57 +0000 Subject: [PATCH 2/3] docs/style: fix stale item-1 reference; add derived-requirement tags, return annotation CodeRabbit review batch on PR #407: - CLAUDE.md's cascade-consent entry still said it closed "docs/open-questions.md item 1" -- stale after #406 removed that resolved question and item 1 now refers to the unrelated cache-lane question. Switched to descriptive wording. - Added the repo's own "# derived requirement: " tag comments (Key Conventions table) at the caret-parity implementation and its regression test group -- the docstrings already explained the why, but the convention wants the grep-able tag too. - Added an Optional[re.Match] return annotation to _first_unescaped (Ruff ANN202). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01A4iE1BRSkUETwz237XeuTW --- CLAUDE.md | 3 ++- tests/test_audit_console_messages.py | 3 +++ tools/audit_console_messages.py | 9 ++++++++- 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index a549d8a7..f36b084e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -953,7 +953,8 @@ of a second or third pin actually needing it. item that tracked renumbering them is closed with no further action. - **Cascade consent gate design (timed prompt, decline-by-default) kept exactly as shipped, - 2026-07-26 owner decision -- closes `docs/open-questions.md` item 1.** Following the + 2026-07-26 owner decision -- closes the former cascade-consent question in + `docs/open-questions.md`.** Following the cascade-vs-postexec fix (see Closed Backlog), a deeper investigation into the reliability of the `HP_CASCADE_CANDIDATE` signal was requested and completed -- a full truth table over its two constituent build-time static signals, with estimated odds the `.exe` still runs fine and diff --git a/tests/test_audit_console_messages.py b/tests/test_audit_console_messages.py index fa6116e7..566afa91 100644 --- a/tests/test_audit_console_messages.py +++ b/tests/test_audit_console_messages.py @@ -100,6 +100,9 @@ def test_extract_records_keeps_call_log_with_redirect_on_later_chained_command(t assert records == [(1, '[INFO] visible')] +# derived requirement: cmd.exe caret-escape parity is odd-count-escapes / even-count-active, +# not "any preceding caret escapes" -- see tools/audit_console_messages.py's +# _is_caret_escaped() docstring for the full rule this group regression-tests. def test_extract_records_keeps_call_log_with_doubled_caret_separator(tmp_path): # cmd.exe caret parity: '^^&' is a literal '^' followed by an ACTIVE '&' (the first caret # escapes the second, leaving the '&' unescaped) -- so this is a real command separator, and diff --git a/tools/audit_console_messages.py b/tools/audit_console_messages.py index d71ed010..e7d70409 100644 --- a/tools/audit_console_messages.py +++ b/tools/audit_console_messages.py @@ -36,6 +36,7 @@ import re import sys from pathlib import Path +from typing import Optional TEST_ONLY_RE = re.compile( r'HP_TEST|\[TEST\]|simulating|inject(ed|ing)|corrupt_conda|corrupt_uv', @@ -53,6 +54,12 @@ _SEGMENT_TOKEN_RE = re.compile(r'[&|]') +# derived requirement: cmd.exe caret-escape parity is not a one-char lookaround -- ^X escapes +# X, but ^^X is a literal caret followed by an UNESCAPED X (the first caret escapes the +# second). A naive `(? bool: """True if the character at `pos` is caret-escaped, per cmd.exe's parity rule: a caret escapes the NEXT character, and a caret itself is escaped by a preceding caret. So an ODD @@ -68,7 +75,7 @@ def _is_caret_escaped(text: str, pos: int) -> bool: return carets % 2 == 1 -def _first_unescaped(pattern: re.Pattern, text: str): +def _first_unescaped(pattern: re.Pattern, text: str) -> Optional[re.Match]: """Return the first match of `pattern` in `text` whose start position is not caret-escaped, or None.""" for m in pattern.finditer(text): From d50cfcac266fe27de08273f31195d29aefaa51ce Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 1 Aug 2026 00:10:37 +0000 Subject: [PATCH 3/3] docs: add tools/README.md scoping which files are informal vs. load-bearing Requested to reduce review-nitpick friction on genuinely manual, hand-run scripts like audit_console_messages.py -- but most of tools/ is CI-wired or a canonical source for an embedded run_setup.bat payload, so a blanket "nothing here needs rigor" claim would be wrong and risk a future regression. Verified against .github/workflows/*.yml and each file's own header/docstring before writing: only audit_console_messages.py and audit_batch_exit_paths.py are genuinely informal by that standard; everything else defaults to full rigor. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01A4iE1BRSkUETwz237XeuTW --- tools/README.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 tools/README.md diff --git a/tools/README.md b/tools/README.md new file mode 100644 index 00000000..06054b21 --- /dev/null +++ b/tools/README.md @@ -0,0 +1,40 @@ +# tools/ + +This directory is mixed-purpose. Most of it is load-bearing -- do not assume anything here is +low-stakes without checking which category a given file falls into. + +## Most files here need full review rigor + +- **CI-wired scripts**, invoked directly by `.github/workflows/*.yml` (e.g. `apply_patch.py`, + `check_ndjson_registry.py`, `check_workflows_yaml.py`, `inline_model_fix.py`, `parse_warn.py`, + `prep_requirements.py`, `diag/ndjson_fail_list.py`, `diag/publish_index.py`) or that gate CI + decisions even if not directly listed in a workflow step (`iterate_gate.ps1` -- "the active gate + used by CI"; `sanitize_iterate_payload.py` -- defines the current NDJSON contract). +- **Canonical sources for `run_setup.bat`'s embedded `HP_*` payloads** (e.g. `pyproj_deps.py`, + `prep_requirements.py`, `collect_submodules.py`, `hidden_import_scan.py`, `dep_check.py`, + `env_state.py`, `detect_python.py`, `detect_visa.py`, `find_entry.py`, `embed_extract.ps1`, + `embed_pyver_check.py`, `exe_smokerun.ps1`, `failfast_probe.ps1`, `run_installer_with_timeout.ps1`, + `pvw_known_idempotent.py`, `autopep_merge.py`, `pep723_writeback.py`). These ship as part of the + actual deliverable (base64-embedded in `run_setup.bat`) -- arguably higher stakes than a CI + script, not lower. Update via `tools/sync_payload.py`, never by hand-editing the embedded + base64 (see `docs/agent-lessons-learned.md`'s "Embedded Helper Update Workflow"). +- **Required dev-workflow tooling**, run before every commit even though no `.yml` file calls it + directly: `check_delimiters.py`, `run_sanity_sweep.sh`, `ps-compileall.ps1`, `sync_payload.py`. + See CLAUDE.md's "Mandatory Sanity Checks" section. +- Everything else not explicitly named below. + +## Exceptions: manual, hand-run, not required by any flow + +These two are genuinely informal -- standalone diagnostic/audit scripts, run by hand on demand, +not invoked by CI, not a canonical payload source, and not part of any required pre-commit check. +Normal-rigor review asks (type annotations, `# derived requirement:` comment tags, etc.) don't +apply the same way here; use judgment rather than treating every linter nitpick as actionable. + +- `audit_console_messages.py` -- own docstring: "Not wired into CI -- run by hand when + re-reviewing docs/demo-bootstrapper-output.md... or after a batch of new lines." +- `audit_batch_exit_paths.py` -- a one-off audit tool for tracing `run_setup.bat`'s exit paths; + own module docstring notes it is "not a final answer on its own." + +If you add a new script that's genuinely in this category, say so explicitly in its own +docstring/header (matching the two examples above) and add it to this list in the same commit -- +don't rely on this README alone to establish that later.