fix: caret-parity bug in audit tool; correct future-dated backlog entries

CLAUDE.md

@@ -485,7 +485,7 @@ Item numbers are informal labels for cross-referencing within a session or PR, n
 guaranteed-unique ID scheme -- pick anything that looks free in the list below when filing a new
 item; do not cross-check it against `docs/agent-closed-backlog.md`'s history first, and do not
 renumber an item if it later turns out to coincidentally repeat an older, already-closed item's
-number. (Owner decision 2026-08-01, see Known Findings below: the earlier renumber-on-collision
+number. (Owner decision 2026-07-31, see Known Findings below: the earlier renumber-on-collision
 convention was more rigor than a plain-text backlog needs -- a real uniqueness guarantee belongs
 in an actual issue tracker, e.g. GitHub Issues, not a hand-maintained numbering scheme here.)
 Once an item is fully resolved it is removed from here entirely and archived (keeping its
@@ -695,7 +695,7 @@ start at 1 and has gaps.
 
 - **19. The `cache` CI lane's corruption recovery is a one-way trap: once a restored cache is
    flagged corrupted, nothing in that lane ever produces a fresh, valid cache again -- found
-   2026-08-01 while investigating a maintainer report that the lane "never works," always logging
+   2026-07-31 while investigating a maintainer report that the lane "never works," always logging
    `Cache corrupted, skipping fast-path tests (HP_CACHE_CORRUPTED=1)`, confirmed against the
    current `.github/workflows/batch-check.yml` source, not just the symptom report.** Traced the
    full mechanism: the cache key is `win-...-conda-${{ hashFiles('run_setup.bat') }}-<pipreqs_ver>`
@@ -936,7 +936,7 @@ of a second or third pin actually needing it.
 
 ## Known Findings (diagnosed, no action warranted)
 
-- **Backlog item numbering: renumber-on-collision convention dropped, 2026-08-01 owner decision.**
+- **Backlog item numbering: renumber-on-collision convention dropped, 2026-07-31 owner decision.**
   A prior pass found that Active Backlog items 9 and 11 (filed 2026-07-29) each collided with an
   older, already-closed item of the same number, and -- following a convention this file used to
   document -- renumbered them to 17 and 16 when archiving (see `docs/agent-closed-backlog.md`'s
@@ -953,7 +953,8 @@ of a second or third pin actually needing it.
   item that tracked renumbering them is closed with no further action.
 
 - **Cascade consent gate design (timed prompt, decline-by-default) kept exactly as shipped,
-  2026-07-26 owner decision -- closes `docs/open-questions.md` item 1.** Following the
+  2026-07-26 owner decision -- closes the former cascade-consent question in
+  `docs/open-questions.md`.** Following the
   cascade-vs-postexec fix (see Closed Backlog), a deeper investigation into the reliability of
   the `HP_CASCADE_CANDIDATE` signal was requested and completed -- a full truth table over its
   two constituent build-time static signals, with estimated odds the `.exe` still runs fine and

docs/open-questions.md

@@ -11,7 +11,7 @@ changelog-style sections are for.
 
 **1. Is the `cache` CI lane's self-perpetuating-corruption bug (CLAUDE.md Active Backlog item 19)
 worth fixing, given the lane is explicitly non-gating/informational by design?** Diagnosed
-2026-08-01: once a restored cache is flagged corrupted, the lane has no code path that ever
+2026-07-31: once a restored cache is flagged corrupted, the lane has no code path that ever
 produces a fresh valid cache again (the one step that could do a fresh install is skipped
 whenever corruption is detected, and the save step is gated on the same flag) -- so it likely
 never recovers on its own, consistent with the maintainer's own report that it "never works."
@@ -24,12 +24,3 @@ deliberately excluded from PR-merge gating either way, is it worth the multi-cyc
 effort, or should it stay as documented, low-priority backlog debt? No strong reason not to
 attempt it in a dedicated future loop either way -- flagging so the priority call is explicit
 rather than assumed.
-
----
-
-The cascade-consent design question (was item 1: whether the cascade
-decision should remain an interruptive consent prompt at all, given `HP_CASCADE_CANDIDATE`'s own
-mixed reliability) is resolved -- kept exactly as shipped, no code change. See CLAUDE.md's
-Known Findings entry for the decision and `docs/agent-interconnect.md`'s "Cascade signal
-reliability" subsection (under "Post-execution checkpoint") for the full truth-table analysis
-that informed it.

tests/test_audit_console_messages.py

@@ -100,6 +100,37 @@ def test_extract_records_keeps_call_log_with_redirect_on_later_chained_command(t
     assert records == [(1, '[INFO] visible')]
 
 
+# derived requirement: cmd.exe caret-escape parity is odd-count-escapes / even-count-active,
+# not "any preceding caret escapes" -- see tools/audit_console_messages.py's
+# _is_caret_escaped() docstring for the full rule this group regression-tests.
+def test_extract_records_keeps_call_log_with_doubled_caret_separator(tmp_path):
+    # cmd.exe caret parity: '^^&' is a literal '^' followed by an ACTIVE '&' (the first caret
+    # escapes the second, leaving the '&' unescaped) -- so this is a real command separator, and
+    # the later command's redirect must not cause the call :log record to be dropped.
+    bat = tmp_path / 'run_setup.bat'
+    bat.write_text('call :log "[INFO] visible" ^^& echo hidden > log.txt\n', encoding='ascii')
+    records = acm.extract_records(bat)
+    assert records == [(1, '[INFO] visible')]
+
+
+def test_extract_records_keeps_call_log_with_doubled_caret_pipe_separator(tmp_path):
+    # Same parity rule for '|': '^^|' is a literal '^' followed by an ACTIVE '|'.
+    bat = tmp_path / 'run_setup.bat'
+    bat.write_text('call :log "[INFO] visible" ^^| echo hidden > log.txt\n', encoding='ascii')
+    records = acm.extract_records(bat)
+    assert records == [(1, '[INFO] visible')]
+
+
+def test_extract_records_skips_call_log_with_single_caret_escaped_ampersand(tmp_path):
+    # Contrast case: a SINGLE caret ('^&') genuinely escapes the '&' into literal text -- there
+    # is no real separator here, so the whole line (including its trailing redirect) is one
+    # command and the call :log record is correctly dropped.
+    bat = tmp_path / 'run_setup.bat'
+    bat.write_text('call :log "[INFO] hidden" ^& echo also_hidden > log.txt\n', encoding='ascii')
+    records = acm.extract_records(bat)
+    assert records == []
+
+
 def test_extract_records_skips_call_log_with_redirect_in_same_segment(tmp_path):
     # A redirect immediately after the closing quote, with no intervening command
     # separator, DOES belong to this call :log -- it must still be skipped.

tools/README.md

@@ -0,0 +1,40 @@
+# tools/
+
+This directory is mixed-purpose. Most of it is load-bearing -- do not assume anything here is
+low-stakes without checking which category a given file falls into.
+
+## Most files here need full review rigor
+
+- **CI-wired scripts**, invoked directly by `.github/workflows/*.yml` (e.g. `apply_patch.py`,
+  `check_ndjson_registry.py`, `check_workflows_yaml.py`, `inline_model_fix.py`, `parse_warn.py`,
+  `prep_requirements.py`, `diag/ndjson_fail_list.py`, `diag/publish_index.py`) or that gate CI
+  decisions even if not directly listed in a workflow step (`iterate_gate.ps1` -- "the active gate
+  used by CI"; `sanitize_iterate_payload.py` -- defines the current NDJSON contract).
+- **Canonical sources for `run_setup.bat`'s embedded `HP_*` payloads** (e.g. `pyproj_deps.py`,
+  `prep_requirements.py`, `collect_submodules.py`, `hidden_import_scan.py`, `dep_check.py`,
+  `env_state.py`, `detect_python.py`, `detect_visa.py`, `find_entry.py`, `embed_extract.ps1`,
+  `embed_pyver_check.py`, `exe_smokerun.ps1`, `failfast_probe.ps1`, `run_installer_with_timeout.ps1`,
+  `pvw_known_idempotent.py`, `autopep_merge.py`, `pep723_writeback.py`). These ship as part of the
+  actual deliverable (base64-embedded in `run_setup.bat`) -- arguably higher stakes than a CI
+  script, not lower. Update via `tools/sync_payload.py`, never by hand-editing the embedded
+  base64 (see `docs/agent-lessons-learned.md`'s "Embedded Helper Update Workflow").
+- **Required dev-workflow tooling**, run before every commit even though no `.yml` file calls it
+  directly: `check_delimiters.py`, `run_sanity_sweep.sh`, `ps-compileall.ps1`, `sync_payload.py`.
+  See CLAUDE.md's "Mandatory Sanity Checks" section.
+- Everything else not explicitly named below.
+
+## Exceptions: manual, hand-run, not required by any flow
+
+These two are genuinely informal -- standalone diagnostic/audit scripts, run by hand on demand,
+not invoked by CI, not a canonical payload source, and not part of any required pre-commit check.
+Normal-rigor review asks (type annotations, `# derived requirement:` comment tags, etc.) don't
+apply the same way here; use judgment rather than treating every linter nitpick as actionable.
+
+- `audit_console_messages.py` -- own docstring: "Not wired into CI -- run by hand when
+  re-reviewing docs/demo-bootstrapper-output.md... or after a batch of new lines."
+- `audit_batch_exit_paths.py` -- a one-off audit tool for tracing `run_setup.bat`'s exit paths;
+  own module docstring notes it is "not a final answer on its own."
+
+If you add a new script that's genuinely in this category, say so explicitly in its own
+docstring/header (matching the two examples above) and add it to this list in the same commit --
+don't rely on this README alone to establish that later.

tools/audit_console_messages.py

@@ -36,31 +36,63 @@
 import re
 import sys
 from pathlib import Path
+from typing import Optional
 
 TEST_ONLY_RE = re.compile(
     r'HP_TEST|\[TEST\]|simulating|inject(ed|ing)|corrupt_conda|corrupt_uv',
     re.IGNORECASE,
 )
 
-# A real redirect operator: one or two literal '>' not preceded by a caret escape (a caret
-# protects an INNER command's own redirect, e.g. the `2^>nul` on a nested `powershell -Command`
-# call, from being read as this line's own redirect) and not immediately followed by '=' (so a
+# A candidate redirect token: one or two literal '>' not immediately followed by '=' (so a
 # genuine console message containing '>=' -- e.g. "running (>=30 days since last update)" --
-# is never mistaken for a redirect).
-REDIRECT_RE = re.compile(r'(?<!\^)>{1,2}(?!=)')
-
-# A real command separator (& / | / && / ||), not caret-escaped. A redirect appearing AFTER one
-# of these belongs to a later, separately-chained command -- it must not cause the FIRST
-# command's own record (e.g. a `call :log` that already reached the console before the
-# separator) to be dropped.
-SEGMENT_END_RE = re.compile(r'(?<!\^)[&|]')
+# is never mistaken for a redirect). Caret-escape state is NOT checked by the regex itself --
+# see _is_caret_escaped() below -- a single trailing (?<!\^) lookbehind cannot express cmd.exe's
+# parity rule (^X is escaped, ^^X is a literal caret followed by an UNESCAPED X).
+_REDIRECT_TOKEN_RE = re.compile(r'>{1,2}(?!=)')
+
+# A candidate command separator (& / | / && / ||). Same caret-escape caveat as above.
+_SEGMENT_TOKEN_RE = re.compile(r'[&|]')
+
+
+# derived requirement: cmd.exe caret-escape parity is not a one-char lookaround -- ^X escapes
+# X, but ^^X is a literal caret followed by an UNESCAPED X (the first caret escapes the
+# second). A naive `(?<!\^)` regex lookbehind mistook every ^^-prefixed separator/redirect for
+# an escaped one, silently dropping call :log records whose real command boundary sat right
+# after a doubled caret (found via CodeRabbit review on PR #406/#407; see
+# docs/agent-lessons-learned.md's caret-escaping entries for the general hazard class).
+def _is_caret_escaped(text: str, pos: int) -> bool:
+    """True if the character at `pos` is caret-escaped, per cmd.exe's parity rule: a caret
+    escapes the NEXT character, and a caret itself is escaped by a preceding caret. So an ODD
+    number of consecutive carets immediately before `pos` means `text[pos]` is escaped (^X);
+    an EVEN number (including zero) means it is not -- e.g. ^^& is a literal caret followed by
+    an ACTIVE '&', not an escaped one. A single-character regex lookbehind cannot express this
+    for an arbitrary run of carets, so it is checked by explicit backward scan instead."""
+    carets = 0
+    i = pos - 1
+    while i >= 0 and text[i] == '^':
+        carets += 1
+        i -= 1
+    return carets % 2 == 1
+
+
+def _first_unescaped(pattern: re.Pattern, text: str) -> Optional[re.Match]:
+    """Return the first match of `pattern` in `text` whose start position is not
+    caret-escaped, or None."""
+    for m in pattern.finditer(text):
+        if not _is_caret_escaped(text, m.start()):
+            return m
+    return None
+
+
+def _has_unescaped_redirect(text: str) -> bool:
+    return _first_unescaped(_REDIRECT_TOKEN_RE, text) is not None
 
 
 def _own_command_segment(tail: str) -> str:
-    """Truncate `tail` at the first real command separator, so a redirect belonging to a
-    later chained command is never mistaken for one that applies to the command being
-    checked."""
-    m = SEGMENT_END_RE.search(tail)
+    """Truncate `tail` at the first real (unescaped) command separator, so a redirect
+    belonging to a later chained command is never mistaken for one that applies to the
+    command being checked."""
+    m = _first_unescaped(_SEGMENT_TOKEN_RE, tail)
     return tail[:m.start()] if m else tail
 
 
@@ -95,7 +127,7 @@ def extract_records(bat_path: Path):
             # is indistinguishable from message content by position alone -- scan the whole raw
             # line. No current echo message contains a real (non-'>=', non-caret-escaped) '>',
             # confirmed by a full-file sweep at the time this check was added.
-            if REDIRECT_RE.search(raw):
+            if _has_unescaped_redirect(raw):
                 continue
             records.append((i, normalize(body)))
             continue
@@ -107,7 +139,7 @@ def extract_records(bat_path: Path):
             # inside the matched group, never scanned here. Truncate at the first real command
             # separator first: a redirect on a LATER, `&`-chained command (e.g. `call :log "..."
             # & echo hidden > file`) does not mean call :log itself was redirected.
-            if REDIRECT_RE.search(_own_command_segment(line[m.end():])):
+            if _has_unescaped_redirect(_own_command_segment(line[m.end():])):
                 continue
             records.append((i, normalize(m.group(1))))
     return records