Switch EXE fast-path freshness check from mtime to content-hash (Item 39)

.github/workflows/batch-check.yml

@@ -627,6 +627,21 @@ jobs:
         run: |
           & tests\selfapps_warnfix_venv_repair.ps1
 
+      # CLAUDE.md Active Backlog Item 39: the EXE fast path's freshness check switched from
+      # mtime-only to a content-hash comparison, so a genuinely changed source file whose
+      # mtime is backdated (the ZIP/xcopy/robocopy delivery scenario) is no longer silently
+      # treated as fresh. uv lane only (non-gating for first landing -- the isolated
+      # tools/fast_check.ps1 script is already verified locally via real pwsh, but this is
+      # the first time the full :try_fast_exe -> :run_entry_smoke -> :success ->
+      # :write_fast_hash cycle runs end-to-end on real Windows CI). See
+      # selfapps_fastpath_hash.ps1's own header comment for the full mechanism.
+      - name: "Self-test: EXE fast path detects a backdated-mtime content change (uv lane only)"
+        if: ${{ matrix.mode == 'uv' }}
+        continue-on-error: true
+        shell: pwsh
+        run: |
+          & tests\selfapps_fastpath_hash.ps1
+
       # REQ-005.11: PEP 723 header write-back (uv add --script). v1-scoped to HP_ENV_MODE=uv,
       # so wired to the dedicated uv lane only (non-gating) for this first CI pass rather than
       # the gating real lane -- lets these brand-new scenarios prove out in real Windows CI
@@ -1893,6 +1908,16 @@ jobs:
             tests\~selftest_exefastpath\~exefastpath_run2.log
             tests/~selftest_exefastpath/~setup.log
             tests\~selftest_exefastpath\~setup.log
+            tests/~selftest_fastpath_hash/~fastpath_hash_run1.log
+            tests\~selftest_fastpath_hash\~fastpath_hash_run1.log
+            tests/~selftest_fastpath_hash/~fastpath_hash_run2.log
+            tests\~selftest_fastpath_hash\~fastpath_hash_run2.log
+            tests/~selftest_fastpath_hash/~setup.log
+            tests\~selftest_fastpath_hash\~setup.log
+            tests/~selftest_fastpath_hash/~run.out.txt
+            tests\~selftest_fastpath_hash\~run.out.txt
+            tests/~selftest_fastpath_hash/~fast_check.hash.txt
+            tests\~selftest_fastpath_hash\~fast_check.hash.txt
             tests/~pandas_excel/~pandas_excel.log
             tests\~pandas_excel\~pandas_excel.log
             tests/~pipgap/~pipgap.log

CLAUDE.md

@@ -865,6 +865,96 @@ but several represent real gaps worth closing before calling the path fully rele
   **Coverage gap to close in the same slice**: no scenario backdates a source file's mtime below
   the EXE's to test this. Add one.
 
+  **Implemented: freshness is now a content-hash comparison, closing both exposures at once.**
+  `HP_FAST_CHECK` (canonical source `tools/fast_check.ps1`, previously inline-only with no
+  `tools/` file) now hashes the SAME `*.py` file set already scanned (unchanged filter), extended
+  to include `requirements.txt`/`pyproject.toml`/`runtime.txt` when present -- a combined SHA256
+  over each file's own `relpath|filehash` line, sorted by path for determinism. This structurally
+  cannot be fooled by a preserved/backdated mtime (exposure a) and now sees dependency-file
+  changes regardless of mtime (exposure b), closing both suggested fix directions at once rather
+  than picking one. Two modes via a second positional arg: `check` (default, called from
+  `:try_fast_exe`, unchanged call site) prints `fresh` iff a stored `~fast_check.hash.txt` exists
+  and matches; a missing stored hash (first run under this fix, or an EXE built by an older
+  `run_setup.bat`) is a safe, one-time-cost "not fresh" default, never a false "fresh." `write`
+  (new subroutine `:write_fast_hash`, called from `:success`) unconditionally (re)writes the
+  hash after a genuine fresh build attempt. Uses `[System.Security.Cryptography.SHA256]`
+  directly, not `Get-FileHash`, per this repo's own "Prefer raw .NET types over Utility-module
+  cmdlets" lesson (Utility-module auto-load is not guaranteed in the `-File` invocation shape
+  embedded helpers run under on real Windows PowerShell 5.1, invisible to local `pwsh`/Linux
+  testing).
+
+  **A real bug in the first-shipped `write` gate, found by CodeRabbit's review of this same
+  PR (#460) before landing.** The original gate was `if not defined HP_FASTPATH_USED call
+  :write_fast_hash` -- but `HP_FASTPATH_USED` being unset does not prove a build actually
+  SUCCEEDED this run; a skipped or FAILED rebuild also leaves it unset, and if a stale
+  `dist\<env>.exe` from an EARLIER successful run was still sitting there untouched, this gate
+  would write the CURRENT (changed) sources' hash paired against that OLD binary -- the next
+  run would then wrongly trust the stale EXE as "fresh," silently reintroducing the exact class
+  of bug this item exists to fix, just relocated one layer down. Fixed with a new, more precise
+  flag, `HP_FRESH_BUILD_OK` -- reset once per fresh build attempt (alongside
+  `HP_NUITKA_FALLBACK_USED` in `:run_entry_smoke`'s own init block) and set ONLY in the 4
+  genuine build-success branches (PyInstaller producing `dist\%ENVNAME%.exe` cleanly, or Tier
+  A/Nuitka succeeding after PyInstaller didn't, across all 3 of its own failure sub-branches) --
+  never on any `:warn_build_incomplete` path. `HP_FRESH_BUILD_OK` being set already implies the
+  fast path was not taken (a build can only be attempted after the fast path declines to fire),
+  so this is a strictly more precise REPLACEMENT for the old `HP_FASTPATH_USED` check, not an
+  additional condition alongside it -- `:success` now gates on `if defined HP_FRESH_BUILD_OK`
+  alone. A later repair-loop rebuild (`:hidden_import_recover`/`:dll_bundle_recover`) failing
+  after the initial build already succeeded does NOT unset the flag -- reasoned as correct,
+  since those loops fix BUNDLING issues, not SOURCE CONTENT, so the already-built EXE from the
+  same run still genuinely reflects the current sources even if a later repair attempt fails;
+  `:write_fast_hash`'s own pre-existing `if not exist "dist\%ENVNAME%.exe" exit /b 0` guard
+  remains the correct backstop for the rarer case where a repair-loop failure actually removes
+  the file.
+
+  **Coverage gap closed**: `tests/selfapps_fastpath_hash.ps1` (uv lane, non-gating for first
+  landing) is the literal scenario this item asked for -- run 1 builds the EXE printing a V1
+  token, and asserts `~fast_check.hash.txt` was genuinely written before proceeding (a
+  CodeRabbit-review addition: without this, a completely broken write side would be
+  indistinguishable from a working one, since a missing hash file also safely forces a rebuild);
+  entry.py is rewritten to print V2 and its mtime backdated to 2001-09-09 (well before the EXE's
+  own mtime) before run 2; asserts run 2's captured EXE stdout shows V2 (not V1) and the "Fast
+  path: reusing" line is absent, proving a genuine rebuild happened rather than the old
+  mtime-only check wrongly reusing the stale EXE. `tests/test_fast_check.py` unit-tests the
+  isolated script directly via real `pwsh` (6 scenarios: no stored hash, write-then-fresh,
+  backdated-mtime content change, dependency-file-only change, rewrite-after-change, missing
+  EXE) plus a `PayloadSync` byte-equality check against `tools/fast_check.ps1`.
+  `tests/harness.ps1`'s `batch.fastpath.hash_write` statically guards the `:write_fast_hash`
+  wiring -- label exists, called from `:success` with the `HP_FRESH_BUILD_OK` gate, invokes the
+  payload in `write` mode -- via SCOPED body extraction (bounded by each subroutine's own next
+  label, another CodeRabbit-review fix: the original check used whole-file `-match`, which could
+  pass even if the call or the write invocation were removed, as long as matching text happened
+  to exist anywhere else in the file). See `docs/agent-ndjson.md` for the full row registry entry.
+
+  **A second CodeRabbit review round on the same PR caught four more findings, all fixed before
+  landing.** (1) A defensive `set "HP_FRESH_BUILD_OK="` was added at the very top of the file
+  (alongside the pre-existing `HP_FASTPATH_USED=` reset, before the first `:try_fast_exe` call of
+  the whole run) -- `:run_entry_smoke`'s own per-build-attempt reset sits past a preflight-failure
+  early-return (`HP_PREFLIGHT_FAILED`), so a provider-cascade re-entry hitting that early return
+  would skip the in-subroutine reset; no concrete exploit was confirmed against the current
+  single-invocation call graph (every write site pairs a genuinely fresh build with unchanged
+  sources even across this path), but the fix is a zero-cost, unconditional close of the whole
+  class of risk regardless. (2) `:write_fast_hash` now captures the PowerShell write call's own
+  exit code and logs `[WARN] Fast-path hash write failed; the next run will rebuild.` on a nonzero
+  result, instead of silently deleting the helper and returning success either way -- matches this
+  repo's own "Bootstrap must fail fast and explicitly -- no silent fallbacks unless explicitly
+  logged" principle; still safe either way (a failed write leaves no stored hash, forcing exactly
+  one more rebuild), just no longer silent. (3) `tests/selfapps_fastpath_hash.ps1`'s
+  `hashWrittenAfterRun1` check was `Test-Path`-only (existence, not content) -- a truncated or
+  malformed write would pass the same way a working one would, since a broken file is just as
+  "not fresh" as a missing one to the read side. Now reads the file and validates it matches
+  `tools/fast_check.ps1`'s own canonical format (a 64-char lowercase hex SHA256 digest, no
+  trailing newline) via `hashContentValid`, folded into `$pass`. (4) The same test never confirmed
+  its own core precondition -- that `entry.py`'s backdated mtime assignment actually took, and is
+  genuinely older than the built EXE's own mtime -- before trusting run 2's outcome as evidence the
+  NEW hash-based path (not the OLD mtime-only path, coincidentally rebuilding for an unrelated
+  reason) did the detecting. Added `mtimePreconditionHolds` (compares `entry.py`'s and the built
+  EXE's `LastWriteTimeUtc`), also folded into `$pass`. `.github/workflows/batch-check.yml`'s
+  "Upload test logs" step gained both path-style variants for `~fast_check.hash.txt` under
+  `tests/~selftest_fastpath_hash/` (a new observable artifact this PR introduced, per this repo's
+  own coding guideline that every new artifact needs both slash-style upload paths) -- previously
+  only the run logs/`~setup.log`/`~run.out.txt` were wired.
+
 - **Item 42: console output is verbose across all log levels, and every fresh build ends with two
   unexplained Y/N prompts -- both plausibly overwhelming for the actual target audience
   (beginners with no setup experience).** Confirmed reasoned-from-source and CI capture. New; not

docs/agent-interconnect.md

@@ -1184,6 +1184,51 @@ individual test file pins it locally -- see the "Accepted gap" entry in
 path is completely provider-independent. The test correctly validates the EXE fast path, not
 the env-state or uv venv fast path.
 
+**`HP_FAST_CHECK`'s own freshness signal (CLAUDE.md Active Backlog Item 39): content-hash, not
+mtime -- read side and write side are two separate call sites, must stay paired.** `:try_fast_exe`
+(read/`check` mode, unchanged call site) compares a stored `~fast_check.hash.txt` against a
+composite SHA256 over the same non-infra `*.py` files already scanned, extended to
+`requirements.txt`/`pyproject.toml`/`runtime.txt` when present. **The write side is a SEPARATE
+subroutine, `:write_fast_hash`, called from `:success`** (not inlined there -- a set-then-read of
+the same var inside one parenthesized block would hit the classic parse-time-`%VAR%`-expansion
+trap, see `docs/agent-lessons-learned.md`), gated on `if defined HP_FRESH_BUILD_OK`.
+
+**`HP_FRESH_BUILD_OK`, not `HP_FASTPATH_USED`, is the correct gate -- a real bug CodeRabbit's
+review caught before this shipped.** The first-shipped gate was `if not defined HP_FASTPATH_USED`,
+reasoning "skip the redundant re-hash when the fast path already proved nothing changed." But
+`HP_FASTPATH_USED` being unset only proves the fast path was NOT taken -- it does NOT prove a
+build SUCCEEDED. A skipped or genuinely FAILED rebuild also leaves it unset, and if a stale
+`dist\<env>.exe` from an EARLIER successful run was still sitting there untouched, that gate would
+write the CURRENT (changed) sources' hash paired against the OLD binary -- the next run would then
+wrongly trust the stale EXE as "fresh," reintroducing Item 39's own bug one layer down. Fixed with
+`HP_FRESH_BUILD_OK`: reset once per fresh build attempt in `:run_entry_smoke`'s own init block
+(alongside `HP_NUITKA_FALLBACK_USED`), set ONLY in the 4 genuine build-success branches inside that
+subroutine's PyInstaller/Tier-A build block (PyInstaller producing `dist\%ENVNAME%.exe` cleanly, or
+Tier A/Nuitka succeeding after PyInstaller didn't, across all 3 of ITS OWN failure sub-branches) --
+never on any `:warn_build_incomplete` path. Since a build can only be ATTEMPTED after the fast path
+has already declined to fire, `HP_FRESH_BUILD_OK` being set strictly implies `HP_FASTPATH_USED` was
+never set either -- so this is a precision REPLACEMENT for the old gate, not an additional
+condition layered on top of it; `:success` now checks `HP_FRESH_BUILD_OK` alone.
+
+**A later repair-loop rebuild failing does NOT need to unset the flag, by design.**
+`:hidden_import_recover`/`:dll_bundle_recover` can each rebuild `dist\<env>.exe` again AFTER the
+initial build already succeeded and set `HP_FRESH_BUILD_OK=1` -- but these loops repair BUNDLING
+issues (a missing DLL, a missing hidden import), not SOURCE CONTENT, so even if a later repair
+attempt fails, the EXE from the initial successful build in the SAME run still genuinely reflects
+the current source snapshot the hash is computed from. `:write_fast_hash`'s own pre-existing
+`if not exist "dist\%ENVNAME%.exe" exit /b 0` guard remains the correct backstop for the rarer case
+where a repair-loop failure actually removes the file rather than merely leaving it imperfect.
+
+**Any future change to either the scanned file set or the hash algorithm must update BOTH
+`:try_fast_exe`'s check invocation and `:write_fast_hash`'s write invocation identically** -- they
+share the single `tools/fast_check.ps1` script (mode-dispatched via a second positional arg), so a
+change to the shared script's file enumeration or hashing logic automatically stays in sync between
+the two call sites; only a change to the CALL SITES themselves (e.g. adding a new mode, or passing
+different exe/cwd context) risks the two drifting apart. A missing stored hash (first run after
+upgrading to this fix, or an
+EXE built by an older `run_setup.bat` with no hash file at all) is a safe "not fresh" default --
+forces exactly one rebuild, never a false "fresh."
+
 ### ~dependency_installed.txt: pip freeze output and its consumers
 
 `~dependency_installed.txt` is written after install via `pip freeze` (run_setup.bat lines 1122-1134):

docs/agent-ndjson.md

@@ -197,6 +197,7 @@ batch.conda.probe.deferred,
 uv.python.preference.configured,
 batch.script_root.trailing_backslash,
 batch.progress.conda_create, batch.progress.pyi_build, batch.progress.dep_install,
+batch.fastpath.hash_write,
 batch.pyi.collect.precheck,
 batch.pyi.hidden_import.recover,
 batch.smoke.kill_warn,
@@ -991,6 +992,37 @@ Non-gating for its first landing, matching this repo's established graduation pa
 self.exe.warnfix.venv_repair
 ```
 
+## selfapps-fastpath-hash NDJSON rows (selfapps_fastpath_hash.ps1, uv lane only, non-gating)
+
+CLAUDE.md Active Backlog Item 39: the EXE fast path's freshness check (`HP_FAST_CHECK`,
+`tools/fast_check.ps1`) switched from mtime-only over `*.py` files to a content-hash
+comparison over the same file set, extended to `requirements.txt`/`pyproject.toml`/
+`runtime.txt`. Closes two exposures: (a) a timestamp-preserving delivery method (a ZIP,
+xcopy, robocopy) carrying a genuinely changed file whose mtime still predates the built
+EXE was previously silently treated as fresh; (b) a dependency-file-only change was
+previously invisible to the scan regardless of mtime. `:write_fast_hash` (called from
+`:success`, gated on `HP_FASTPATH_USED` being unset so the already-fast reuse case never
+pays a redundant re-hash pass) writes the stored hash after a genuine fresh build attempt.
+
+This test is the coverage-gap Item 39 itself named ("no scenario backdates a source file's
+mtime below the EXE's to test this"): run 1 builds the EXE from an entry file printing a V1
+token; entry.py is then rewritten to print a V2 token and its mtime is set to 2001-09-09
+(well before `dist\<env>.exe`'s own mtime) before run 2. Asserts run 2's captured EXE
+stdout (`~run.out.txt`, written by both the fast-path-reuse launch and the fresh-build
+smokerun) shows the V2 token, not V1 -- direct evidence a genuine rebuild happened, not
+just that some log line says "rebuilding" -- and that the "Fast path: reusing" log line is
+absent. The isolated `tools/fast_check.ps1` script itself is unit-tested directly via real
+`pwsh` in `tests/test_fast_check.py` (6 scenarios: no stored hash, write-then-fresh,
+backdated-mtime content change, dependency-file-only change, rewrite-after-change, missing
+EXE) -- this selfapps test is what proves the full `:try_fast_exe -> :run_entry_smoke ->
+:success -> :write_fast_hash` cycle wires correctly end-to-end on real Windows CI, which
+`test_fast_check.py` alone cannot. Non-gating for its first landing, matching this repo's
+established graduation pattern (see CLAUDE.md's "CI lane gating maturity" periodic check).
+
+```
+self.fastpath.hash.backdated_mtime
+```
+
 ## selfapps-cache-selfheal NDJSON rows (test_ci_cache_selfheal.ps1, `real` lane only, GATING)
 
 Item 19 follow-on (docs/agent-closed-backlog.md): the cache-lane self-heal logic