Skip to content
Merged
85 changes: 85 additions & 0 deletions .github/workflows/batch-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,27 @@ jobs:
run: |
& tests\selfapps_isolation.ps1

# CLAUDE.md Active Backlog Item 46 Bucket A Batch 5: :ci_skip_entry's own
# ~find_entry.py staging failure used to fall through call :die into a redundant second
# call :die ("find_entry helper syntax error") for the same root cause. Cheap
# (HP_CI_SKIP_ENV path, no conda/uv needed) so runs in every lane, matching the
# HP_CI_SKIP_ENV-based steps directly above.
# derived requirement: per-step continue-on-error, not just the job-level one -- see the
# identical note above the PEP 723 write-back steps (this same file, "REQ-005.11: PEP 723
# header write-back") for why: without it, a genuine failure here silently skips every
# later step in the SAME job on a non-gating lane. Scoped to non-gating lanes only
# (matches matrix.mode != 'real' && matrix.mode != 'conda-full', the complement of the
# job-level continue-on-error list on the job definition itself) -- a real failure on the
# two gating lanes must still fail the job.
- name: "Self-test: die-emit-fallthrough ci_skip_entry (CI-only)"
if: ${{ !cancelled() && env.HP_CACHE_CORRUPTED != '1' }}
continue-on-error: ${{ matrix.mode != 'real' && matrix.mode != 'conda-full' }}
env:
DIE_EMIT_SCENARIO: ci_skip_entry
shell: pwsh
run: |
& tests\selfapps_die_emit_fallthrough.ps1

Comment thread
coderabbitai[bot] marked this conversation as resolved.
- name: "Self-test: env-name sanitization (leading hyphen)"
if: ${{ !cancelled() && env.HP_CACHE_CORRUPTED != '1' }}
shell: pwsh
Expand Down Expand Up @@ -1105,6 +1126,46 @@ jobs:
run: |
& tests\selfapps_entrysmoke_no_interpreter.ps1

# CLAUDE.md Active Backlog Item 46 Bucket A Batch 2: :conda_create_done's own
# "python.exe missing" check (the NON-cascade path, distinct from selfapps_cascade_
# conda_create_fail.ps1's own missing_python scenario) used to fall through call :die into
# the rest of its body, logging a misleading "[BOOT] ... Selected Python provider: Conda
# (Portable)." success line right after the error. Self-contained (own
# HP_FORCE_CONDA_ONLY=1 + HP_TEST_FORCE_CONDA_MISSING_PYTHON=1), gating from first landing
# (same reasoning as the step directly above).
- name: "Self-test: die-emit-fallthrough missing_python (conda-full only)"
if: ${{ !cancelled() && matrix.mode == 'conda-full' && steps.conda_avail.outputs.available == 'true' }}
env:
DIE_EMIT_SCENARIO: missing_python
shell: pwsh
run: |
& tests\selfapps_die_emit_fallthrough.ps1

# CLAUDE.md Active Backlog Item 46 Bucket A Batch 4: :conda_create_done's own .condarc
# staging failure used to fall through call :die into a doomed copy attempt, triggering a
# SECOND, redundant call :die plus the same misleading success line as the scenario above.
# New hook: HP_TEST_FORCE_EMIT_FAIL=<VARNAME> (run_setup.bat, :emit_from_base64) fails one
# specific embedded-helper write deterministically.
- name: "Self-test: die-emit-fallthrough condarc (conda-full only)"
if: ${{ !cancelled() && matrix.mode == 'conda-full' && steps.conda_avail.outputs.available == 'true' }}
env:
DIE_EMIT_SCENARIO: condarc
shell: pwsh
run: |
& tests\selfapps_die_emit_fallthrough.ps1

# CLAUDE.md Active Backlog Item 46 Bucket A Batch 3: :determine_entry runs twice per normal
# bootstrap; a genuine failure on the FIRST call used to fall through call :die into the
# entire dependency-install/pipreqs/warnfix block (real work, not just a redundant message)
# before reproducing the identical failure at the second call site.
- name: "Self-test: die-emit-fallthrough determine_entry (conda-full only)"
if: ${{ !cancelled() && matrix.mode == 'conda-full' && steps.conda_avail.outputs.available == 'true' }}
env:
DIE_EMIT_SCENARIO: determine_entry
shell: pwsh
run: |
& tests\selfapps_die_emit_fallthrough.ps1

- name: "Test pandas/openpyxl heuristic (conda-full only)"
if: ${{ !cancelled() && matrix.mode == 'conda-full' && steps.conda_avail.outputs.available == 'true' }}
shell: pwsh
Expand Down Expand Up @@ -2280,6 +2341,30 @@ jobs:
tests\~selftest_entrysmoke_no_interpreter\~setup.log
tests/~selftest_entrysmoke_no_interpreter/~bootstrap.status.json
tests\~selftest_entrysmoke_no_interpreter\~bootstrap.status.json
tests/~selftest_die_emit_missing_python/~die_emit_missing_python_bootstrap.log
tests\~selftest_die_emit_missing_python\~die_emit_missing_python_bootstrap.log
tests/~selftest_die_emit_missing_python/~setup.log
tests\~selftest_die_emit_missing_python\~setup.log
tests/~selftest_die_emit_missing_python/~bootstrap.status.json
tests\~selftest_die_emit_missing_python\~bootstrap.status.json
tests/~selftest_die_emit_condarc/~die_emit_condarc_bootstrap.log
tests\~selftest_die_emit_condarc\~die_emit_condarc_bootstrap.log
tests/~selftest_die_emit_condarc/~setup.log
tests\~selftest_die_emit_condarc\~setup.log
tests/~selftest_die_emit_condarc/~bootstrap.status.json
tests\~selftest_die_emit_condarc\~bootstrap.status.json
tests/~selftest_die_emit_ci_skip_entry/~die_emit_ci_skip_entry_bootstrap.log
tests\~selftest_die_emit_ci_skip_entry\~die_emit_ci_skip_entry_bootstrap.log
tests/~selftest_die_emit_ci_skip_entry/~setup.log
tests\~selftest_die_emit_ci_skip_entry\~setup.log
tests/~selftest_die_emit_ci_skip_entry/~bootstrap.status.json
tests\~selftest_die_emit_ci_skip_entry\~bootstrap.status.json
tests/~selftest_die_emit_determine_entry/~die_emit_determine_entry_bootstrap.log
tests\~selftest_die_emit_determine_entry\~die_emit_determine_entry_bootstrap.log
tests/~selftest_die_emit_determine_entry/~setup.log
tests\~selftest_die_emit_determine_entry\~setup.log
tests/~selftest_die_emit_determine_entry/~bootstrap.status.json
tests\~selftest_die_emit_determine_entry\~bootstrap.status.json
tests/extracted/**
if-no-files-found: ignore
include-hidden-files: true
Expand Down
26 changes: 24 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,10 +135,17 @@ pwsh -c "
\$ErrorActionPreference = 'Stop'
\$fail = 0
Get-ChildItem /home/user/Python_vs_Windows/tests/*.ps1, /home/user/Python_vs_Windows/tools/*.ps1 | ForEach-Object {
try { [System.Management.Automation.Language.Parser]::ParseFile(\$_.FullName, [ref]\$null, [ref]\$null) | Out-Null }
catch { Write-Host \"PARSE FAIL: \$(\$_.FullName): \$_\"; \$fail = 1 }
\$errs = \$null
try { [System.Management.Automation.Language.Parser]::ParseFile(\$_.FullName, [ref]\$null, [ref]\$errs) | Out-Null }
catch { Write-Host \"PARSE FAIL: \$(\$_.FullName): \$_\"; \$fail = 1; return }
if (\$errs.Count -gt 0) {
Write-Host \"PARSE FAIL: \$(\$_.FullName):\"
\$errs | ForEach-Object { Write-Host \" \$_\" }
\$fail = 1
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}
if (\$fail -eq 0) { Write-Host 'PS PARSE SWEEP DONE - ALL CLEAN' }
exit \$fail
"
python -m pytest /home/user/Python_vs_Windows/tests/test_*.py -q 2>&1 | tail -5
```
Expand Down Expand Up @@ -1129,6 +1136,21 @@ way (no live Windows execution available here), that is noted explicitly rather
a time, full 8-lane matrix CI proof to completion before the next batch starts, no blanket sweep
across every remaining site in one PR.

**Batch 1 merged 2026-08-28 (PR #468). Batches 2/3/5 and one site of Batch 4 landed as a
follow-on PR stacked on Batch 1, same session.** Batch 4's scope corrected from 7 sites to 1
(the `Could not stage ~condarc` site only -- the other 6 traced individually and found to fall
through into benign, silently-degraded continuations, not a redundant-`:die` cascade;
reclassified alongside the "Active Python interpreter not resolved" sink, not fixed). Batch 6
traced further (a second `:try_conda_install` call site found inside `:cascade_acquire_conda`)
and remains deferred -- Batch 1's own fix already shrinks its residual value to "one fewer
redundant pause in an already-rare scenario." New shared test hook
`HP_TEST_FORCE_EMIT_FAIL=<VARNAME>` added to `:emit_from_base64` itself. New test file
`tests/selfapps_die_emit_fallthrough.ps1` (4 scenarios:
`missing_python`/`condarc`/`ci_skip_entry`/`determine_entry`) also surfaced a genuine, pre-
existing, unfixed quirk: `:after_env_skip` writes `state=ok` unconditionally regardless of an
earlier `call :die` in the same run (near-zero exposure, `HP_CI_SKIP_ENV` is test-infrastructure-
only). Full detail: `docs/plan-die-fatal-remediation.md`'s "Implementation Status" section.

## Cold Storage (promising ideas, deliberately shelved -- revisit only if a named trigger fires)

Moved to `docs/agent-cold-storage.md` (2026-07-31, to reduce this file's per-session context
Expand Down
29 changes: 29 additions & 0 deletions docs/agent-interconnect.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,21 @@ doesn't leave stale locks for other users of the same folder.

---

## `:emit_from_base64` (shared by ~10+ call sites) now carries a test-injection hook -- any future call site inherits it automatically

CLAUDE.md Active Backlog Item 46 Bucket A Batches 2/3/4/5 (`docs/plan-die-fatal-remediation.md`)
added `HP_TEST_FORCE_EMIT_FAIL=<VARNAME>` to `:emit_from_base64` itself (the one subroutine every
embedded-helper write in the file funnels through -- `~detect_python.py`, `~condarc`,
`~print_pyver.py`, `~prep_requirements.py`, `~find_entry.py`, `~detect_visa.py`, and any future
addition). Checked FIRST, before the real PowerShell write: if defined and equal to the CALL's own
`%VAR%` argument, logs a `[TEST]` line and `exit /b 1` immediately, simulating a genuine disk-write/
AV-lock failure for that ONE payload without touching any other payload emitted in the same run.
REQ-019-compliant (absence changes nothing). Any future new embedded helper (a new `HP_*` payload
variable) automatically gains this same test-injection capability with zero additional wiring --
worth knowing before adding a bespoke failure-simulation mechanism for a new helper's write step.

---

## AV-Safe Build Path Tier A (`:try_nuitka_tier_a`) and hidden-import auto-recovery

**Touch either subroutine, must understand the other.** `:try_nuitka_tier_a` (AV-Safe Build Path
Expand Down Expand Up @@ -1096,6 +1111,20 @@ fall-through blast radius as a side effect, even before Batch 6 itself is touche
this fix) now asserts the resulting cascade collapses to exactly this chain's first site plus the
sink, not the full 5-6-pause worst case.

**Status: implemented, landed as a follow-on PR stacked on Batch 1 (2026-08-28).** A second
`:try_conda_install` call site was found while tracing Batch 6 (the `:tci_both_failed` failure
sites themselves) -- `:cascade_acquire_conda` (the REQ-009 uv-to-conda cascade's own on-demand
Miniconda acquisition) calls it independently of the main install-if-missing block (inside
`:uv_first_skip`). Its OWN fall-through does NOT re-enter this probe chain at all (that chain is
specific to the first-attempt, non-cascade path, reached only from `:uv_first_skip`'s own call
site) -- it proceeds fairly directly to a
conda-create attempt that already `goto`s correctly (slice 1, pre-existing). This means Batch 1's
fix only reduces THIS section's own described blast-radius reduction for the FIRST call site;
`docs/plan-die-fatal-remediation.md`'s "Implementation Status" section has the full trace for both
call sites and why Batch 6 (a caller-side coordination flag) remains deferred as a result -- its
now-confirmed remaining value is "one fewer redundant pause in an already-rare scenario," smaller
than originally scoped.

Regression test: `tests/selfapps_entrysmoke_no_interpreter.ps1` (`self.entrysmoke.no_interpreter_
guard`, `conda-full` lane) -- asserts `[ERROR] python.exe missing from conda environment.` is
ABSENT (proving the second `:handle_conda_failure` call is skipped) and the new
Expand Down
35 changes: 35 additions & 0 deletions docs/agent-lessons-learned.md
Original file line number Diff line number Diff line change
Expand Up @@ -668,6 +668,41 @@ workflows.

---

## `"$var:text"` inside a double-quoted PowerShell string is a scope-qualified-variable parse error, not string interpolation followed by a literal colon

**Real bug that shipped and broke ALL 8 CI lanes of a PR (2026-08-28, `tests/selfapps_die_emit_
fallthrough.ps1`, PR #469).** `"die-emit-fallthrough $scenario: run_setup.bat not found"` looks
like ordinary interpolation ("the value of `$scenario`, then a literal colon") but PowerShell's
tokenizer instead reads `$scenario:` as an attempt at a SCOPE-QUALIFIED variable reference
(`$scope:name`, the same syntax as `$env:VAR`/`$global:VAR`) -- any bare `$name` immediately
followed by `:` and non-identifier text is a `ParserError`: "Variable reference is not valid. ':'
was not followed by a valid variable name character. Consider using ${} to delimit the name."
This is a PARSE-time error, so the entire `.ps1` FILE fails to load -- not a runtime exception in
one function, every test/row in the file silently never runs (an NDJSON row this file emits was
simply ABSENT from the CI artifact, not failing -- easy to misdiagnose as "the row wasn't wired
up" instead of "the file never parsed").

**Fix: `${scenario}:` (curly-brace delimiting), not `$scenario` + string concatenation.** Same
rule applies to ANY interpolated variable immediately followed by `:` in a double-quoted string.

**Why this passed local review and the sanity sweep before shipping: the sweep's own PS parse
check was silently discarding parse errors.** `[System.Management.Automation.Language.Parser]::
ParseFile($path, [ref]$tokens, [ref]$errors)` reports parse errors via the **`$errors`
out-parameter**, not via a thrown/catchable exception -- passing `[ref]$null` for that parameter
(as both `CLAUDE.md`'s inline sanity-sweep block and `tools/run_sanity_sweep.sh`'s
`check_ps_parse_sweep()` did) silently discards every parse error, and a `try/catch` around the
call does NOT catch them (only a catastrophic failure like a missing file throws). The sweep
reported "ALL CLEAN" on a file that could not actually be loaded by PowerShell. **Fixed in both
places**: capture into `[ref]$errs`, then check `$errs.Count -gt 0` after the call and print each
error. `tools/ps-compileall.ps1` (a separate, older tool in this repo) already did this correctly
via `[ref]$parseErrs` -- it was never the bug, only the two sanity-sweep copies were.

**Rule: any future `[...]::ParseFile(...)` call in this repo's tooling must capture and check the
errors out-parameter -- never pass `[ref]$null` for it.** If a third copy of this pattern is ever
added, apply the same fix immediately rather than trusting `try/catch` alone.

---

## Windows `move` onto an existing destination: atomic replace for FILES, silent NESTING for DIRECTORIES -- do not assume the same post-check works for both

Requirement 9's swap-verification fix (`:offer_optimized_build`, own section below) established a
Expand Down
52 changes: 52 additions & 0 deletions docs/agent-ndjson.md
Original file line number Diff line number Diff line change
Expand Up @@ -1275,6 +1275,58 @@ self.entrysmoke.no_interpreter_guard

---

## selfapps-die-emit-fallthrough NDJSON rows (selfapps_die_emit_fallthrough.ps1, mixed lanes, GATING)

CLAUDE.md Active Backlog Item 46 Bucket A, Batches 2/3/4/5 (`docs/plan-die-fatal-remediation.md`'s
"Batch Roadmap" -- the full 20-site trace). Four scenarios (`DIE_EMIT_SCENARIO` env var):
`missing_python` (Batch 2, `:conda_create_done`'s own "python.exe missing" check, the NON-cascade
path -- distinct from `self.cascade.conda_create_fail`'s own `missing_python` scenario, a REQ-009
cascade RE-ENTRY that never reaches this site at all), `condarc` (Batch 4, the same subroutine's
`.condarc`-staging failure), `ci_skip_entry` (Batch 5, `:ci_skip_entry`'s own `~find_entry.py`
staging failure), `determine_entry` (Batch 3, `:determine_entry`'s first of its two per-run calls).
All four emit the SAME literal id, `self.die_emit_fallthrough`, with `details.scenario` as the
discriminator -- matches `self.exe.build.xfail`'s own established multi-scenario precedent (see
`docs/agent-lessons-learned.md`'s "A multi-scenario PowerShell test's NDJSON `id` must stay a
literal string" entry for why a per-scenario id computed into a shared variable would silently
break `tools/check_ndjson_registry.py`'s static scan).

New hook: `HP_TEST_FORCE_EMIT_FAIL=<VARNAME>` (`run_setup.bat`, `:emit_from_base64`) fails ONE
specific embedded-helper write deterministically (matched against the subroutine's own `%VAR%`
argument, e.g. `HP_CONDARC` or `HP_FIND_ENTRY`) without touching any other payload emitted in the
same run -- used by `condarc`/`ci_skip_entry`/`determine_entry`. `missing_python` instead reuses
the already-existing `HP_TEST_FORCE_CONDA_MISSING_PYTHON` hook (see `self.cascade.conda_create_
fail`'s own header comment), just without any cascade env vars set.

Three of the four (`condarc`/`ci_skip_entry`/`determine_entry`) set `HP_SKIP_ENTRY_SMOKE=1` /
`HP_SKIP_EXE_SMOKERUN=1` -- purely about setup-flow fall-through, not a full PyInstaller build/run
cycle. `missing_python` deliberately does NOT set these: the point is letting `:run_entry_smoke`
actually run so CLAUDE.md Active Backlog Item 45's own "if not exist `%HP_PY%`" guard (already
proven by `self.entrysmoke.no_interpreter_guard` for a different trigger) gets a chance to catch
the broken-but-defined interpreter and confirms no PyInstaller build is attempted against it.

`ci_skip_entry` documents a real, pre-existing, NOT-fixed-by-this-change quirk found while writing
its own assertions: `:after_env_skip` (the label `:after_env_bootstrap` routes to under
`HP_CI_SKIP_ENV=1`) calls `call :write_status ok 0 %PYCOUNT%` unconditionally -- it does not read
`HP_BOOTSTRAP_STATE`, so this scenario's own status file reads `state=ok` even though a real
`call :die` fired earlier in the same run. This is not a regression from the Batch 5 fix (the OLD,
unfixed fall-through already reached this same unconditional-ok write further downstream, just
after more wasted work) -- asserted as the known, unchanged behavior, not silently ignored. Near-
zero real-world exposure either way (`HP_CI_SKIP_ENV` is test-infrastructure-only, never set by a
real user or the default bootstrap path).

Lane: `conda-full` only for `missing_python`/`condarc`/`determine_entry` (real conda create
needed, gated on `steps.conda_avail.outputs.available == 'true'`, same gate 28+ other conda-full-
only self-tests use); every lane for `ci_skip_entry` (no conda/uv dependency at all, matches
`selfapps_isolation.ps1`'s own sibling `HP_CI_SKIP_ENV`-based steps). Gating from first landing --
deterministic and self-contained, same reasoning as `self.entrysmoke.no_interpreter_guard`'s own
precedent.

```
self.die_emit_fallthrough
```

---

## Key facts for debugging missing rows

- `self.exe.hidden_import.exhaust` (CLAUDE.md Active Backlog item 11, `tests/selfapps_
Expand Down
Loading
Loading