diff --git a/CLAUDE.md b/CLAUDE.md index 3e42de1a..03a299e9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -973,21 +973,45 @@ way (no live Windows execution available here), that is noted explicitly rather to a python.exe that does not exist -> if not exist (call :die, pause #2) -> falls through -> more code using the broken HP_PY -> call :die (pause #3) -> ...` produces exactly the "hit pause several times" symptom reported, each pause looking like a fresh, unrelated failure rather than - one root cause cascading. - - **Candidate fix shapes, not yet chosen between**: (a) a global `HP_FATAL` flag set by `:die`, - checked via `if defined HP_FATAL goto :fatal_exit` after every one of the ~24 continuing call - sites; (b) change what `:die` itself does on exit (e.g. a real process-halting `exit`, not + one root cause cascading. **Item 45 (closed) already prevents this specific chain from reaching a + third pause** -- the `if not exist "%HP_PY%"` guard it added to `:run_entry_smoke` means a broken + `HP_PY` surviving past `:conda_create_done`'s own second `:die` call can no longer also trigger + the doomed PyInstaller build/warnfix/repair block's own `:die` sites afterward. The first two + pauses in that chain (both inside the conda-create-failure handling itself, before + `:after_env_mode_selection` is ever reached) are UNCHANGED by Item 45 and remain open scope here. + + **Bucket B (closed 2026-08-17): the 3 PyInstaller-build-failure `:die` sites + (`reason=test_forced_fail`/`build_error`/`missing_output`) migrated to a new, non-pausing sibling + subroutine, `:warn_build_incomplete`.** These are a genuinely different, NOT-doomed case from the + chain above: by the time any of them is reached, both PyInstaller and the Nuitka fallback have + already failed, but the environment/dependencies are still valid and the interpreter-fallback + verification a few hundred lines below still genuinely runs and still genuinely decides + success/failure -- pausing here (before that verification even happens) misleadingly looks like + the terminal state when real work is still ahead. `:warn_build_incomplete` (defined right after + `:die` in `run_setup.bat`) still sets `HP_BOOTSTRAP_STATE=error` (so the eventual final report + stays honest, matching `selfapps_pyinstaller_fail.ps1`'s unchanged `state=='error'` assertions), + but skips `:die`'s pause, premature `:release_lock`, and premature `:write_status` -- see + `docs/agent-lessons-learned.md`'s `:die` entry for the full mechanism. The vestigial, now-doubly- + redundant `set "HP_BOOTSTRAP_STATE=error"` lines that used to follow each `call :die` at these 3 + sites were removed in the same change (the subroutine itself sets it). Not every `:die` site + qualifies for this treatment -- only ones where genuinely useful, outcome-determining work still + follows; the two conda-create-failure pauses in the "Consequence" chain above are NOT one of + these (nothing useful follows a totally-exhausted provider chain with no valid interpreter at + all), so they remain candidates for Bucket A's `HP_FATAL` mechanism instead, not this pattern. + + **Bucket A (not yet started): a global `HP_FATAL` flag for the remaining, genuinely-doomed `:die` + sites** -- candidate fix shapes, not yet chosen between: (a) a global `HP_FATAL` flag set by + `:die`, checked via `if defined HP_FATAL goto :fatal_exit` after every remaining continuing call + site; (b) change what `:die` itself does on exit (e.g. a real process-halting `exit`, not `exit /b`) for the cases where it is known to be called from the top-level call stack rather than a nested subroutine -- riskier, since the top-level-vs-nested distinction is not always obvious from a given call site, and a bare `exit` closes the console window immediately for a double-click user with no chance to read the message first (see the existing `pause`-before-exit - convention this file already relies on); (c) do nothing beyond Item 45's narrower mitigation for - now, since it already kills the specific worst compounding case (repeated build/repair attempts) - even without touching `:die` itself. Given the number of call sites and `:die`'s central, - load-bearing role throughout the file, treat this as EXTREME CAUTION on the same order as the - DLL-bundling/hidden-import repair loops elsewhere in this backlog -- one incremental slice at a - time, not a single sweeping change across all 31 sites. + convention this file already relies on). Given the number of remaining call sites and `:die`'s + central, load-bearing role throughout the file, treat this as EXTREME CAUTION on the same order + as the DLL-bundling/hidden-import repair loops elsewhere in this backlog -- one incremental slice + at a time (Bucket B above is the first proof this works), not a single sweeping change across + every remaining site. - **Item 47: no PowerShell capability preflight beyond bare presence.** The new line-ending self-check (Item 44's mitigation) added a `where powershell` presence guard as its own diff --git a/docs/agent-lessons-learned.md b/docs/agent-lessons-learned.md index d5404c7b..9f28397d 100644 --- a/docs/agent-lessons-learned.md +++ b/docs/agent-lessons-learned.md @@ -477,6 +477,20 @@ list; the recurring traps that have actually bitten us: covers every existing AND future call site with one line, rather than an easy-to-forget companion `set` per site. Regression coverage: `tests/selfapps_pyinstaller_fail.ps1`, `self.embed.fallback.decline`/`self.ux.system.gate.real`-style tests asserting `state=='error'`. + **`:warn_build_incomplete` (CLAUDE.md Active Backlog Item 46, Bucket B, closed) is a + non-pausing sibling of `:die`, added right after it.** Same shape (`set "MSG=%~1"`, sets + `HP_BOOTSTRAP_STATE=error`), but deliberately skips `:die`'s three doomed-run-only actions -- + no `pause` (the run isn't over: real verification work still follows), no `:release_lock` + (a concurrent second instance must not start while that verification is still ahead), no + `:write_status` (`:after_cascade_decision` calls `:write_status` with whatever + `HP_BOOTSTRAP_STATE` holds by then, before `goto :success` dispatches to + `:print_no_exe_briefing` -- the briefing itself only prints, it writes nothing). Used at the 3 + PyInstaller-build-failure call sites (`reason=test_forced_fail`/`build_error`/`missing_output`) -- reached only after + the Nuitka fallback has ALSO failed, but the interpreter-fallback verification a few hundred + lines below still genuinely runs and still genuinely decides success/failure, so these are not + doomed the way most `:die` call sites are. **Not every `:die` site qualifies for this + treatment** -- see CLAUDE.md's own former Item 46 entry (now in `docs/agent-closed-backlog.md`) + for the doomed-vs-not-doomed classification method before converting any other site. **PowerShell adjacent traps:** `-or`/`-and` outside a conditional are parsed as parameter names ("parameter name 'or'"); `tools/check_delimiters.py` flags these. Multi-line `run:` diff --git a/docs/demo-bootstrapper-output.md b/docs/demo-bootstrapper-output.md index c7d4efc8..294dc0da 100644 --- a/docs/demo-bootstrapper-output.md +++ b/docs/demo-bootstrapper-output.md @@ -3107,18 +3107,20 @@ one's console text, since it's really a REQ-027 demo). The first two additionall #### 38a. `execfail` -- the PyInstaller build command itself fails -Real CI capture, run `29788624195`, job `88506013028` ("real" lane), with the `reason=` -token (CLAUDE.md Item 33, added after that capture) spliced in verbatim from `run_setup.bat`'s -current literal source -- the rest of the block is unmodified real console output. **This -scenario is triggered by two CI test hooks (forcing both the PyInstaller build and the Nuitka -fallback to fail), but a real user hitting the same genuine failure sees nothing at either -trigger point** -- both PyInstaller's and Nuitka's own error output go only to the log file, -never the console: +Real CI capture, run `29788624195`, job `88506013028` ("real" lane), with the `[WARN]`-prefixed +message line (CLAUDE.md Active Backlog Item 46, Bucket B -- this call site now routes through +:warn_build_incomplete instead of :die, since every build tool has already failed but the run +itself is not doomed, real verification work still follows) spliced in verbatim from +`run_setup.bat`'s current literal source -- the rest of the block is unmodified real console +output. **This scenario is triggered by two CI test hooks (forcing both the PyInstaller build +and the Nuitka fallback to fail), but a real user hitting the same genuine failure sees nothing +at either trigger point** -- both PyInstaller's and Nuitka's own error output go only to the log +file, never the console: ``` [INFO] Building standalone executable -- this may take a minute or two... [INFO] Standard build did not complete; attempting a fallback build (this may take a minute or two). -[ERROR] PyInstaller execution failed. reason=test_forced_fail +[WARN] PyInstaller execution failed; will verify your code directly via Python instead. reason=test_forced_fail [DEBUG] warnfix: warn file not found [INFO] PyInstaller build artifacts cleaned up. [WARN] EXE smokerun: dist\.exe not found; skipping @@ -3130,8 +3132,9 @@ never the console: ``` The `reason=test_forced_fail` token (CLAUDE.md Item 33) reflects the actual trigger here -- this -scenario reaches `:die` via the `HP_TEST_FORCE_PYINSTALLER_FAIL` test hook, never a genuine -PyInstaller error. A real, unforced build failure would instead read `reason=build_error` -- +scenario reaches `:warn_build_incomplete` via the `HP_TEST_FORCE_PYINSTALLER_FAIL` test hook, +never a genuine PyInstaller error. A real, unforced build failure would instead read +`reason=build_error` -- `[Extrapolated Branch]`, since no deterministic CI hook forces a genuine nonzero PyInstaller exit (the same "real trigger, no CI hook" situation as Tier A's own Nuitka compiler-failure hint, Part VIII above). @@ -3157,7 +3160,7 @@ code path that announces either one, so both are omitted below (see 38a's note f ``` [INFO] Building standalone executable -- this may take a minute or two... [INFO] Standard build did not complete; attempting a fallback build (this may take a minute or two). -[ERROR] PyInstaller did not produce dist\.exe reason=missing_output +[WARN] PyInstaller did not produce dist\.exe; will verify your code directly via Python instead. reason=missing_output [DEBUG] warnfix: warn file found [INFO] warnfix: some modules could not be automatically bundled (full list in ~warnfile.txt / ~setup.log); modules such as posix, fcntl, grp, pwd, resource, _scproxy, _posixsubprocess, collections.abc, and _frozen_importlib_external are expected on Windows and are filtered out automatically. [INFO] PyInstaller build artifacts cleaned up. diff --git a/run_setup.bat b/run_setup.bat index 8c187dc4..c1a47206 100644 --- a/run_setup.bat +++ b/run_setup.bat @@ -3581,8 +3581,16 @@ if not defined HP_BUILD_OK ( call :log "[TEST] HP_TEST_FORCE_PYINSTALLER_FAIL: simulating PyInstaller build failure." call :try_nuitka_tier_a if errorlevel 1 ( - call :die "[ERROR] PyInstaller execution failed. reason=test_forced_fail" - set "HP_BOOTSTRAP_STATE=error" + rem CLAUDE.md Active Backlog Item 46 (Bucket B): every build tool (PyInstaller AND the + rem Nuitka fallback) has already failed by this point, but the environment/dependencies + rem are still valid and the interpreter-fallback verification a few hundred lines below + rem still genuinely runs and still genuinely decides success/failure -- this is not a + rem doomed state, so :die's own mid-run pause (which would stop the user here, before + rem that verification even happens) and premature lock release are both wrong. Warn and + rem keep going instead; :warn_build_incomplete still sets HP_BOOTSTRAP_STATE=error so the + rem final ~bootstrap.status.json/postflight panel report this honestly once the run + rem actually finishes. + call :warn_build_incomplete "[WARN] PyInstaller execution failed; will verify your code directly via Python instead. reason=test_forced_fail" ) else ( set "HP_NUITKA_FALLBACK_USED=1" ) @@ -3595,8 +3603,9 @@ if not defined HP_BUILD_OK ( rem nonzero -- distinct from reason=missing_output below (exit 0 but no EXE) and rem reason=test_forced_fail above (no real build ever ran). Mirrors the existing rem UV_FALLBACK reason= token convention; see docs/agent-lessons-learned.md. - call :die "[ERROR] PyInstaller execution failed. reason=build_error" - set "HP_BOOTSTRAP_STATE=error" + rem CLAUDE.md Active Backlog Item 46 (Bucket B): see the test_forced_fail branch above + rem for why this is a warn-and-continue site, not a :die site. + call :warn_build_incomplete "[WARN] PyInstaller execution failed; will verify your code directly via Python instead. reason=build_error" ) else ( set "HP_NUITKA_FALLBACK_USED=1" ) @@ -3608,8 +3617,9 @@ if not defined HP_BUILD_OK ( if not exist "dist\%ENVNAME%.exe" ( call :try_nuitka_tier_a if errorlevel 1 ( - call :die "[ERROR] PyInstaller did not produce dist\%ENVNAME%.exe reason=missing_output" - set "HP_BOOTSTRAP_STATE=error" + rem CLAUDE.md Active Backlog Item 46 (Bucket B): see the test_forced_fail branch + rem above for why this is a warn-and-continue site, not a :die site. + call :warn_build_incomplete "[WARN] PyInstaller did not produce dist\%ENVNAME%.exe; will verify your code directly via Python instead. reason=missing_output" ) else ( set "HP_NUITKA_FALLBACK_USED=1" ) @@ -5106,6 +5116,22 @@ if not defined HP_CI_LANE ( pause ) exit /b %RC% + +rem CLAUDE.md Active Backlog Item 46 (Bucket B): a non-pausing sibling of :die for a call site +rem that has failed at ONE task (e.g. every build tool tried) but the run itself is NOT doomed -- +rem real, useful work still follows (e.g. the interpreter-fallback verification) that will itself +rem determine the honest final outcome. Deliberately does NOT pause (the run isn't over yet, so +rem stopping here would misleadingly look like the terminal state), does NOT call :release_lock +rem (a concurrent second instance must not be able to start while this run still has real +rem verification work ahead of it), and does NOT call :write_status (the eventual :success/ +rem :print_no_exe_briefing dispatch writes the real final status once HP_BOOTSTRAP_STATE is +rem settled). Still sets HP_BOOTSTRAP_STATE=error, same as :die, so the final report is honest. +:warn_build_incomplete +set "MSG=%~1" +call :log "%MSG%" +set "HP_BOOTSTRAP_STATE=error" +exit /b 0 + :rotate_log powershell -NoProfile -ExecutionPolicy Bypass -Command ^ "if (Test-Path '%LOG%') { if ((Get-Item '%LOG%').Length -gt 10485760) { Move-Item -Force '%LOG%' '%LOGPREV%' } }" diff --git a/tests/selfapps_pyinstaller_fail.ps1 b/tests/selfapps_pyinstaller_fail.ps1 index f7b6eac9..3ce9ba5e 100644 --- a/tests/selfapps_pyinstaller_fail.ps1 +++ b/tests/selfapps_pyinstaller_fail.ps1 @@ -12,6 +12,16 @@ # consented to (HP_BUILD_OK). Fixed by setting HP_BOOTSTRAP_STATE=error at the PyInstaller # build call site, mirroring the existing preflight-failure precedent in :run_entry_smoke. # +# CLAUDE.md Active Backlog Item 46 (Bucket B, closed): all three call sites now route through +# :warn_build_incomplete instead of :die -- by the time any of them is reached, every build tool +# (PyInstaller AND the Nuitka fallback) has already failed, but the run is not doomed (the +# interpreter-fallback verification below still genuinely runs and still genuinely decides +# success/failure), so :die's mid-run pause and premature lock release were both wrong here. +# :warn_build_incomplete still sets HP_BOOTSTRAP_STATE=error, so every assertion below is +# unchanged -- only the message's [WARN]/[ERROR] prefix and the pause/lock-release timing +# changed, neither of which this test's own assertions inspect (CI never pauses at :die either +# way, since HP_CI_LANE is always set). +# # Three scenarios via PYI_FAIL_SCENARIO env var (research Finding 2, # docs/prd-av-safe-build-path.md): "execfail" forces the build command itself to fail # (HP_TEST_FORCE_PYINSTALLER_FAIL=1); "output_vanish" lets a real build succeed, then deletes @@ -22,8 +32,10 @@ # failure the other two scenarios' clean-exiting stub app never reaches. # # Asserts (all scenarios): the final ~bootstrap.status.json reads state=error (not silently -# overwritten back to ok), the correct [ERROR] message appears in the log, and (docs/ -# open-questions.md item 1) the dedicated :print_no_exe_briefing panel is shown. For "execfail"/ +# overwritten back to ok), the correct failure message (now [WARN]-prefixed, see the Bucket B +# note above -- the assertion itself matches the message substring and reason= token only, not +# the prefix) appears in the log, and (docs/open-questions.md item 1) the dedicated +# :print_no_exe_briefing panel is shown. For "execfail"/ # "output_vanish", the stub app runs cleanly via the interpreter fallback despite total packaging # failure, so the final console [STATUS] line alone would otherwise read identically to a real # success -- the plain (non-caveat) panel text is asserted. For "execfail_runtimefail", the