prompt: close out the Clipper handover — retire shipped phase 1, file what was owed - #190
Merged
Merged
Conversation
Closes out the Clipper handover. Three findings from autolens_profiling#128 were living only inside other prompt files, where nothing indexes them and nobody can pick them up. All three are now filed prompts. Two incidental PyAutoFit bugs, split out of prior_support_clipper.md's "do not lose these" section, which asked for them to be filed separately once confirmed. Both confirmed against PyAutoFit main 1f4b66a93: - save_json_numpy_scalar_typeerror.md — paths/directory.py:80 is a bare json.dump with no default= hook, and there is no JSON encoder anywhere in autofit/. numpy float32 raises TypeError at the END of a successful run. It did not fire while 14/16 lanes were dead, so it fires exactly when lane survival improves — any fix to the prior-support problem will surface it. - crashed_run_poisons_resume.md — the chain, each link read on main: directory.py:79 opens "w+" (truncates, no atomic rename), so the crash above leaves a partial file; is_complete (directory.py:180) tests only the .completed marker, so abstract_search.py:582 dispatches to resume; the resume reads the truncated JSON; and JSONDecodeError subclasses ValueError, so the guard at multi_start_gradient/search.py:720 catching (FileNotFoundError, TypeError, KeyError) does not fall through to the fresh-start branch. Result is a zero-step no-op reported as a clean run — the phase-2 campaign currently documents this as a trap to work around. One research task, follow-up (2) owed by active/mge_lane_death.md: - ell_comps_trapping_unmasked.md — the plateau was masked, not cleared. The baseline n_constrained_lane_steps = 0 meant "nothing got that far"; lanes died of prior-exit first. Records that the 27.79% came from the prior-neutered diagnostic arm and is NOT a citable production number, so the task sequences after phase 1 and measures the clipped run instead. active.md now points at all five filed prompts (these three plus the two Clipper phases) so the thread is followable from the ledger. dashboard.md regenerated; lifecycle check OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
…ting gap Phase 1 has landed. PyAutoFit#1477 merged as 1f4b66a — the current tip of PyAutoFit main — shipping AbstractClipper / ClipperNone / ClipperPriorBox in autofit/non_linear/clipper.py, wired into both AbstractMultiStartGradient and AbstractBFGS, opt-in, default ClipperNone, bit-identical under it, 22 tests. The Mind had not noticed: prior_support_clipper.md was still sitting in draft/ as a high-priority backlog item on a dashboard people pick work from. Retired to complete/2026/08/prior-support-clipper.md via lifecycle.py record, with the original prompt folded in and the findings worth keeping written up (the inset keyed on bound kind rather than unguarded upper-lower; the LogGaussianPrior support mismatch; scipy reading a bounds tuple as (min, max) pairs and silently mis-fitting two-parameter models). Clip-count reporting, filed as clipper_usage_in_search_summary.md. The counting half already exists — n_clipped_lane_steps is accumulated per-lane (not per-coordinate) at multi_start_gradient/search.py:863, written to search_internal and restored on resume as a lifetime total. The storing half does not, and the gap is not where it looks: - search.summary is written by text_util.search_summary_to_file, which only ever sees samples and timings. There is no search-specific channel at all, for any counter — save_summary never sees search_internal. Routing is the work, not the arithmetic, which is why the prompt asks for a general hook serving the other multi-start counters rather than one threaded parameter. - LBFGS produces no count whatsoever. It hands optimize.Bounds to scipy and never calls project, so there is no mask to count. Emitting 0 there would read as "never fired" when it means "cannot know". Phase 2 (clipper_validation_campaign.md) marked UNBLOCKED with what phase 1 changed for it: record n_clipped_lane_steps as a fourth counter from search_internal; a ClipperPriorBox arm reporting zero clips has not exercised the clipper and its "no change" result is meaningless; the momentum-reset arm needs writing since nothing yet consumes the mask; and the float32 save_json crash was NOT fixed by phase 1 (verified at 1f4b66a), so that trap stands and will start firing precisely because clipping keeps lanes alive. Repointed all seven references to the moved prompt. lifecycle check + index check OK; dashboard regenerated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
…mentation The clip-count work is written and tested against PyAutoFit 1f4b66a, but push access to PyAutoFit was denied in this session, so it lands as a patch rather than a PR. Prompt updated to say so and to carry the verification evidence. Corrects a wrong premise in the prompt as first filed. It claimed search.summary had "no search-specific channel at all". That is false: search_summary_from_samples (text_util.py:115-161) already reads samples.samples_info and already emits Resurrections, Value-NaN Lane-Steps, Gradient-NaN Lane-Steps and both rates, key-guarded so other searches are unaffected. I had read search_summary_to_file and not the function it calls. Left uncorrected it would have sent an implementer off to build a routing mechanism that already exists. The real gap is narrow: n_clipped_lane_steps never reaches samples_info, and n_constrained_lane_steps reaches it but is never emitted — so PyAutoFit#1475's trapped-lane counter has been invisible in search.summary since it shipped. As built (tmp/clipper-search-summary.patch, gitignored, 4 files +243/-0): multistart publishes clipper name + n_clipped_lane_steps into samples_info; bfgs publishes the clipper name and deliberately no count, since it is declarative and scipy enforces, so a 0 would read as "never fired" when it means "cannot know"; text_util grows _clipper_summary_from handling the three cases plus the owed Constrained Lane-Steps line; 6 new tests. Verified on 3.13: non_linear + text = 501 passed, 3 skipped, 1 failed, the failure (nautilus single-core pool) reproduced identically on a stashed clean tree, as was the astropy collection error — both already recorded under complete/2026/08/frozen-lane-counter.md. Also verified end-to-end against the search.summary files four real searches actually wrote, giving Clipped Lane-Steps = 414 at rate 0.958 on the clipped multistart arm. Incidental, worth keeping: on that toy 3-parameter Gaussian the unclipped arm showed Value-NaN Lane-Steps = 378 (94.5%) and the clipped arm 0 — the #128 mechanism reproducing on a model with nothing astrophysical in it, which makes a far cheaper regression fixture than the imaging/mge cell. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
…1478 Advance to complete/ on merge. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
save_json_numpy_scalar_typeerror -> PyAutoFit#1479. Reproduced against the unfixed tree before fixing. Fixed with a NumpyEncoder wired into BOTH output writers -- save_json and, which this prompt did not name, Samples.info_to_json. That second site is the more dangerous: samples_info is a search's own diagnostic channel, so every counter added to it could reintroduce the crash. crashed_run_poisons_resume -> PyAutoFit#1480, and carries a CORRECTION. The prompt claimed the poisoned rerun is "a 4-second no-op that reads as a clean result". That did not reproduce. What reproduces is a hard JSONDecodeError on every rerun of the same search name, naming no file and offering no remedy. The no-op variant presumably needs a surviving search_internal whose restored total_steps short-circuits the loop; the crash path deletes it first. Same root cause, fix covers both, but only the crash is evidenced -- recorded so the unobserved symptom is not cited as fact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
This was referenced Aug 16, 2026
Jammy2211
added a commit
that referenced
this pull request
Aug 16, 2026
…llow-ups filed (#194) Two sessions independently wrote a completion record for the prior-support Clipper. #189 was opened first, by the session that actually shipped PyAutoFit#1477; #190 (mine) landed later because I never listed open PRs before starting. This consolidates them so main holds one record, not two. complete/2026/08/prior-support-clipper.md is now the union. The detailed body comes from #189 and is materially richer than what I wrote: the bound-kind design decision (why one relative margin is wrong in two silent ways), eight traps measured against a running install, the verification log (bit-identity 10/10, core promise 8/8, guards verified by inversion), and the process lesson -- a first commit shipped an undefined `optimize` in LBFGS._fit and the full 1790-test suite passed against it, because nothing in the suite executes an LBFGS fit. My "what shipped after" section is appended, since #189 predates #1478/#1479/#1480 and so records follow-ups 1 and 2 as open when they are now fixed. #189's phase-2 harness traps salvaged into the campaign prompt -- the .completed short-circuit, fit() rebuilding search.paths so instance-level patches are discarded, search_internal being deleted on success, seeding both random AND numpy, and a box containing the optimum never exercising the clipper. Also that arm 3 does not exist yet (phase 1 ships the mask, no reset) and two phase-1 measurements to carry in as priors. Follow-ups 3 and 4 filed as prompts rather than left in a record: loggaussian_prior_declares_own_support.md and clipper_in_search_identifier.md. The second is Autonomy: human-required -- both answers orphan or collide someone's stored results, and it is best decided BEFORE phase 3 so the re-baseline and any re-keying are not entangled. One correction absorbed from #189: test_nautilus single_core_builds_no_pool PASSES in CI and fails only in local venvs (both 3.12 and 3.13, two independent sessions). "Not caused by this work" is right; "pre-existing on clean main" -- the looser phrasing used in #1479 and #1480's bodies -- is wrong. Recorded. Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Aug 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes out the Clipper handover. Nothing from the autolens_profiling#128 investigation is left living only inside another prompt file, and the Mind's view of the Clipper work now matches reality.
Why this needs to be on
main: the same reason #188 gave. A cold session gets whatever is in its checkout, which comes frommain. Until this merges, the phase-1 prompt still reads as unstarted high-priority backlog on the dashboard people pick work from, and the four filed follow-ups do not exist.Phase 1 had already shipped and the Mind had not noticed
PyAutoFit#1477 merged as
1f4b66a— the current tip of PyAutoFitmain— shippingAbstractClipper/ClipperNone/ClipperPriorBox, wired into bothAbstractMultiStartGradientandAbstractBFGS, opt-in, defaultClipperNone, bit-identical under it, 22 tests.prior_support_clipper.mdwas still sitting indraft/as a high-priority backlog item. Retired tocomplete/2026/08/prior-support-clipper.mdvialifecycle.py record, prompt folded in, with the findings worth keeping written up: the inset keyed on bound kind rather than unguardedupper - lower;LogGaussianPriorreporting(-inf, inf)for a support of(0, inf); scipy reading a bounds tuple as(min, max)pairs and silently mis-fitting two-parameter models.Four prompts filed, three of them previously homeless
Two incidental PyAutoFit bugs that
prior_support_clipper.mdflagged under "do not lose these — file separately if confirmed". Both confirmed by reading PyAutoFitmain, and both still present at1f4b66a— phase 1 did not fix either. Both are now implemented and in flight:save_json_numpy_scalar_typeerror.md→ PyAutoFit#1479.paths/directory.py:80is a barejson.dumpwith nodefault=hook, and there is no JSON encoder anywhere inautofit/.float32raises at the end of a successful run. It did not fire while 14/16 lanes were dead, so it starts firing exactly when lane survival improves. Implementing it turned up a second unguarded site the prompt never named —Samples.info_to_json— which is the more dangerous of the two, sincesamples_infois a search's own diagnostic channel.crashed_run_poisons_resume.md→ PyAutoFit#1480.directory.py:79opens"w+"(truncates, no atomic rename) →is_completetests only the.completedmarker →abstract_search.py:582dispatches to resume → the resume reads the truncated JSON → andJSONDecodeErrorsubclassesValueError, so the guard atmulti_start_gradient/search.py:720catching(FileNotFoundError, TypeError, KeyError)never falls through to the fresh-start branch.Correction, from reproducing it. The prompt as filed described the result as "a zero-step no-op reported as a clean run". That did not reproduce. What reproduces against
mainis a hardJSONDecodeErroron every rerun of that search name, naming no file and offering no remedy. The no-op variant presumably needs a survivingsearch_internalwhose restoredtotal_stepsshort-circuits the loop — the crash path deletes it first. Same root cause, and #1480 covers both paths, but only the crash is evidenced, and the prompt now says so.One research task, follow-up (2) owed by
active/mge_lane_death.md:ell_comps_trapping_unmasked.md— the plateau was masked, not cleared. The baselinen_constrained_lane_steps = 0meant "nothing got that far"; lanes died of prior-exit first. Records that the 27.79% came from the prior-neutered diagnostic arm and is not a citable production number, so the task sequences after phase 1 and measures the clipped run instead.And one new feature task:
clipper_usage_in_search_summary.md→ PyAutoFit#1478. Surface the clip count insearch.summary.Phase 2 unblocked
clipper_validation_campaign.mdmarked ready, with what phase 1 changed for it: recordn_clipped_lane_stepsas a fourth counter fromsearch_internal; aClipperPriorBoxarm reporting zero clips has not exercised the clipper and its "no change" result is meaningless; the momentum-reset arm needs writing since nothing yet consumes the mask; and thesave_jsontrap stands until #1479 lands.Bookkeeping
All seven references to the moved prompt repointed.
lifecycle.py checkandindex --checkboth OK;dashboard.mdregenerated.🤖 Generated with Claude Code
https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR