prompt: retire the three shipped Clipper follow-ups to complete/

active.md

@@ -12,8 +12,8 @@
 - reproduction: 16x150 cloud CPU gave 1446/18/0/0 and `alive 2/16` against the filed 1498/9/0/0 and the same 2/16. The survival identity is exact: `sum(150 - k_i) = 14*150 - 654 = 1446` = `n_value_nan_lane_steps`.
 - COUNTER-FINDING, corrects the framing: the `ell_comps` plateau was MASKED, not cleared. The baseline's `n_constrained_lane_steps = 0` was a correctly-measured zero (the positive control was sound) but it meant "nothing got that far" — lanes died of prior-exit first. With the prior deaths removed the constrained count is 667 (27.79%). #1475's trapped-lane counter is measuring a live failure mode on this cell, hidden behind a larger one. Lanes stop being dead and start being STUCK.
 - follow-ups owed (both out of this task's boundary), ALL NOW FILED 2026-08-16 — nothing from this investigation lives only inside another prompt: (1) PyAutoFit — bounded stepping (projection/clipping onto prior support) or soft-walled priors; `resurrect=True` is NOT the fix, it redraws a lane that then walks out again. **PHASE 1 SHIPPED 2026-08-16** as PyAutoFit#1477 (`1f4b66a`, tip of main) → record `complete/2026/08/prior-support-clipper.md`; phase 2 `draft/feature/autofit/clipper_validation_campaign.md` is now UNBLOCKED and is the next thing to run; phase 3 (flip the default) must NOT be written until phase 2 has run. (2) the ell_comps trapping at 27.79%, now that it is visible → `draft/research/autolens_profiling/ell_comps_trapping_unmasked.md` — note that 27.79% came from the prior-neutered DIAGNOSTIC arm and is not a citable production number; it runs on top of phase 1 and can share phase 2's arms.
-- clipper-reporting gap, filed 2026-08-16: `n_clipped_lane_steps` IS counted per-lane by phase 1 (`multi_start_gradient/search.py:863`) and written to `search_internal`, but `search.summary` carries no search-specific counters at all — `text_util.search_summary_to_file` only ever sees `samples` and timings, so there is no channel for a search to report anything. The LBFGS path produces no count whatsoever, because it hands bounds to scipy and never calls `project`. Filed as `draft/feature/autofit/clipper_usage_in_search_summary.md`. Phase 2 should read the counter from `search_internal` meanwhile — and a `ClipperPriorBox` arm reporting ZERO clips has not exercised the clipper, so its "no change" result would be meaningless.
-- incidental PyAutoFit bugs found while investigating, both filed 2026-08-16 and both verified against PyAutoFit main `1f4b66a93`: `draft/bug/autofit/save_json_numpy_scalar_typeerror.md` (bare `json.dump` at `paths/directory.py:80` dies on `float32` at the END of a SUCCESSFUL run — it fires exactly when lane survival improves, so any fix to the above will surface it) and `draft/bug/autofit/crashed_run_poisons_resume.md` (the truncated file that crash leaves makes the next same-named run resume into it and report a zero-step no-op as a clean result; `JSONDecodeError` is a `ValueError` and is not caught by the resume guard's `(FileNotFoundError, TypeError, KeyError)` at `multi_start_gradient/search.py:720`). The second is a documented trap the phase-2 campaign currently works around — landing it removes that workaround.
+- clipper-reporting: SHIPPED 2026-08-16 as PyAutoFit#1478 (`bbceff6`) → record `complete/2026/08/clipper-usage-in-search-summary.md`. `search.summary` now reports `Clipper`, `Clipped Lane-Steps`, `Clipped Lane-Step Rate` and `Constrained Lane-Steps`. CORRECTS an earlier note here that claimed `search.summary` had no search-specific channel at all — false; `search_summary_from_samples` already read `samples_info` and already emitted the NaN counters. The real gap was two omissions, not a missing mechanism. Phase 2 reads the clip count straight from `search.summary` now, and a `ClipperPriorBox` arm reporting ZERO clips has not exercised the clipper — a broken arm, not a null result.
+- incidental PyAutoFit bugs found while investigating: BOTH SHIPPED 2026-08-16. `save_json` float32 crash → PyAutoFit#1479 (`b6e89cd`), record `complete/2026/08/save-json-numpy-scalar-typeerror.md`; fixing it turned up a second unguarded writer the prompt never named, `Samples.info_to_json`, which is the more dangerous since `samples_info` gains a counter every time a search does. Crashed-run-poisons-resume → PyAutoFit#1480 (`5c9244b`), record `complete/2026/08/crashed-run-poisons-resume.md`. CORRECTION carried in that record: the "zero-step no-op reported as a clean result" symptom did NOT reproduce; what reproduces is a hard `JSONDecodeError` on every rerun of the same search name. `JSONDecodeError` subclasses `ValueError`, which is why it slipped past every guard on the resume path. Both traps the phase-2 campaign worked around are now closed.
 - caveat: one baseline death (lane 9, step 39) re-evaluates finite in every term with all params inside their boxes — the jitted/vmapped float32 path differs from the eager recompute there, unexplained. Single seed per arm, CPU, x64 off.
 - reading the number: 62% is a survival integral, not a hazard rate — a frozen lane keeps counting every subsequent step, so the same death curve reports ~75% at 300 steps. Inverting it gives a mean death step of ~43 of 150 (mid-descent, not bad initial draws). Grade any re-run on the alive-versus-step CURVE, not on recovering the scalar.
 - ordering (deliberate, do not revert): cause-finding FIRST on the existing ~6-min CPU run, then the `resurrect=True` budget-recovery measurement, then the production/GPU/seed confirmation. Do not queue for a GPU before the cause step has been attempted.

draft/feature/autofit/clipper_usage_in_search_summary.md → complete/2026/08/clipper-usage-in-search-summary.md

@@ -1,3 +1,71 @@
+- library-prs: https://github.com/PyAutoLabs/PyAutoFit/pull/1478
+- merge-commits: PyAutoFit `bbceff62a8e55d57ad5bd968f9ca17c87b435288` (2026-08-16)
+- issue: none — filed from the Clipper handover and implemented in the same session
+- summary: `search.summary` now reports prior-support clipping. `n_clipped_lane_steps`
+  reaches `samples_info`, and `search_summary_from_samples` emits `Clipper`,
+  `Clipped Lane-Steps` and `Clipped Lane-Step Rate` — plus
+  `Constrained Lane-Steps`, the trapped-lane counter from #1475, which had
+  reached `samples_info` when it shipped but was never printed.
+- validation: 6 new tests; `non_linear` + `text` = 501 passed. Verified
+  end-to-end against the `search.summary` files four real searches wrote, not
+  just the formatting helper.
+- release: not performed; merged PR remains in the pending-release queue.
+
+## The premise the prompt got wrong
+
+Worth recording, because it nearly caused a rebuild of something that existed.
+The prompt as first filed claimed `search.summary` had "no search-specific
+channel at all". **False.** `search_summary_from_samples`
+(`text/text_util.py:115-161`) already read `samples.samples_info` and already
+emitted the NaN counters and their rates, key-guarded so other searches are
+unaffected. The error came from reading `search_summary_to_file` and not the
+function it calls.
+
+The real gap was two specific omissions, not a missing mechanism:
+
+- `n_clipped_lane_steps` never left `search_internal` for `samples_info`.
+- `n_constrained_lane_steps` reached `samples_info` but was never emitted.
+
+## Three cases, and the last two are the point
+
+- **No clipper** (`ClipperNone`, or a search predating the `Clipper`) — emits
+  nothing. The default path's summary is unchanged, which matters because the
+  file is read by tooling and sits in every archived run's output.
+- **Clipped and counted** (`MultiStartGradient`) — it enforces the constraint
+  itself every step via `project`, so it knows how often it fired.
+- **Clipped but not observable** (`LBFGS`, other bound-supporting scipy methods)
+  — declarative: it hands `optimize.Bounds` to scipy and lets scipy enforce, so
+  `project` is never called and no mask exists. Reporting `0` would read as
+  "never fired" when it means "cannot know". It says
+  `not measured (bounds enforced by scipy)` instead.
+
+## Decisions worth keeping
+
+- The clipper is published as its **class name**, not a bool, so a later
+  strategy needs no schema change.
+- The count is **per-lane, not per-coordinate** — a lane clipped in three
+  parameters on one step is one clipped lane-step — keeping all four counters
+  directly comparable.
+- It is a **lifetime total** restored from `search_internal`, so a resumed run
+  reports the whole run rather than the current process's share.
+
+## A finding worth more than the feature
+
+While verifying, the autolens_profiling#128 mechanism reproduced on a **toy
+3-parameter Gaussian** (`autofit.example.Gaussian`, `centre` truth 30 outside a
+`UniformPrior(5, 20)`, 4 starts x 200 steps, `learning_rate=1.0`): unclipped
+`Value-NaN Lane-Steps = 378` (94.5%), clipped `0`, with
+`Clipped Lane-Steps = 414` at rate 0.958. Far cheaper as a regression fixture
+than the `imaging/mge` cell, and a seconds-long smoke test for "is the clipper
+wired up and firing at all".
+
+## Behaviour change to remember
+
+Multi-start summaries gained a `Constrained Lane-Steps` line. Everything else is
+additive and gated.
+
+## Original prompt
+
 # Report how much the Clipper actually fired — surface the count in `search.summary`
 
 Type: feature

draft/bug/autofit/crashed_run_poisons_resume.md → complete/2026/08/crashed-run-poisons-resume.md

@@ -1,3 +1,76 @@
+- library-prs: https://github.com/PyAutoLabs/PyAutoFit/pull/1480
+- merge-commits: PyAutoFit `5c9244bc1d0c000804899a0378dcb0187f9716af` (2026-08-16)
+- issue: none — split out of the prior-support Clipper prompt's "do not lose these"
+- summary: A run interrupted while writing output left a half-written JSON file,
+  and every later run of that search name died on it — from inside an *optional*
+  sanity check — until the output directory was deleted by hand. Fixed in three
+  legs: `open_atomic` (temp file + `os.replace`) for `save_json` and
+  `save_search_internal`; `Fitness.check_log_likelihood` treating a corrupt
+  summary as absent, with a warning; and the multi-start resume guard widened so
+  corrupt state falls into the fresh-start branch below it.
+- validation: 9 new tests; full suite 1800 passed / 4 skipped / 1 failed
+  (pre-existing). Merged result with #1478 + #1479: 1819 passed.
+- release: not performed; merged PR remains in the pending-release queue.
+
+## The one fact behind all three legs
+
+**`json.JSONDecodeError` subclasses `ValueError`.** So it is neither a
+`FileNotFoundError`, a `TypeError` nor a `KeyError`, and it fell through every
+guard on the resume path — `check_log_likelihood`'s `except FileNotFoundError`
+and the multi-start `except (FileNotFoundError, TypeError, KeyError)` alike.
+Asserted directly in a test so it cannot quietly stop being true.
+
+## CORRECTION to the filed prompt — reproduce before you fix
+
+The prompt described the poisoned rerun as "a 4-second no-op run that reads as a
+clean result (zero deaths, because zero steps)". **That did not reproduce.**
+
+What reproduces against `main`, using the real trigger (a float32 killing
+`save_json` at the end of a successful fit):
+
+```
+run 1: TypeError: Object of type float32 is not JSON serializable
+run 2: JSONDecodeError: Expecting value: line 1 column 13 (char 12)
+```
+
+A hard crash naming no file and offering no remedy, on *every* rerun of that
+name. The silent-no-op variant presumably needs a **surviving**
+`search_internal`, whose restored `total_steps` short-circuits the loop — but
+the crash path deletes that directory first. Same root cause, and the fix covers
+both paths, but only the crash is evidenced. **Do not cite the no-op as
+observed.**
+
+Also worth knowing: LBFGS does *not* poison. It simply refits and overwrites.
+The hazard is specific to searches that read prior output while resuming.
+
+## Legs, and why each is separate
+
+1. **Atomicity.** `open(path, "w+")` truncates first and writes second, so any
+   failure destroys the file that was there. `open_atomic` catches
+   `BaseException`, not `Exception` — a `KeyboardInterrupt` mid-write leaves
+   identical debris. Applied to `save_search_internal` too, which matters more:
+   that is what a resumed run restores its step count and counters from.
+2. **Recovery.** `check_log_likelihood` already returned early on a *missing*
+   summary; a *corrupt* one is the same situation, since there is no
+   trustworthy old likelihood either way. It warns rather than staying silent —
+   an unreadable file is a real event, unlike a missing one.
+3. **The resume guard.** Found by audit, not by the prompt: the same narrow
+   except-tuple meant a corrupt `search_internal` raised instead of falling into
+   the fresh-start branch directly beneath it.
+
+Legs 1 and 2 are independent by design — 1 stops the debris being created, 2
+recovers from debris created by older versions, killed processes or a full disk.
+
+## Trap in testing this
+
+The end-to-end regression test must assert the second run actually **takes the
+resume path**. Without that, a run short-circuited as already-complete
+(`is_complete` -> `result_via_completed_fit`) passes the test while asserting
+nothing. The `.completed` marker also is not found by `rglob(".completed")` under
+the test config — use `paths._has_completed_path`.
+
+## Original prompt
+
 # A crashed run poisons the next run of the same name — silently, as a clean result
 
 Type: bug

draft/bug/autofit/save_json_numpy_scalar_typeerror.md → complete/2026/08/save-json-numpy-scalar-typeerror.md

@@ -1,3 +1,56 @@
+- library-prs: https://github.com/PyAutoLabs/PyAutoFit/pull/1479
+- merge-commits: PyAutoFit `b6e89cd5480018c9b9661ac2b214ec1d084e3964` (2026-08-16)
+- issue: none — split out of the prior-support Clipper prompt's "do not lose these"
+- summary: Adds `NumpyEncoder` in `autofit/tools/util.py` (`np.ndarray` ->
+  `tolist()`, `np.generic` -> `item()`, everything else deferred to the base
+  class so a genuinely unserialisable object still raises), wired into both
+  output-path writers. A successful fit could previously die at its output step
+  with `TypeError: Object of type float32 is not JSON serializable`.
+- validation: 13 new tests; full suite 1804 passed / 4 skipped / 1 failed, the
+  failure pre-existing (nautilus single-core pool) and identical on a clean tree.
+- release: not performed; merged PR remains in the pending-release queue.
+
+## Why it hid for so long
+
+`np.float64` subclasses Python's `float`, so `json` serialises it without help.
+**`np.float32` subclasses nothing `json` knows** — nor does `np.int32`/`np.int64`
+(not a Python `int` on every platform) or `np.bool_`.
+
+So a float64 run is fine, and a float32 run is fine *right up until it writes its
+results*, at which point the whole computation is discarded at the last step.
+
+It surfaced on the `imaging/mge` profiling cell during the Clipper prototype
+(autolens_profiling#128) on a path that cell had apparently never taken: it did
+**not** fire while 14 of 16 lanes were dead, because the float32 values never
+reached the saved object. **It fires precisely when lane survival improves** —
+which is what the phase-2 campaign exists to cause.
+
+## The second site the prompt never named
+
+The prompt described `paths/directory.py:80` only. Implementing it turned up
+`Samples.info_to_json` (`samples/samples.py:338`), a second bare `json.dump` —
+and the more dangerous of the two. `samples_info` is a search's own diagnostic
+channel, so **every new counter added to it is another chance to reintroduce
+this**. PyAutoFit#1478 added two counters to that very dict on the same day.
+
+That is the argument for fixing at the encoder rather than at each producer:
+the producers are search-specific and keep multiplying. The encoder closes the
+class; chasing producers closes one instance.
+
+## Guarantees asserted, not assumed
+
+- `float64` unchanged, compared byte-for-byte against a bare `json.dumps`.
+- No invented precision — `.item()` widens float32 to a Python double, and the
+  test asserts equality with `float(np.float32(0.1))` rather than a re-rounded
+  decimal.
+- Still strict — `json.dumps({"x": object()}, cls=NumpyEncoder)` raises.
+
+Reproduced against the unfixed tree before fixing:
+`paths.save_json(name="c", object_dict={"clipped": np.float32(4.0)})` raised
+exactly the reported `TypeError`.
+
+## Original prompt
+
 # `save_json` crashes on numpy scalars — a successful run dies at output
 
 Type: bug

complete/index.md

@@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema:
 only then grep a dated bucket. Curators: edit the band between the CURATED
 markers; everything below GENERATED is rebuilt.
 
-993 records across 7 buckets.
+996 records across 7 buckets.
 
 <!-- CURATED:START -->
 ## Highlights
@@ -32,12 +32,14 @@ _(curate hard-won records here — survives regeneration.)_
 - [border-relocator-backend-parity](2026/08/border-relocator-backend-parity.md) — Isolated the native NumPy/JAX likelihood gap to mathematically non-unique PCA axes for a near-isotropic border…
 - [border-relocator-degenerate-pca](2026/08/border-relocator-degenerate-pca.md) — Stabilized near-isotropic border PCA axes with a deterministic axis-aligned branch at a scale-aware eigenvalue…
 - [circular-ell-comps-image-gradient](2026/08/circular-ell-comps-image-gradient.md) — Replaced the Sersic image path's Cartesian-to-polar ellipticity conversion with an algebraically equivalent Ca…
+- [clipper-usage-in-search-summary](2026/08/clipper-usage-in-search-summary.md) — `search.summary` now reports prior-support clipping. `n_clipped_lane_steps`
 - [compile-axis-campaign-coverage](2026/08/compile-axis-campaign-coverage.md)
 - [compile-axis-triage-drift](2026/08/compile-axis-triage-drift.md)
 - [compile-warm-baseline-dashboard](2026/08/compile-warm-baseline-dashboard.md)
 - [conductor-discovery-lifecycle-split](2026/08/conductor-discovery-lifecycle-split.md) — closed on merge
 - [correct-circular-sersic-hazard](2026/08/correct-circular-sersic-hazard.md) — Corrected the circular Sersic hazard in the actual fitted ell_comps coordinates. The q-angle structural findin…
 - [covariance-interpolator-rng-seed](2026/08/covariance-interpolator-rng-seed.md) — auto-closed by the merge
+- [crashed-run-poisons-resume](2026/08/crashed-run-poisons-resume.md) — A run interrupted while writing output left a half-written JSON file,
 - [curvature-diag-doc-config-drift](2026/08/curvature-diag-doc-config-drift.md) — Aligned PyAutoArray's curvature-diagonal helper and Settings docs with the packaged 1e-3 default; numerical be…
 - [curvature-floor-scale-aware-prototype](2026/08/curvature-floor-scale-aware-prototype.md) — Corrected the Phase 2 conditioning denominator to the curvature entries actually floored. The absolute policy …
 - [database-guide-sample-weight-threshold](2026/08/database-guide-sample-weight-threshold.md) — `guides/results/database/start_here.py` ran its own Nautilus fits capped at n_like_max=300 then indexed sample…
@@ -105,6 +107,7 @@ _(curate hard-won records here — survives regeneration.)_
 - [resolve-curvature-floor-doc-drift](2026/08/resolve-curvature-floor-doc-drift.md) — Reconciled the curvature-floor documentation finding after PyAutoArray#444. The detector now requires both run…
 - [resolve-sersic-ell-comps-gradient](2026/08/resolve-sersic-ell-comps-gradient.md) — Reconciled the numerical-hazard evidence after the Sersic Cartesian-radius fix. The stable conditional detecto…
 - [samplers-surface-autolens-tiers](2026/08/samplers-surface-autolens-tiers.md) — the samplers faculty's SamplerSurface now scans the findings
+- [save-json-numpy-scalar-typeerror](2026/08/save-json-numpy-scalar-typeerror.md) — Adds `NumpyEncoder` in `autofit/tools/util.py` (`np.ndarray` ->
 - [script-title-underline-off-by-one](2026/08/script-title-underline-off-by-one.md)
 - [simulator-util-to-af-ex](2026/08/simulator-util-to-af-ex.md) — moved the four 1D-Gaussian simulator helpers out of the duplicated
 - [small-datasets-loader-pixel-scales](2026/08/small-datasets-loader-pixel-scales.md)