prompt: ship autofit-prior-support-clipper (PyAutoFit#1477)

active.md

@@ -1,5 +1,25 @@
 # Active Tasks
 
+## clipper-validation-campaign
+- issue: https://github.com/PyAutoLabs/autolens_profiling/issues/129
+- prompt: active/clipper_validation_campaign.md (issued as autolens_profiling#129; advanced to active/ on filing, per the lifecycle)
+- session: claude cloud (web) 2026-08-16 — **HANDOFF, work part-done, read this whole entry before running anything**
+- status: IN PROGRESS — machinery built and first arm pair measured; campaign NOT complete and NO PR opened
+- upstream: phase 1 shipped as PyAutoFit#1477 (`1f4b66a`), record `complete/2026/08/autofit-prior-support-clipper.md`. Root cause autolens_profiling#128 / the `mge-lane-death` entry below.
+- repos:
+  - autolens_profiling: claude/clipper-validation-campaign-o3jotv (pushed, 1 commit)
+- HEADLINE RESULT (imaging/mge hst, 16x150, single seed, cloud CPU float32): value-NaN lane-steps **2268 -> 47** with 1920 clips, and `max_log_likelihood` is **IDENTICAL between arms to every printed digit** (-15529.587986751998). 11 of 16 lanes end pinned to a bound. That fires **two of the four pre-registered falsification conditions**. **DO NOT WRITE PHASE 3 ON THIS EVIDENCE.**
+- NOT the caching artefact — this was the predicted way to fake an identical result, so it was checked first: the runs differ in value-NaN (2268 vs 47), grad-NaN (0 vs 9) and wall (286.5s vs 249.6s). Arm 2 genuinely ran and converged to the same best point. Likely mechanism: the winning lane never left the prior box, so clipping never touched it.
+- THE CAVEAT THAT LIMITS EVERYTHING: both arms are **~47,316 nats** from the Nautilus bar (31786.782462). Neither is converged, so the comparison happens in a regime where the search has not found the basin at all. The truth-recovery claim being graded against came from GPU/Prodigy/longer budgets. The load-bearing question is **unanswered, not answered negatively**. Do not report this as "clipping doesn't work".
+- BLOCKER, needs a decision: **multi-seed is currently impossible.** `_broad_starts` seeds with a hardcoded `np.random.default_rng(0)`, so every run draws IDENTICAL starts; seeding `random`/`numpy` perturbs only the initializer. The campaign requires >=2 seeds per arm. Fix is a small `seed` arg on `AbstractMultiStartGradient` — a PyAutoFit change, so its own prompt/PR. **The human was asked and has not answered yet.**
+- BLOCKER: the **momentum-reset arm does not exist**. Phase 1 ships the clipped mask it needs, but no reset. 11/16 pinned makes this the decisive arm.
+- what was built (all on the branch): `scripts/misc/searches/clipper_campaign.py` (the arm driver) and a `SEARCHES_CLIPPER=none|prior_box` knob in `scripts/misc/searches/_samplers.py`. Findings note at `results/notes/clipper_campaign/README.md`, raw rows in the sibling JSON.
+- why a bespoke driver: the ordinary `searches/` runner records **none** of the lane counters, and `search_internal` (where they live) is **deleted on successful completion**. The driver captures it as written, patching `save_search_internal` at **CLASS** level — `fit()` rebuilds `search.paths`, so an instance-level hook is silently discarded.
+- ENVIRONMENT (the fiddly part, reproduce exactly): py3.12 venv; `pip install autolens jaxnnls`; **then** `pip install -e <PyAutoFit checkout> --no-deps`; then assert `autofit.__file__` resolves to the checkout. The clipper is UNRELEASED, so a PyPI autofit silently has no `clipper` argument and every number would be meaningless. In this session the venv was `/tmp/.../scratchpad/venv` with PyAutoFit at `/workspace/pyautofit` — both ephemeral, so a new session rebuilds.
+- cost/feasibility measured: one 16x150 arm is ~250-290s wall on this cloud CPU box, and JIT compile dominates short runs (4x5 took ~128s). Two arms ~9 min. GPU cells (pixelized meshes) are NOT runnable from a cloud session — no HPC/SSH access.
+- next, in order: (1) a **converged budget** (GPU / more steps / Prodigy) so the comparison is meaningful; (2) the `seed` arg, then >=2 seeds/arm; (3) the momentum-reset arm; (4) remaining cells — pixelized (GPU), point_source, and the unbounded-prior negative control.
+- do-not: do NOT open a PR on the campaign branch yet — the campaign is part-run and the note would read as a verdict. Do NOT flip any default.
+
 ## mge-lane-death
 - issue: https://github.com/PyAutoLabs/autolens_profiling/issues/128
 - prompt: active/mge_lane_death.md

draft/feature/autofit/clipper_validation_campaign.md → active/clipper_validation_campaign.md

@@ -164,6 +164,70 @@ not swept up as a clipping artefact.
 - **`0` and `null` are different findings.** Read counters with `.get()` and a
   `null` means the search never wrote the key — broken plumbing, not a clean cell.
 
+## What phase 1 actually shipped (added 2026-08-16, after it landed)
+
+Phase 1 is **MERGED**: PyAutoFit#1477 as `1f4b66a`, closing #1476. Record:
+`complete/2026/08/autofit-prior-support-clipper.md`. Four things change how this
+task must be run:
+
+1. **Arm 3 (momentum reset) does NOT exist yet.** Phase 1 ships the clipped
+   **mask** from `project`, which is what a momentum reset would need — but no
+   reset is implemented, and the search does not touch `opt_state` on a clip.
+   So arm 3 either gets a small addition first, or this campaign runs arms 1-2
+   and reports the pinned-lane count as the input to that decision. Do not
+   silently drop it: the prototype's 5/16 pinned lanes are why the arm exists.
+
+2. **A new counter to record: `n_clipped_lane_steps`**, in `search_internal`
+   alongside the existing three. Per-LANE per-step (`any(mask, axis=-1)`), same
+   convention as the others. It is the direct measure of how hard the wall is
+   being hit, and it distinguishes "clipping rarely fires" from "clipping fires
+   constantly and the population is living on the bound".
+
+3. **THE ARM-COLLISION HAZARD — read before running anything.** Verified with
+   real fits during phase 1: the `clipper` does **not** enter the search
+   identifier. `no clipper` / `ClipperNone` / `ClipperPriorBox` all resolve to
+   **the same output directory**. That is good for back-compat (existing results
+   are not orphaned) and *actively dangerous here*, because arms 1 and 2 differ
+   in exactly nothing else. Stacked with two other confirmed behaviours:
+   - a `.completed` marker makes `fit()` **short-circuit and return the cached
+     result without entering `_fit`**, and
+   - the `search_internal` folder is **deleted on successful completion**, so it
+     cannot be read back afterwards,
+
+   the failure mode is that **arm 2 silently returns arm 1's numbers**, looking
+   like a clean run. This is the same cached-result hazard #128 already recorded,
+   with a new and much sharper trigger. Mitigation, all three: give every arm a
+   **unique `name`**, delete `output/<name>/` between arms, and assert the
+   recorded `total_steps == n_steps` before believing any counter. Also worth
+   deciding as a deliverable: *should* the clipper enter the identifier? Phase 3
+   wants a re-baseline, and colliding arms make that harder.
+
+4. **The `float32` `save_json` bug is NOT fixed** (filed as a follow-up, not
+   carried by phase 1). So this prompt's trap stands and applies: capture the
+   counters **independently of the result object**. The method proven in phase 1
+   is to patch `DirectoryPaths.save_search_internal` at **class** level and copy
+   the dict as it is written — instance-level patching is silently discarded,
+   because `fit()` rebuilds `search.paths`.
+
+Two more phase-1 measurements worth carrying in as priors, both CPU/float32 on a
+toy Gaussian, so directional only:
+
+- With the truth deliberately **outside** the prior box, lane deaths went
+  **249 → 0** with 252 clips, and the clipped run pinned `centre` at the upper
+  bound. That is the "pinning is a result, not a failure" case reproducing in
+  miniature, and an independent confirmation of the momentum-pinning mechanism.
+- The **negative control this prompt asks for already passes at unit level**: a
+  `GaussianPrior` coordinate is provably untouched by `ClipperPriorBox`, and there
+  is a regression test guarding it. The cell-level negative control is still
+  worth running, but a failure there would point at the cell, not the bounds
+  extraction.
+
+Also fixed in phase 1 and relevant when reading the source: the
+`AbstractMultiStartGradient` class docstring used to claim the rule steps "on the
+unconstrained (unit-cube) parameterization". It never did — `_broad_starts` maps
+draws to physical parameters. Corrected, but older checkouts still carry the false
+sentence.
+
 ## Deliberately out of scope
 
 - Flipping any default (phase 3).

draft/feature/autofit/prior_support_clipper.md → complete/2026/08/autofit-prior-support-clipper.md

@@ -1,3 +1,167 @@
+Shipped the search-agnostic `Clipper` — prior-support enforcement for the
+gradient searches, as the pluggable sibling of `Initializer`. This is follow-up
+(1) owed by the `mge-lane-death` investigation (autolens_profiling#128), which
+found the MGE lane deaths are the **prior** term, not the likelihood.
+
+- issue: PyAutoFit#1476
+- pr: PyAutoFit#1477, **MERGED** 2026-08-16 as `1f4b66a` (squash), +798/-3 over
+  7 files. Full CI green on both runs (`unittest` 3.12, `unittest` 3.13,
+  `docs / docs-build`).
+
+## What shipped
+
+`AbstractClipper` / `ClipperNone` / `ClipperPriorBox` in a new
+`autofit/non_linear/clipper.py`, wired **opt-in** into the two exposed searches:
+`AbstractMultiStartGradient` projects after `optax.apply_updates`; `AbstractBFGS`
+hands box bounds to scipy (`L-BFGS-B` supports them natively — they were simply
+never passed). `project` returns the clipped **mask** as well as the vector, so a
+caller can zero optimiser momentum along clipped directions.
+
+Default is `ClipperNone` and the PR is **bit-identical** with it. Flipping the
+default is PR 2 — `draft/feature/autofit/clipper_validation_campaign.md`, now
+unblocked.
+
+## The design decision worth remembering: inset by BOUND KIND, never by width
+
+The obvious implementation, one relative `margin * (upper - lower)`, is wrong in
+two separate and silent ways. Both stem from computing the width unconditionally.
+Three cases instead:
+
+| Bound kind | Example | Inset |
+|---|---|---|
+| two-sided finite | Uniform, LogUniform, TruncatedGaussian | relative `margin * width` |
+| unbounded | Gaussian | **none, and no width arithmetic at all** |
+| half-open, exclusive | LogGaussian's `0` | absolute `strict_epsilon` |
+
+And note the two-sided margin is **not** for prior support — measurement showed
+those bounds are *inclusive* (`UniformPrior.log_prior(2.0) = 0.0`,
+`TruncatedGaussian.log_prior(1.0) = -0.5`), so `margin=0` would be valid. It
+exists to avoid parking a lane exactly **on** a prior edge, where the model's own
+transforms are singular — the same reason the broad-start band defaults to the
+interior `(0.15, 0.85)` rather than `(0, 1)`.
+
+## Traps, all measured against a running install
+
+1. **The naive margin turns every unbounded prior into `NaN`.** `-inf + (inf -
+   -inf) * m` is `NaN`, and clipping against `NaN` bounds destroys the coordinate
+   and the whole objective (`sum(log_prior) = nan`). This would have made the
+   feature **actively harmful** on exactly the models it targets — the MGE
+   reference model carries `GaussianPrior`s — with a symptom indistinguishable
+   from the bug being fixed. Every bit-identity test still passes against it,
+   because `ClipperNone` never computes a margin.
+2. **scipy reads `bounds=(lower_array, upper_array)` as a sequence of `(min,
+   max)` PAIRS.** At n=2 it returns a silently wrong fit (`[0.,1.]` where the
+   answer is `[1.,1.]`), no error, no warning; at every other n it raises
+   `ValueError`. So it fails loudly for most models and silently for
+   two-parameter ones. Build an explicit `optimize.Bounds`. This was the
+   *prompt's own* specified return type.
+3. **`LogGaussianPrior` misreports its own support.** Its `TransformedMessage`
+   defaults limits to `±inf` and is never passed any, yet `log_prior_from_value`
+   is `-inf` for `value <= 0`. Declared in the clipper, prior class untouched.
+4. **Plain `BFGS` does not reject bounds — it IGNORES them** behind a
+   `UserWarning` and returns the unconstrained optimum. "Guard or warn" is too
+   weak; raise.
+5. **`prior.lower_limit` resolves for every prior type** via `Prior.__getattr__`
+   delegating to the message (`AbstractMessage` defaults `±inf`). No type switch
+   needed — except for trap 3.
+6. **The NumPy and JAX paths disagree on support.**
+   `UniformPrior.log_prior_from_value` is `if xp is np: return 0.0` —
+   unconditional, no bound test. Only the JAX branch walls off the box, so LBFGS
+   is exposed only in its `analysis._use_jax` branch.
+7. **float32 makes the box check asymmetric.** `2.0000001` is not representable
+   distinctly from `2.0` and reads as in-box, while `-1e-7` against a lower bound
+   of `0.0` is caught. A test asserting "overshoot is detected" must use a bound
+   near zero or float64, or it passes vacuously.
+8. **The `AbstractMultiStartGradient` class docstring was factually wrong** —
+   claimed the rule steps "on the unconstrained (unit-cube) parameterization"
+   while `_broad_starts` maps draws to physical. That is the sentence that would
+   tell the next reader this class of bug cannot exist. Corrected.
+
+## The process lesson: a green suite is not coverage
+
+The first commit shipped an **undefined `optimize` in `LBFGS._fit`** — any real
+`LBFGS.fit()` raised `NameError`. The **full 1790-test suite passed against it**,
+because nothing in the library suite ever executes an LBFGS fit. It was caught
+only by a randomised end-to-end stress run, after the code was already pushed.
+Fixed in the second commit with a smoke test that runs a real `LBFGS.fit()`,
+verified to fail with exactly that `NameError` if the import is removed again.
+
+When a change touches a path, check whether anything actually *executes* it
+before trusting the suite.
+
+## Verification performed (beyond the committed tests)
+
+- **Bit-identity 10/10 on both searches** across randomly generated models mixing
+  every prior type — `no clipper arg` vs explicit `ClipperNone`.
+- **Core promise 8/8** — with `ClipperPriorBox`, final `sum(log_prior)` finite
+  every time, lane deaths **0 in every case** vs 62–96 without.
+- **End-to-end**: Gaussian fit with the truth outside the box, lane deaths
+  **249 → 0** with 252 clips; the clipped run pins `centre` at the upper bound,
+  which is the correct MAP answer under a prior excluding the truth, and an
+  independent reproduction of the momentum pinning.
+- **Guards verified by inversion** — patching back to the naive width form makes
+  5 tests fail, including both named regression guards.
+- **Resume path** — a `search_internal` lacking `n_clipped_lane_steps` resumes
+  without `KeyError`.
+- **Identifiers unchanged** — real fits produce a single identifier dir shared by
+  `no clipper` / `ClipperNone` / `ClipperPriorBox`, so existing on-disk results
+  are not orphaned. Flip side: two runs differing only in clipper currently
+  COLLIDE on one output dir — matters for PR 2's re-baseline.
+
+## Harness traps that cost time (for whoever writes the PR 2 measurements)
+
+- **`.completed` marker short-circuits `fit()`** — a resumed or re-run search
+  returns the cached result without entering `_fit`. Three successive versions of
+  a resume test "passed" while testing nothing. Also bites when a script is
+  re-run with stale output from its previous execution.
+- **`fit()` rebuilds `search.paths`**, so an instance-level monkeypatch on
+  `paths.save_search_internal` is silently discarded. Patch at CLASS level.
+- **The search_internal folder is deleted on successful completion**, so it
+  cannot be read back after the fit — capture it as it is written.
+- **Two identically-constructed searches did not resolve to the same identifier
+  dir**, so "resume" silently started fresh. The reliable method is patching
+  `DirectoryPaths.load_search_internal` at class level.
+- **Seed `random` AND `numpy` before every fit** — the initializer draws from
+  both, and an unseeded comparison reports a spurious bit-identity mismatch.
+  (This produced one false alarm on the bit-identity gate.)
+- **A box containing the optimum never exercises the clipper.** The first
+  efficacy attempt measured 0 clips for exactly this reason; put the truth
+  outside the box.
+
+## Corrections issued
+
+`test_nautilus.py::test__single_core_builds_no_pool` **passes in CI**. It failed
+only in the local py3.12 venv used for verification. It was correctly identified
+as not caused by this task (verified by stashing), but was wrongly described as
+"pre-existing on clean main" in an earlier revision of the PR body and in the
+`active.md` notes; both were corrected.
+
+## Follow-ups owed (filed, not fixed)
+
+1. `float32` is not JSON serializable in result output —
+   `autofit/non_linear/paths/directory.py:80` `save_json` raises `TypeError` at
+   the end of a successful clipped run. Surfaced only because clipping let lanes
+   survive onto a code path this cell had never taken.
+2. A crashed run poisons the next run of the same name: the half-written output
+   from (1) makes the next search with the same `name` fail with
+   `JSONDecodeError` while resuming — a 4-second no-op that *looks like* a clean
+   result. A new form of the cached-result hazard in
+   `complete/2026/08/multistart-nan-step-diagnostics.md`.
+3. Declare `LogGaussianPrior`'s `(0, ∞)` support on the prior itself, retiring
+   the clipper's special case.
+4. Decide whether the clipper should enter the search identifier — relevant to
+   PR 2's benchmark re-baseline (see "Identifiers unchanged" above).
+5. **NUTS remains out of scope** — HMC entering a `-inf` region diverges rather
+   than freezing. Different mechanism, its own task.
+
+## Repos / worktree
+
+- PyAutoFit: `claude/autofit-clipper-prior-support-o3jotv` (merged, deletable).
+- No worktree was created — this ran in a cloud session from a direct clone at
+  `/workspace/pyautofit`.
+
+## Original prompt
+
 # Search-agnostic prior-support enforcement: a Clipper class
 
 Type: feature

complete/index.md

@@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema:
 only then grep a dated bucket. Curators: edit the band between the CURATED
 markers; everything below GENERATED is rebuilt.
 
-992 records across 7 buckets.
+993 records across 7 buckets.
 
 <!-- CURATED:START -->
 ## Highlights
@@ -24,6 +24,7 @@ _(curate hard-won records here — survives regeneration.)_
 - [autoarray-input-validation-guards](2026/08/autoarray-input-validation-guards.md)
 - [autocti-util-dataset-export](2026/08/autocti-util-dataset-export.md) — no GitHub issue — worked directly from the draft prompt in a cloud session on explicit human instruction; the …
 - [autofit-mock-all-ones-ell-comps](2026/08/autofit-mock-all-ones-ell-comps.md)
+- [autofit-prior-support-clipper](2026/08/autofit-prior-support-clipper.md)
 - [autogalaxy-assistant](2026/08/autogalaxy-assistant.md) — epic; human to close
 - [autogalaxy-profile-validation-guards](2026/08/autogalaxy-profile-validation-guards.md)
 - [autohands-firewall-allowlist](2026/08/autohands-firewall-allowlist.md) — auto-closed by the merge