From cfb88713c8740dcd9f7fcef769ea88eed898c0f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 00:25:05 +0000 Subject: [PATCH 1/8] prompt: route autofit-prior-support-clipper (#1476) -> start_library --- active.md | 19 +++++++++++++++++++ .../prior_support_clipper.md | 0 2 files changed, 19 insertions(+) rename {draft/feature/autofit => active}/prior_support_clipper.md (100%) diff --git a/active.md b/active.md index 421eb944..67ce396f 100644 --- a/active.md +++ b/active.md @@ -1,5 +1,24 @@ # Active Tasks +## autofit-prior-support-clipper +- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1476 +- prompt: active/prior_support_clipper.md +- session: claude cloud (web) 2026-08-16 +- status: library-dev +- worktree: ~/Code/PyAutoLabs-wt/autofit-prior-support-clipper (not created — this is a cloud session working from a direct clone at /workspace/pyautofit) +- repos: + - PyAutoFit: claude/autofit-clipper-prior-support-o3jotv +- scope: PR 1 only — `AbstractClipper` / `ClipperNone` / `ClipperPriorBox` plus opt-in wiring into `MultiStartGradient` and `LBFGS`. Default is `ClipperNone` and the PR must be BIT-IDENTICAL with it. Flipping the default is PR 2 (`draft/feature/autofit/clipper_validation_campaign.md`, held until this lands). +- upstream-cause: autolens_profiling#128 / the `mge-lane-death` entry below — the deaths are the PRIOR term, not the likelihood. This task is follow-up (1) owed by that investigation. +- ADVERSARIAL REVIEW BEFORE CODING (2026-08-16) — the plan was verified against a running PyAutoFit install (py3.12 venv, jax 0.11.0 / optax 0.2.8, CPU), not just read. Two of the prompt's own specifications were found to be SILENT-WRONG-ANSWER bugs and are corrected in the issue: + - **margin NaN**: `lower + margin*(upper-lower)` is `-inf + inf` = **NaN** for a `GaussianPrior`, and clipping against NaN bounds propagates (`sum(log_prior) = nan`). The naive form would KILL lanes on exactly the models this rescues — the MGE reference model carries GaussianPriors — with a symptom indistinguishable from the bug being fixed. Inset must be keyed on BOUND KIND, never on unguarded width arithmetic. + - **scipy bounds tuple**: the prompt's `bounds_from_model -> tuple[lower, upper]` is read by `optimize.minimize` as a sequence of `(min,max)` PAIRS. At n=2 it returns a silently wrong fit (`[0.,1.]` vs correct `[1.,1.]`), no error/warning; at every other n it raises `ValueError`. Must build `optimize.Bounds(lower, upper)` explicitly. +- design (settled by measurement, replaces the prompt's single `1e-6` guess): three cases — two-sided finite bounds get a RELATIVE inset (not for prior support, which is inclusive, but to avoid parking a lane on a prior edge where likelihood transforms are singular, same reason the start band is `(0.15,0.85)`); unbounded gets NO width arithmetic at all; half-open/exclusive (`LogGaussian`'s `0`) gets an ABSOLUTE `strict_epsilon`, since a relative margin is identically zero there and would clip onto `0.0` where `log_prior=-inf`. +- other measured findings: `prior.lower_limit` resolves for every prior type via `Prior.__getattr__` → message; `LogGaussianPrior` reports `(-inf,inf)` though its support is `(0,∞)` (special-cased in the Clipper, prior class untouched); plain `BFGS` does NOT reject bounds, it IGNORES them behind a `UserWarning` and returns the unbounded optimum (so raise); batched `(n_starts,n_params)` clipping broadcasts against `(n_params,)` bounds with no `vmap`; the momentum-pinning effect reproduces in isolation (7 of 8 steps pinned), confirming the returned mask is load-bearing. +- doc bug to fix in the PR: the `AbstractMultiStartGradient` class docstring says it steps "on the unconstrained (unit-cube) parameterization" — false, `_broad_starts` maps draws to PHYSICAL parameters. That sentence is what would tell the next reader this bug cannot exist. +- follow-ups to file, not fix: the two incidental bugs from #128 (float32 not JSON serializable in `paths/directory.py:80`; a crashed run poisoning the next run of the same name via `JSONDecodeError` on resume), plus declaring `LogGaussianPrior`'s `(0,∞)` support on the prior itself so the Clipper special case can be retired. +- NOT verified: the real `imaging/mge` cell was not run, and nothing was on GPU or in float64. The 60.25% → 17.71% / 14-16 → 5/16 figures are inherited from #128, which is why the regression test asserts a direction and a wide margin rather than a figure. + ## mge-lane-death - issue: https://github.com/PyAutoLabs/autolens_profiling/issues/128 - prompt: active/mge_lane_death.md diff --git a/draft/feature/autofit/prior_support_clipper.md b/active/prior_support_clipper.md similarity index 100% rename from draft/feature/autofit/prior_support_clipper.md rename to active/prior_support_clipper.md From 0d0f45868d59f5896cc9adda3e08a38a1755c9fc Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 00:36:44 +0000 Subject: [PATCH 2/8] prompt: record autofit-prior-support-clipper implementation (#1476, bf8c302) --- active.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/active.md b/active.md index 67ce396f..ba7a37d6 100644 --- a/active.md +++ b/active.md @@ -4,7 +4,12 @@ - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1476 - prompt: active/prior_support_clipper.md - session: claude cloud (web) 2026-08-16 -- status: library-dev +- status: library-dev — IMPLEMENTED and pushed 2026-08-16 as `bf8c302` on `claude/autofit-clipper-prior-support-o3jotv`. No PR opened (not requested). Remaining: human review, then PR + merge. +- validation: full `test_autofit` suite **1790 passed / 1 failed**, the one failure (`test_nautilus.py::test__single_core_builds_no_pool`) CONFIRMED PRE-EXISTING by re-running it with the changes stashed on clean `main`. New `test_autofit/non_linear/test_clipper.py` = 21 tests, all passing. +- bit-identity gate PASSED on both searches: `LBFGS` and `MultiStartAdam` each run with no clipper argument and with an explicit `ClipperNone` return identical final parameters. TRAP HIT ONCE — the first run of the gate reported a mismatch that was the HARNESS, not the code: the initializer draws its start from `random`/`numpy` and the two runs were not reseeded. Seed before each fit or the gate lies. +- end-to-end EFFICACY (Gaussian fit, truth deliberately OUTSIDE the prior box, 8 starts x 40 steps, CPU): value-NaN lane-steps **249 → 0**, with 252 clips recorded. The clipped run pins `centre` at 43.999986 against a `[30, 44]` box — the correct MAP answer under a prior that excludes the truth, and an independent reproduction of the momentum-pinning the prompt predicted. A first attempt at this check measured NOTHING (0 clips) because the box CONTAINED the optimum — a box the search never wants to leave does not exercise a clipper. +- regression guards verified by inversion: patching `ClipperPriorBox` back to the naive `margin * (upper - lower)` form makes 5 tests fail, including both named guards. The guards are load-bearing, not decorative. +- convention notes for the reviewer: `black` would reformat `bfgs/search.py` and `multi_start_gradient/search.py`, but BOTH were already unformatted on clean `main` — deliberately NOT reformatted, to keep the diff free of unrelated churn. `scipy` is imported lazily inside `_bounds_from` because no module in `autofit/` pulls scipy in at import time. Library tests stay NumPy-only per the note atop `test_multi_start_gradient.py` ("keeping JAX out of the library unit suite"); the JAX end-to-end checks above were run locally and are recorded here rather than committed. - worktree: ~/Code/PyAutoLabs-wt/autofit-prior-support-clipper (not created — this is a cloud session working from a direct clone at /workspace/pyautofit) - repos: - PyAutoFit: claude/autofit-clipper-prior-support-o3jotv From 3e91a0bc2b0b59e1d20c05ce03211ee89a711b8e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 02:19:08 +0000 Subject: [PATCH 3/8] prompt: record clipper stress-test findings and the LBFGS _fit fix (#1476, 4ef04e1) --- active.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/active.md b/active.md index ba7a37d6..41369b80 100644 --- a/active.md +++ b/active.md @@ -4,7 +4,11 @@ - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1476 - prompt: active/prior_support_clipper.md - session: claude cloud (web) 2026-08-16 -- status: library-dev — IMPLEMENTED and pushed 2026-08-16 as `bf8c302` on `claude/autofit-clipper-prior-support-o3jotv`. No PR opened (not requested). Remaining: human review, then PR + merge. +- status: library-dev — IMPLEMENTED and pushed 2026-08-16 as `bf8c302` + `4ef04e1` on `claude/autofit-clipper-prior-support-o3jotv`. No PR opened (not requested). Remaining: human review, then PR + merge. +- BUG CAUGHT POST-COMMIT by a randomised stress run, fixed in `4ef04e1` — `bf8c302` shipped an **undefined `optimize` in `LBFGS._fit`**, so ANY real `LBFGS.fit()` raised `NameError`. Cause: inserting the `bounds = self._bounds_from(model)` line replaced `_fit`'s local `from scipy import optimize`, and a module-level import added in the same edit masked it until that import was removed again (to match the convention that no autofit module imports scipy at import time). **The full 1790-test suite passed against this**, because the library suite never executes an LBFGS fit at all. Lesson: a green suite is not coverage — check whether the path you touched is executed by anything. A smoke test running a real `LBFGS.fit()` is now in `test_clipper.py` and was verified to fail with exactly this `NameError` when the import is removed again. +- stress-test results (randomised differential, 10 random models mixing every prior type): **bit-identity 10/10 on BOTH searches** (`no clipper arg` vs explicit `ClipperNone`); with `ClipperPriorBox`, final `sum(log_prior)` finite **8/8** and lane deaths **0 in every case** vs 62–96 without; does-no-harm holds (optimum inside the box → result unchanged); LBFGS end-to-end with a live clipper respects the box on all coordinates including the unbounded `Gaussian` (not NaN) and the half-open `LogGaussian`. +- IDENTIFIER SAFETY (checked because a resume test hinted at it): adding the `clipper` constructor arg does **NOT** change a search's identifier — verified with real fits, which produced a single identifier dir on disk shared by `no clipper` / `ClipperNone` / `ClipperPriorBox`. So existing on-disk results are not orphaned and resume still finds them. Note the flip side: two runs differing only in clipper currently COLLIDE on the same output dir, which matters for PR 2's benchmark re-baseline — decide there whether the clipper should enter the identifier. +- resume path (`.get(key, 0)` for `n_clipped_lane_steps`) VERIFIED: a legacy `search_internal` with the key stripped resumes without `KeyError`, continues past the restored step count, and the new counter appears. Took four attempts — three earlier versions silently tested NOTHING (a `.completed` marker short-circuits `fit()`; `fit()` rebuilds `paths` so an instance-level spy is discarded; and two identically-constructed searches did not resolve to the same identifier dir). The working method is to patch `DirectoryPaths.load_search_internal` at CLASS level to return the legacy dict. A "passing" resume test that never entered `_fit` is the trap here. - validation: full `test_autofit` suite **1790 passed / 1 failed**, the one failure (`test_nautilus.py::test__single_core_builds_no_pool`) CONFIRMED PRE-EXISTING by re-running it with the changes stashed on clean `main`. New `test_autofit/non_linear/test_clipper.py` = 21 tests, all passing. - bit-identity gate PASSED on both searches: `LBFGS` and `MultiStartAdam` each run with no clipper argument and with an explicit `ClipperNone` return identical final parameters. TRAP HIT ONCE — the first run of the gate reported a mismatch that was the HARNESS, not the code: the initializer draws its start from `random`/`numpy` and the two runs were not reseeded. Seed before each fit or the gate lies. - end-to-end EFFICACY (Gaussian fit, truth deliberately OUTSIDE the prior box, 8 starts x 40 steps, CPU): value-NaN lane-steps **249 → 0**, with 252 clips recorded. The clipped run pins `centre` at 43.999986 against a `[30, 44]` box — the correct MAP answer under a prior that excludes the truth, and an independent reproduction of the momentum-pinning the prompt predicted. A first attempt at this check measured NOTHING (0 clips) because the box CONTAINED the optimum — a box the search never wants to leave does not exercise a clipper. From d6a4341df9af52710a4d05cd296aa3921dd38144 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 02:46:53 +0000 Subject: [PATCH 4/8] prompt: record PR #1477 open with full CI green (autofit-prior-support-clipper) --- active.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/active.md b/active.md index 41369b80..bac157e1 100644 --- a/active.md +++ b/active.md @@ -4,7 +4,10 @@ - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1476 - prompt: active/prior_support_clipper.md - session: claude cloud (web) 2026-08-16 -- status: library-dev — IMPLEMENTED and pushed 2026-08-16 as `bf8c302` + `4ef04e1` on `claude/autofit-clipper-prior-support-o3jotv`. No PR opened (not requested). Remaining: human review, then PR + merge. +- pr: https://github.com/PyAutoLabs/PyAutoFit/pull/1477 (OPEN, mergeable_state `clean`, 3 commits, +798/-3 over 7 files). Session is subscribed to its PR activity. +- status: library-dev — IMPLEMENTED and pushed 2026-08-16 as `bf8c302` + `4ef04e1` + `7cb84f7` on `claude/autofit-clipper-prior-support-o3jotv`. **FULL CI GREEN** on head `7cb84f7`: `unittest (3.12)`, `unittest (3.13)`, `docs / docs-build` all success. Remaining: human review, then merge. +- CI CORRECTION, important for anyone reading the earlier notes: `test_nautilus.py::test__single_core_builds_no_pool` **PASSES in CI**. It failed only in the local py3.12 venv used for verification (missing optional dep, most likely). The earlier claim that it was "pre-existing on clean main" was right that it was not caused by this task — verified by stashing — but wrong to imply the repo carries a failing test. Do not propagate that. The PR description was edited to carry this correction too. +- docs: `ClipperNone`/`ClipperPriorBox` were exported into the `af` namespace without being added to `docs/api/searches.rst`, where their `Initializer*` siblings live — fixed in `7cb84f7`, pushed as its own commit BECAUSE the docs job fails on a warning-count regression against `docs/sphinx_warning_baseline.txt` and this was the first time Sphinx processed the clipper docstrings. Kept attributable; the follow-up run was green, so no new warnings. - BUG CAUGHT POST-COMMIT by a randomised stress run, fixed in `4ef04e1` — `bf8c302` shipped an **undefined `optimize` in `LBFGS._fit`**, so ANY real `LBFGS.fit()` raised `NameError`. Cause: inserting the `bounds = self._bounds_from(model)` line replaced `_fit`'s local `from scipy import optimize`, and a module-level import added in the same edit masked it until that import was removed again (to match the convention that no autofit module imports scipy at import time). **The full 1790-test suite passed against this**, because the library suite never executes an LBFGS fit at all. Lesson: a green suite is not coverage — check whether the path you touched is executed by anything. A smoke test running a real `LBFGS.fit()` is now in `test_clipper.py` and was verified to fail with exactly this `NameError` when the import is removed again. - stress-test results (randomised differential, 10 random models mixing every prior type): **bit-identity 10/10 on BOTH searches** (`no clipper arg` vs explicit `ClipperNone`); with `ClipperPriorBox`, final `sum(log_prior)` finite **8/8** and lane deaths **0 in every case** vs 62–96 without; does-no-harm holds (optimum inside the box → result unchanged); LBFGS end-to-end with a live clipper respects the box on all coordinates including the unbounded `Gaussian` (not NaN) and the half-open `LogGaussian`. - IDENTIFIER SAFETY (checked because a resume test hinted at it): adding the `clipper` constructor arg does **NOT** change a search's identifier — verified with real fits, which produced a single identifier dir on disk shared by `no clipper` / `ClipperNone` / `ClipperPriorBox`. So existing on-disk results are not orphaned and resume still finds them. Note the flip side: two runs differing only in clipper currently COLLIDE on the same output dir, which matters for PR 2's benchmark re-baseline — decide there whether the clipper should enter the identifier. From 7077d63ba328710d67218e755b29322d9e8cf762 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 04:17:24 +0000 Subject: [PATCH 5/8] prompt: ship autofit-prior-support-clipper (#1476, PR #1477 merged 1f4b66a) --- active.md | 31 ---- .../2026/08/autofit-prior-support-clipper.md | 164 ++++++++++++++++++ complete/index.md | 3 +- dashboard.md | 13 +- 4 files changed, 172 insertions(+), 39 deletions(-) rename active/prior_support_clipper.md => complete/2026/08/autofit-prior-support-clipper.md (53%) diff --git a/active.md b/active.md index bac157e1..421eb944 100644 --- a/active.md +++ b/active.md @@ -1,36 +1,5 @@ # Active Tasks -## autofit-prior-support-clipper -- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1476 -- prompt: active/prior_support_clipper.md -- session: claude cloud (web) 2026-08-16 -- pr: https://github.com/PyAutoLabs/PyAutoFit/pull/1477 (OPEN, mergeable_state `clean`, 3 commits, +798/-3 over 7 files). Session is subscribed to its PR activity. -- status: library-dev — IMPLEMENTED and pushed 2026-08-16 as `bf8c302` + `4ef04e1` + `7cb84f7` on `claude/autofit-clipper-prior-support-o3jotv`. **FULL CI GREEN** on head `7cb84f7`: `unittest (3.12)`, `unittest (3.13)`, `docs / docs-build` all success. Remaining: human review, then merge. -- CI CORRECTION, important for anyone reading the earlier notes: `test_nautilus.py::test__single_core_builds_no_pool` **PASSES in CI**. It failed only in the local py3.12 venv used for verification (missing optional dep, most likely). The earlier claim that it was "pre-existing on clean main" was right that it was not caused by this task — verified by stashing — but wrong to imply the repo carries a failing test. Do not propagate that. The PR description was edited to carry this correction too. -- docs: `ClipperNone`/`ClipperPriorBox` were exported into the `af` namespace without being added to `docs/api/searches.rst`, where their `Initializer*` siblings live — fixed in `7cb84f7`, pushed as its own commit BECAUSE the docs job fails on a warning-count regression against `docs/sphinx_warning_baseline.txt` and this was the first time Sphinx processed the clipper docstrings. Kept attributable; the follow-up run was green, so no new warnings. -- BUG CAUGHT POST-COMMIT by a randomised stress run, fixed in `4ef04e1` — `bf8c302` shipped an **undefined `optimize` in `LBFGS._fit`**, so ANY real `LBFGS.fit()` raised `NameError`. Cause: inserting the `bounds = self._bounds_from(model)` line replaced `_fit`'s local `from scipy import optimize`, and a module-level import added in the same edit masked it until that import was removed again (to match the convention that no autofit module imports scipy at import time). **The full 1790-test suite passed against this**, because the library suite never executes an LBFGS fit at all. Lesson: a green suite is not coverage — check whether the path you touched is executed by anything. A smoke test running a real `LBFGS.fit()` is now in `test_clipper.py` and was verified to fail with exactly this `NameError` when the import is removed again. -- stress-test results (randomised differential, 10 random models mixing every prior type): **bit-identity 10/10 on BOTH searches** (`no clipper arg` vs explicit `ClipperNone`); with `ClipperPriorBox`, final `sum(log_prior)` finite **8/8** and lane deaths **0 in every case** vs 62–96 without; does-no-harm holds (optimum inside the box → result unchanged); LBFGS end-to-end with a live clipper respects the box on all coordinates including the unbounded `Gaussian` (not NaN) and the half-open `LogGaussian`. -- IDENTIFIER SAFETY (checked because a resume test hinted at it): adding the `clipper` constructor arg does **NOT** change a search's identifier — verified with real fits, which produced a single identifier dir on disk shared by `no clipper` / `ClipperNone` / `ClipperPriorBox`. So existing on-disk results are not orphaned and resume still finds them. Note the flip side: two runs differing only in clipper currently COLLIDE on the same output dir, which matters for PR 2's benchmark re-baseline — decide there whether the clipper should enter the identifier. -- resume path (`.get(key, 0)` for `n_clipped_lane_steps`) VERIFIED: a legacy `search_internal` with the key stripped resumes without `KeyError`, continues past the restored step count, and the new counter appears. Took four attempts — three earlier versions silently tested NOTHING (a `.completed` marker short-circuits `fit()`; `fit()` rebuilds `paths` so an instance-level spy is discarded; and two identically-constructed searches did not resolve to the same identifier dir). The working method is to patch `DirectoryPaths.load_search_internal` at CLASS level to return the legacy dict. A "passing" resume test that never entered `_fit` is the trap here. -- validation: full `test_autofit` suite **1790 passed / 1 failed**, the one failure (`test_nautilus.py::test__single_core_builds_no_pool`) CONFIRMED PRE-EXISTING by re-running it with the changes stashed on clean `main`. New `test_autofit/non_linear/test_clipper.py` = 21 tests, all passing. -- bit-identity gate PASSED on both searches: `LBFGS` and `MultiStartAdam` each run with no clipper argument and with an explicit `ClipperNone` return identical final parameters. TRAP HIT ONCE — the first run of the gate reported a mismatch that was the HARNESS, not the code: the initializer draws its start from `random`/`numpy` and the two runs were not reseeded. Seed before each fit or the gate lies. -- end-to-end EFFICACY (Gaussian fit, truth deliberately OUTSIDE the prior box, 8 starts x 40 steps, CPU): value-NaN lane-steps **249 → 0**, with 252 clips recorded. The clipped run pins `centre` at 43.999986 against a `[30, 44]` box — the correct MAP answer under a prior that excludes the truth, and an independent reproduction of the momentum-pinning the prompt predicted. A first attempt at this check measured NOTHING (0 clips) because the box CONTAINED the optimum — a box the search never wants to leave does not exercise a clipper. -- regression guards verified by inversion: patching `ClipperPriorBox` back to the naive `margin * (upper - lower)` form makes 5 tests fail, including both named guards. The guards are load-bearing, not decorative. -- convention notes for the reviewer: `black` would reformat `bfgs/search.py` and `multi_start_gradient/search.py`, but BOTH were already unformatted on clean `main` — deliberately NOT reformatted, to keep the diff free of unrelated churn. `scipy` is imported lazily inside `_bounds_from` because no module in `autofit/` pulls scipy in at import time. Library tests stay NumPy-only per the note atop `test_multi_start_gradient.py` ("keeping JAX out of the library unit suite"); the JAX end-to-end checks above were run locally and are recorded here rather than committed. -- worktree: ~/Code/PyAutoLabs-wt/autofit-prior-support-clipper (not created — this is a cloud session working from a direct clone at /workspace/pyautofit) -- repos: - - PyAutoFit: claude/autofit-clipper-prior-support-o3jotv -- scope: PR 1 only — `AbstractClipper` / `ClipperNone` / `ClipperPriorBox` plus opt-in wiring into `MultiStartGradient` and `LBFGS`. Default is `ClipperNone` and the PR must be BIT-IDENTICAL with it. Flipping the default is PR 2 (`draft/feature/autofit/clipper_validation_campaign.md`, held until this lands). -- upstream-cause: autolens_profiling#128 / the `mge-lane-death` entry below — the deaths are the PRIOR term, not the likelihood. This task is follow-up (1) owed by that investigation. -- ADVERSARIAL REVIEW BEFORE CODING (2026-08-16) — the plan was verified against a running PyAutoFit install (py3.12 venv, jax 0.11.0 / optax 0.2.8, CPU), not just read. Two of the prompt's own specifications were found to be SILENT-WRONG-ANSWER bugs and are corrected in the issue: - - **margin NaN**: `lower + margin*(upper-lower)` is `-inf + inf` = **NaN** for a `GaussianPrior`, and clipping against NaN bounds propagates (`sum(log_prior) = nan`). The naive form would KILL lanes on exactly the models this rescues — the MGE reference model carries GaussianPriors — with a symptom indistinguishable from the bug being fixed. Inset must be keyed on BOUND KIND, never on unguarded width arithmetic. - - **scipy bounds tuple**: the prompt's `bounds_from_model -> tuple[lower, upper]` is read by `optimize.minimize` as a sequence of `(min,max)` PAIRS. At n=2 it returns a silently wrong fit (`[0.,1.]` vs correct `[1.,1.]`), no error/warning; at every other n it raises `ValueError`. Must build `optimize.Bounds(lower, upper)` explicitly. -- design (settled by measurement, replaces the prompt's single `1e-6` guess): three cases — two-sided finite bounds get a RELATIVE inset (not for prior support, which is inclusive, but to avoid parking a lane on a prior edge where likelihood transforms are singular, same reason the start band is `(0.15,0.85)`); unbounded gets NO width arithmetic at all; half-open/exclusive (`LogGaussian`'s `0`) gets an ABSOLUTE `strict_epsilon`, since a relative margin is identically zero there and would clip onto `0.0` where `log_prior=-inf`. -- other measured findings: `prior.lower_limit` resolves for every prior type via `Prior.__getattr__` → message; `LogGaussianPrior` reports `(-inf,inf)` though its support is `(0,∞)` (special-cased in the Clipper, prior class untouched); plain `BFGS` does NOT reject bounds, it IGNORES them behind a `UserWarning` and returns the unbounded optimum (so raise); batched `(n_starts,n_params)` clipping broadcasts against `(n_params,)` bounds with no `vmap`; the momentum-pinning effect reproduces in isolation (7 of 8 steps pinned), confirming the returned mask is load-bearing. -- doc bug to fix in the PR: the `AbstractMultiStartGradient` class docstring says it steps "on the unconstrained (unit-cube) parameterization" — false, `_broad_starts` maps draws to PHYSICAL parameters. That sentence is what would tell the next reader this bug cannot exist. -- follow-ups to file, not fix: the two incidental bugs from #128 (float32 not JSON serializable in `paths/directory.py:80`; a crashed run poisoning the next run of the same name via `JSONDecodeError` on resume), plus declaring `LogGaussianPrior`'s `(0,∞)` support on the prior itself so the Clipper special case can be retired. -- NOT verified: the real `imaging/mge` cell was not run, and nothing was on GPU or in float64. The 60.25% → 17.71% / 14-16 → 5/16 figures are inherited from #128, which is why the regression test asserts a direction and a wide margin rather than a figure. - ## mge-lane-death - issue: https://github.com/PyAutoLabs/autolens_profiling/issues/128 - prompt: active/mge_lane_death.md diff --git a/active/prior_support_clipper.md b/complete/2026/08/autofit-prior-support-clipper.md similarity index 53% rename from active/prior_support_clipper.md rename to complete/2026/08/autofit-prior-support-clipper.md index 04954d4d..08db42a9 100644 --- a/active/prior_support_clipper.md +++ b/complete/2026/08/autofit-prior-support-clipper.md @@ -1,3 +1,167 @@ +Shipped the search-agnostic `Clipper` — prior-support enforcement for the +gradient searches, as the pluggable sibling of `Initializer`. This is follow-up +(1) owed by the `mge-lane-death` investigation (autolens_profiling#128), which +found the MGE lane deaths are the **prior** term, not the likelihood. + +- issue: PyAutoFit#1476 +- pr: PyAutoFit#1477, **MERGED** 2026-08-16 as `1f4b66a` (squash), +798/-3 over + 7 files. Full CI green on both runs (`unittest` 3.12, `unittest` 3.13, + `docs / docs-build`). + +## What shipped + +`AbstractClipper` / `ClipperNone` / `ClipperPriorBox` in a new +`autofit/non_linear/clipper.py`, wired **opt-in** into the two exposed searches: +`AbstractMultiStartGradient` projects after `optax.apply_updates`; `AbstractBFGS` +hands box bounds to scipy (`L-BFGS-B` supports them natively — they were simply +never passed). `project` returns the clipped **mask** as well as the vector, so a +caller can zero optimiser momentum along clipped directions. + +Default is `ClipperNone` and the PR is **bit-identical** with it. Flipping the +default is PR 2 — `draft/feature/autofit/clipper_validation_campaign.md`, now +unblocked. + +## The design decision worth remembering: inset by BOUND KIND, never by width + +The obvious implementation, one relative `margin * (upper - lower)`, is wrong in +two separate and silent ways. Both stem from computing the width unconditionally. +Three cases instead: + +| Bound kind | Example | Inset | +|---|---|---| +| two-sided finite | Uniform, LogUniform, TruncatedGaussian | relative `margin * width` | +| unbounded | Gaussian | **none, and no width arithmetic at all** | +| half-open, exclusive | LogGaussian's `0` | absolute `strict_epsilon` | + +And note the two-sided margin is **not** for prior support — measurement showed +those bounds are *inclusive* (`UniformPrior.log_prior(2.0) = 0.0`, +`TruncatedGaussian.log_prior(1.0) = -0.5`), so `margin=0` would be valid. It +exists to avoid parking a lane exactly **on** a prior edge, where the model's own +transforms are singular — the same reason the broad-start band defaults to the +interior `(0.15, 0.85)` rather than `(0, 1)`. + +## Traps, all measured against a running install + +1. **The naive margin turns every unbounded prior into `NaN`.** `-inf + (inf - + -inf) * m` is `NaN`, and clipping against `NaN` bounds destroys the coordinate + and the whole objective (`sum(log_prior) = nan`). This would have made the + feature **actively harmful** on exactly the models it targets — the MGE + reference model carries `GaussianPrior`s — with a symptom indistinguishable + from the bug being fixed. Every bit-identity test still passes against it, + because `ClipperNone` never computes a margin. +2. **scipy reads `bounds=(lower_array, upper_array)` as a sequence of `(min, + max)` PAIRS.** At n=2 it returns a silently wrong fit (`[0.,1.]` where the + answer is `[1.,1.]`), no error, no warning; at every other n it raises + `ValueError`. So it fails loudly for most models and silently for + two-parameter ones. Build an explicit `optimize.Bounds`. This was the + *prompt's own* specified return type. +3. **`LogGaussianPrior` misreports its own support.** Its `TransformedMessage` + defaults limits to `±inf` and is never passed any, yet `log_prior_from_value` + is `-inf` for `value <= 0`. Declared in the clipper, prior class untouched. +4. **Plain `BFGS` does not reject bounds — it IGNORES them** behind a + `UserWarning` and returns the unconstrained optimum. "Guard or warn" is too + weak; raise. +5. **`prior.lower_limit` resolves for every prior type** via `Prior.__getattr__` + delegating to the message (`AbstractMessage` defaults `±inf`). No type switch + needed — except for trap 3. +6. **The NumPy and JAX paths disagree on support.** + `UniformPrior.log_prior_from_value` is `if xp is np: return 0.0` — + unconditional, no bound test. Only the JAX branch walls off the box, so LBFGS + is exposed only in its `analysis._use_jax` branch. +7. **float32 makes the box check asymmetric.** `2.0000001` is not representable + distinctly from `2.0` and reads as in-box, while `-1e-7` against a lower bound + of `0.0` is caught. A test asserting "overshoot is detected" must use a bound + near zero or float64, or it passes vacuously. +8. **The `AbstractMultiStartGradient` class docstring was factually wrong** — + claimed the rule steps "on the unconstrained (unit-cube) parameterization" + while `_broad_starts` maps draws to physical. That is the sentence that would + tell the next reader this class of bug cannot exist. Corrected. + +## The process lesson: a green suite is not coverage + +The first commit shipped an **undefined `optimize` in `LBFGS._fit`** — any real +`LBFGS.fit()` raised `NameError`. The **full 1790-test suite passed against it**, +because nothing in the library suite ever executes an LBFGS fit. It was caught +only by a randomised end-to-end stress run, after the code was already pushed. +Fixed in the second commit with a smoke test that runs a real `LBFGS.fit()`, +verified to fail with exactly that `NameError` if the import is removed again. + +When a change touches a path, check whether anything actually *executes* it +before trusting the suite. + +## Verification performed (beyond the committed tests) + +- **Bit-identity 10/10 on both searches** across randomly generated models mixing + every prior type — `no clipper arg` vs explicit `ClipperNone`. +- **Core promise 8/8** — with `ClipperPriorBox`, final `sum(log_prior)` finite + every time, lane deaths **0 in every case** vs 62–96 without. +- **End-to-end**: Gaussian fit with the truth outside the box, lane deaths + **249 → 0** with 252 clips; the clipped run pins `centre` at the upper bound, + which is the correct MAP answer under a prior excluding the truth, and an + independent reproduction of the momentum pinning. +- **Guards verified by inversion** — patching back to the naive width form makes + 5 tests fail, including both named regression guards. +- **Resume path** — a `search_internal` lacking `n_clipped_lane_steps` resumes + without `KeyError`. +- **Identifiers unchanged** — real fits produce a single identifier dir shared by + `no clipper` / `ClipperNone` / `ClipperPriorBox`, so existing on-disk results + are not orphaned. Flip side: two runs differing only in clipper currently + COLLIDE on one output dir — matters for PR 2's re-baseline. + +## Harness traps that cost time (for whoever writes the PR 2 measurements) + +- **`.completed` marker short-circuits `fit()`** — a resumed or re-run search + returns the cached result without entering `_fit`. Three successive versions of + a resume test "passed" while testing nothing. Also bites when a script is + re-run with stale output from its previous execution. +- **`fit()` rebuilds `search.paths`**, so an instance-level monkeypatch on + `paths.save_search_internal` is silently discarded. Patch at CLASS level. +- **The search_internal folder is deleted on successful completion**, so it + cannot be read back after the fit — capture it as it is written. +- **Two identically-constructed searches did not resolve to the same identifier + dir**, so "resume" silently started fresh. The reliable method is patching + `DirectoryPaths.load_search_internal` at class level. +- **Seed `random` AND `numpy` before every fit** — the initializer draws from + both, and an unseeded comparison reports a spurious bit-identity mismatch. + (This produced one false alarm on the bit-identity gate.) +- **A box containing the optimum never exercises the clipper.** The first + efficacy attempt measured 0 clips for exactly this reason; put the truth + outside the box. + +## Corrections issued + +`test_nautilus.py::test__single_core_builds_no_pool` **passes in CI**. It failed +only in the local py3.12 venv used for verification. It was correctly identified +as not caused by this task (verified by stashing), but was wrongly described as +"pre-existing on clean main" in an earlier revision of the PR body and in the +`active.md` notes; both were corrected. + +## Follow-ups owed (filed, not fixed) + +1. `float32` is not JSON serializable in result output — + `autofit/non_linear/paths/directory.py:80` `save_json` raises `TypeError` at + the end of a successful clipped run. Surfaced only because clipping let lanes + survive onto a code path this cell had never taken. +2. A crashed run poisons the next run of the same name: the half-written output + from (1) makes the next search with the same `name` fail with + `JSONDecodeError` while resuming — a 4-second no-op that *looks like* a clean + result. A new form of the cached-result hazard in + `complete/2026/08/multistart-nan-step-diagnostics.md`. +3. Declare `LogGaussianPrior`'s `(0, ∞)` support on the prior itself, retiring + the clipper's special case. +4. Decide whether the clipper should enter the search identifier — relevant to + PR 2's benchmark re-baseline (see "Identifiers unchanged" above). +5. **NUTS remains out of scope** — HMC entering a `-inf` region diverges rather + than freezing. Different mechanism, its own task. + +## Repos / worktree + +- PyAutoFit: `claude/autofit-clipper-prior-support-o3jotv` (merged, deletable). +- No worktree was created — this ran in a cloud session from a direct clone at + `/workspace/pyautofit`. + +## Original prompt + # Search-agnostic prior-support enforcement: a Clipper class Type: feature diff --git a/complete/index.md b/complete/index.md index aeaa4626..9767675c 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -992 records across 7 buckets. +993 records across 7 buckets. ## Highlights @@ -24,6 +24,7 @@ _(curate hard-won records here — survives regeneration.)_ - [autoarray-input-validation-guards](2026/08/autoarray-input-validation-guards.md) - [autocti-util-dataset-export](2026/08/autocti-util-dataset-export.md) — no GitHub issue — worked directly from the draft prompt in a cloud session on explicit human instruction; the … - [autofit-mock-all-ones-ell-comps](2026/08/autofit-mock-all-ones-ell-comps.md) +- [autofit-prior-support-clipper](2026/08/autofit-prior-support-clipper.md) - [autogalaxy-assistant](2026/08/autogalaxy-assistant.md) — epic; human to close - [autogalaxy-profile-validation-guards](2026/08/autogalaxy-profile-validation-guards.md) - [autohands-firewall-allowlist](2026/08/autohands-firewall-allowlist.md) — auto-closed by the merge diff --git a/dashboard.md b/dashboard.md index 1ba7b353..771bb33c 100644 --- a/dashboard.md +++ b/dashboard.md @@ -1,6 +1,6 @@ # PyAutoMind task dashboard - + Every task the Mind is holding, on one page: what is in flight, what is parked, and the whole backlog to pick from. Pick a line, then run `/start_dev ` to start it. @@ -11,18 +11,17 @@ Tasks only — the organism's health lives with the Heart (`/health`), not here. | [In flight](#in-flight) (`active/`) | 8 | | [Parked](#parked) (`parked.md`) | 6 | | [Planned](#planned) (`planned.md`) | 7 | -| [Backlog](#backlog) (`draft/`) | 138 | +| [Backlog](#backlog) (`draft/`) | 137 | Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3Aissue+is%3Aopen&type=issues) · [open pull requests](https://github.com/search?q=org%3APyAutoLabs+is%3Apr+is%3Aopen&type=prs) ## Start here -**Highest priority** (filed as `high`) — showing 12 of 32 +**Highest priority** (filed as `high`) — showing 12 of 31 - [pre_build stages untracked files, publishing uncommitted human work](draft/bug/pyautohands/pre_build_stages_untracked_wip.md) — pyautohands · small · supervised · high - [TRIAGE: needs manual review before routing](draft/triage/jax_zero_contour.md) — medium · safe · high - [Clipper: demonstrate and validate on the profiling search cells](draft/feature/autofit/clipper_validation_campaign.md) — autofit · medium · supervised · high -- [Search-agnostic prior-support enforcement: a Clipper class](draft/feature/autofit/prior_support_clipper.md) — autofit · medium · supervised · high - [PyAutoLens RTD docs: three-regime restructure (multi_galaxy / group / cluster)](draft/docs/autolens/docs_three_regime_restructure.md) — autolens · medium · supervised · high - [Optimize pixelized Prodigy settings on the laptop GPU](draft/research/autolens_workspace_developer/pixelized_prodigy_laptop_gpu_phase_2_settings.md) — autolens_workspace_developer · medium · human-required · high - [Release does not sync __version__ stamps and workspace pins back](draft/bug/pyautobuild/release_version_sync_back_to_main.md) — pyautobuild · medium · supervised · high @@ -31,6 +30,7 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A - [Cluster package: point-source-default narrative + extended-source follow-up feature](draft/docs/workspaces/cluster_regime_narrative.md) — workspaces · medium · supervised · high - [multi_galaxy package: new regime package in autolens_workspace](draft/docs/autolens/multi_galaxy_package.md) — autolens · large · supervised · high - [Tune the JAX multi-start optimizers into a standard option (MGE](draft/experiment/autolens_profiling/jax_optimizer_settings_tuning.md) — autolens_profiling · large · supervised · high +- [Profile and speed up JAX likelihood-function compile times (all use](draft/feature/autolens_profiling/jax_compile_time_profiling.md) — autolens_profiling · large · supervised · high **Quick wins** (small enough, and safe enough to run unattended) @@ -95,7 +95,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**138** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). +**137** filed prompts, not started. Each section is sorted most-pickable first (priority, then size).
bug — 40 @@ -144,10 +144,9 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-feature — 27 +feature — 26 - [Clipper: demonstrate and validate on the profiling search cells](draft/feature/autofit/clipper_validation_campaign.md) — autofit · medium · supervised · high -- [Search-agnostic prior-support enforcement: a Clipper class](draft/feature/autofit/prior_support_clipper.md) — autofit · medium · supervised · high - [Make draft/ staleness detectable — `intake reconcile` measured, and the](draft/feature/pyautomind/draft_staleness_detection_signals.md) — pyautomind · medium · supervised · high - [LACosmic per-frame CR masking option + decouple PSF-star pass from](draft/feature/pyautoreduce/lacosmic_cr_option_and_star_pass_decoupling.md) — pyautoreduce · medium · supervised · high - [Profile and speed up JAX likelihood-function compile times (all use](draft/feature/autolens_profiling/jax_compile_time_profiling.md) — autolens_profiling · large · supervised · high From 5291270e39ae40ea4e5a840d0d097524a6b9df81 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 04:53:12 +0000 Subject: [PATCH 6/8] prompt: carry phase-1 findings into the clipper validation campaign (arm collision, new counter, missing momentum reset) --- .../autofit/clipper_validation_campaign.md | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/draft/feature/autofit/clipper_validation_campaign.md b/draft/feature/autofit/clipper_validation_campaign.md index efdacdcd..ff7501a3 100644 --- a/draft/feature/autofit/clipper_validation_campaign.md +++ b/draft/feature/autofit/clipper_validation_campaign.md @@ -164,6 +164,70 @@ not swept up as a clipping artefact. - **`0` and `null` are different findings.** Read counters with `.get()` and a `null` means the search never wrote the key — broken plumbing, not a clean cell. +## What phase 1 actually shipped (added 2026-08-16, after it landed) + +Phase 1 is **MERGED**: PyAutoFit#1477 as `1f4b66a`, closing #1476. Record: +`complete/2026/08/autofit-prior-support-clipper.md`. Four things change how this +task must be run: + +1. **Arm 3 (momentum reset) does NOT exist yet.** Phase 1 ships the clipped + **mask** from `project`, which is what a momentum reset would need — but no + reset is implemented, and the search does not touch `opt_state` on a clip. + So arm 3 either gets a small addition first, or this campaign runs arms 1-2 + and reports the pinned-lane count as the input to that decision. Do not + silently drop it: the prototype's 5/16 pinned lanes are why the arm exists. + +2. **A new counter to record: `n_clipped_lane_steps`**, in `search_internal` + alongside the existing three. Per-LANE per-step (`any(mask, axis=-1)`), same + convention as the others. It is the direct measure of how hard the wall is + being hit, and it distinguishes "clipping rarely fires" from "clipping fires + constantly and the population is living on the bound". + +3. **THE ARM-COLLISION HAZARD — read before running anything.** Verified with + real fits during phase 1: the `clipper` does **not** enter the search + identifier. `no clipper` / `ClipperNone` / `ClipperPriorBox` all resolve to + **the same output directory**. That is good for back-compat (existing results + are not orphaned) and *actively dangerous here*, because arms 1 and 2 differ + in exactly nothing else. Stacked with two other confirmed behaviours: + - a `.completed` marker makes `fit()` **short-circuit and return the cached + result without entering `_fit`**, and + - the `search_internal` folder is **deleted on successful completion**, so it + cannot be read back afterwards, + + the failure mode is that **arm 2 silently returns arm 1's numbers**, looking + like a clean run. This is the same cached-result hazard #128 already recorded, + with a new and much sharper trigger. Mitigation, all three: give every arm a + **unique `name`**, delete `output//` between arms, and assert the + recorded `total_steps == n_steps` before believing any counter. Also worth + deciding as a deliverable: *should* the clipper enter the identifier? Phase 3 + wants a re-baseline, and colliding arms make that harder. + +4. **The `float32` `save_json` bug is NOT fixed** (filed as a follow-up, not + carried by phase 1). So this prompt's trap stands and applies: capture the + counters **independently of the result object**. The method proven in phase 1 + is to patch `DirectoryPaths.save_search_internal` at **class** level and copy + the dict as it is written — instance-level patching is silently discarded, + because `fit()` rebuilds `search.paths`. + +Two more phase-1 measurements worth carrying in as priors, both CPU/float32 on a +toy Gaussian, so directional only: + +- With the truth deliberately **outside** the prior box, lane deaths went + **249 → 0** with 252 clips, and the clipped run pinned `centre` at the upper + bound. That is the "pinning is a result, not a failure" case reproducing in + miniature, and an independent confirmation of the momentum-pinning mechanism. +- The **negative control this prompt asks for already passes at unit level**: a + `GaussianPrior` coordinate is provably untouched by `ClipperPriorBox`, and there + is a regression test guarding it. The cell-level negative control is still + worth running, but a failure there would point at the cell, not the bounds + extraction. + +Also fixed in phase 1 and relevant when reading the source: the +`AbstractMultiStartGradient` class docstring used to claim the rule steps "on the +unconstrained (unit-cube) parameterization". It never did — `_broad_starts` maps +draws to physical parameters. Corrected, but older checkouts still carry the false +sentence. + ## Deliberately out of scope - Flipping any default (phase 3). From 7b27bda81d9dda72c23069308570d06eb7ab5dd5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 06:44:27 +0000 Subject: [PATCH 7/8] prompt: register clipper-validation-campaign in progress (autolens_profiling#129) with handoff notes --- active.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/active.md b/active.md index 421eb944..429e5d3c 100644 --- a/active.md +++ b/active.md @@ -1,5 +1,25 @@ # Active Tasks +## clipper-validation-campaign +- issue: https://github.com/PyAutoLabs/autolens_profiling/issues/129 +- prompt: draft/feature/autofit/clipper_validation_campaign.md (still in draft/ — NOT advanced to active/, because the campaign is part-run, not issued-and-owned; move it if you formally take the task) +- session: claude cloud (web) 2026-08-16 — **HANDOFF, work part-done, read this whole entry before running anything** +- status: IN PROGRESS — machinery built and first arm pair measured; campaign NOT complete and NO PR opened +- upstream: phase 1 shipped as PyAutoFit#1477 (`1f4b66a`), record `complete/2026/08/autofit-prior-support-clipper.md`. Root cause autolens_profiling#128 / the `mge-lane-death` entry below. +- repos: + - autolens_profiling: claude/clipper-validation-campaign-o3jotv (pushed, 1 commit) +- HEADLINE RESULT (imaging/mge hst, 16x150, single seed, cloud CPU float32): value-NaN lane-steps **2268 -> 47** with 1920 clips, and `max_log_likelihood` is **IDENTICAL between arms to every printed digit** (-15529.587986751998). 11 of 16 lanes end pinned to a bound. That fires **two of the four pre-registered falsification conditions**. **DO NOT WRITE PHASE 3 ON THIS EVIDENCE.** +- NOT the caching artefact — this was the predicted way to fake an identical result, so it was checked first: the runs differ in value-NaN (2268 vs 47), grad-NaN (0 vs 9) and wall (286.5s vs 249.6s). Arm 2 genuinely ran and converged to the same best point. Likely mechanism: the winning lane never left the prior box, so clipping never touched it. +- THE CAVEAT THAT LIMITS EVERYTHING: both arms are **~47,316 nats** from the Nautilus bar (31786.782462). Neither is converged, so the comparison happens in a regime where the search has not found the basin at all. The truth-recovery claim being graded against came from GPU/Prodigy/longer budgets. The load-bearing question is **unanswered, not answered negatively**. Do not report this as "clipping doesn't work". +- BLOCKER, needs a decision: **multi-seed is currently impossible.** `_broad_starts` seeds with a hardcoded `np.random.default_rng(0)`, so every run draws IDENTICAL starts; seeding `random`/`numpy` perturbs only the initializer. The campaign requires >=2 seeds per arm. Fix is a small `seed` arg on `AbstractMultiStartGradient` — a PyAutoFit change, so its own prompt/PR. **The human was asked and has not answered yet.** +- BLOCKER: the **momentum-reset arm does not exist**. Phase 1 ships the clipped mask it needs, but no reset. 11/16 pinned makes this the decisive arm. +- what was built (all on the branch): `scripts/misc/searches/clipper_campaign.py` (the arm driver) and a `SEARCHES_CLIPPER=none|prior_box` knob in `scripts/misc/searches/_samplers.py`. Findings note at `results/notes/clipper_campaign/README.md`, raw rows in the sibling JSON. +- why a bespoke driver: the ordinary `searches/` runner records **none** of the lane counters, and `search_internal` (where they live) is **deleted on successful completion**. The driver captures it as written, patching `save_search_internal` at **CLASS** level — `fit()` rebuilds `search.paths`, so an instance-level hook is silently discarded. +- ENVIRONMENT (the fiddly part, reproduce exactly): py3.12 venv; `pip install autolens jaxnnls`; **then** `pip install -e --no-deps`; then assert `autofit.__file__` resolves to the checkout. The clipper is UNRELEASED, so a PyPI autofit silently has no `clipper` argument and every number would be meaningless. In this session the venv was `/tmp/.../scratchpad/venv` with PyAutoFit at `/workspace/pyautofit` — both ephemeral, so a new session rebuilds. +- cost/feasibility measured: one 16x150 arm is ~250-290s wall on this cloud CPU box, and JIT compile dominates short runs (4x5 took ~128s). Two arms ~9 min. GPU cells (pixelized meshes) are NOT runnable from a cloud session — no HPC/SSH access. +- next, in order: (1) a **converged budget** (GPU / more steps / Prodigy) so the comparison is meaningful; (2) the `seed` arg, then >=2 seeds/arm; (3) the momentum-reset arm; (4) remaining cells — pixelized (GPU), point_source, and the unbounded-prior negative control. +- do-not: do NOT open a PR on the campaign branch yet — the campaign is part-run and the note would read as a verdict. Do NOT flip any default. + ## mge-lane-death - issue: https://github.com/PyAutoLabs/autolens_profiling/issues/128 - prompt: active/mge_lane_death.md From 3de55797356a5d7951628e04220b47e77cd3cc64 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 06:44:53 +0000 Subject: [PATCH 8/8] prompt: advance clipper_validation_campaign to active/ (issued as autolens_profiling#129) --- active.md | 2 +- .../clipper_validation_campaign.md | 0 dashboard.md | 14 +++++++------- 3 files changed, 8 insertions(+), 8 deletions(-) rename {draft/feature/autofit => active}/clipper_validation_campaign.md (100%) diff --git a/active.md b/active.md index 429e5d3c..63778b07 100644 --- a/active.md +++ b/active.md @@ -2,7 +2,7 @@ ## clipper-validation-campaign - issue: https://github.com/PyAutoLabs/autolens_profiling/issues/129 -- prompt: draft/feature/autofit/clipper_validation_campaign.md (still in draft/ — NOT advanced to active/, because the campaign is part-run, not issued-and-owned; move it if you formally take the task) +- prompt: active/clipper_validation_campaign.md (issued as autolens_profiling#129; advanced to active/ on filing, per the lifecycle) - session: claude cloud (web) 2026-08-16 — **HANDOFF, work part-done, read this whole entry before running anything** - status: IN PROGRESS — machinery built and first arm pair measured; campaign NOT complete and NO PR opened - upstream: phase 1 shipped as PyAutoFit#1477 (`1f4b66a`), record `complete/2026/08/autofit-prior-support-clipper.md`. Root cause autolens_profiling#128 / the `mge-lane-death` entry below. diff --git a/draft/feature/autofit/clipper_validation_campaign.md b/active/clipper_validation_campaign.md similarity index 100% rename from draft/feature/autofit/clipper_validation_campaign.md rename to active/clipper_validation_campaign.md diff --git a/dashboard.md b/dashboard.md index 771bb33c..07889103 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,20 +8,19 @@ Tasks only — the organism's health lives with the Heart (`/health`), not here. | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 8 | +| [In flight](#in-flight) (`active/`) | 9 | | [Parked](#parked) (`parked.md`) | 6 | | [Planned](#planned) (`planned.md`) | 7 | -| [Backlog](#backlog) (`draft/`) | 137 | +| [Backlog](#backlog) (`draft/`) | 136 | Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3Aissue+is%3Aopen&type=issues) · [open pull requests](https://github.com/search?q=org%3APyAutoLabs+is%3Apr+is%3Aopen&type=prs) ## Start here -**Highest priority** (filed as `high`) — showing 12 of 31 +**Highest priority** (filed as `high`) — showing 12 of 30 - [pre_build stages untracked files, publishing uncommitted human work](draft/bug/pyautohands/pre_build_stages_untracked_wip.md) — pyautohands · small · supervised · high - [TRIAGE: needs manual review before routing](draft/triage/jax_zero_contour.md) — medium · safe · high -- [Clipper: demonstrate and validate on the profiling search cells](draft/feature/autofit/clipper_validation_campaign.md) — autofit · medium · supervised · high - [PyAutoLens RTD docs: three-regime restructure (multi_galaxy / group / cluster)](draft/docs/autolens/docs_three_regime_restructure.md) — autolens · medium · supervised · high - [Optimize pixelized Prodigy settings on the laptop GPU](draft/research/autolens_workspace_developer/pixelized_prodigy_laptop_gpu_phase_2_settings.md) — autolens_workspace_developer · medium · human-required · high - [Release does not sync __version__ stamps and workspace pins back](draft/bug/pyautobuild/release_version_sync_back_to_main.md) — pyautobuild · medium · supervised · high @@ -31,6 +30,7 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A - [multi_galaxy package: new regime package in autolens_workspace](draft/docs/autolens/multi_galaxy_package.md) — autolens · large · supervised · high - [Tune the JAX multi-start optimizers into a standard option (MGE](draft/experiment/autolens_profiling/jax_optimizer_settings_tuning.md) — autolens_profiling · large · supervised · high - [Profile and speed up JAX likelihood-function compile times (all use](draft/feature/autolens_profiling/jax_compile_time_profiling.md) — autolens_profiling · large · supervised · high +- [Optimize MultiStartProdigy for pixelized meshes on the laptop GPU](draft/research/autolens_workspace_developer/pixelized_prodigy_laptop_gpu.md) — autolens_workspace_developer · large · human-required · high **Quick wins** (small enough, and safe enough to run unattended) @@ -52,6 +52,7 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A Issued — each has an open GitHub issue and usually a branch. The full record for each is in [`active.md`](active.md). - [First benchmark calibration campaign — run the 4 assistant benchmarks](active/benchmark_calibration_runs.md) +- [Clipper: demonstrate and validate on the profiling search cells](active/clipper_validation_campaign.md) - [Address ECEB editorial comments on ECLIPSE-C](active/euclid_eceb_editorial_revision.md) - [JAX-native posterior sampler wave — ranked shortlist from the 2026-07-16](active/jax_native_posterior_sampler_wave.md) — [issue #113](https://github.com/PyAutoLabs/autolens_workspace_developer/issues/113) — PARKED 2026-07-24 — stage (a) POSITIVE: warm-started gradient SMC SAMPLES (acc 0.80->0.17 across tempering, einstein_radius… - [Remove standalone matplotlib-inline comments](active/matplotlib_inline_standalones.md) @@ -95,7 +96,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**137** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). +**136** filed prompts, not started. Each section is sorted most-pickable first (priority, then size).
bug — 40 @@ -144,9 +145,8 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-feature — 26 +feature — 25 -- [Clipper: demonstrate and validate on the profiling search cells](draft/feature/autofit/clipper_validation_campaign.md) — autofit · medium · supervised · high - [Make draft/ staleness detectable — `intake reconcile` measured, and the](draft/feature/pyautomind/draft_staleness_detection_signals.md) — pyautomind · medium · supervised · high - [LACosmic per-frame CR masking option + decouple PSF-star pass from](draft/feature/pyautoreduce/lacosmic_cr_option_and_star_pass_decoupling.md) — pyautoreduce · medium · supervised · high - [Profile and speed up JAX likelihood-function compile times (all use](draft/feature/autolens_profiling/jax_compile_time_profiling.md) — autolens_profiling · large · supervised · high