From 544d095e1a6f274cf5e86e0a27f7c48f7d18e482 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 22 Aug 2026 13:06:22 +0000 Subject: [PATCH 1/3] =?UTF-8?q?prompt:=20route=20jax-grad-local-vs-ci-asse?= =?UTF-8?q?rtions=20(#260)=20=E2=86=92=20start=5Fworkspace?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- active.md | 19 +++++++++++++++ ...ad_local_assertions_fail_but_pass_in_ci.md | 0 dashboard.html | 8 +++---- dashboard.md | 24 +++++++++---------- 4 files changed, 35 insertions(+), 16 deletions(-) rename {draft/bug/autolens_workspace_test => active}/jax_grad_local_assertions_fail_but_pass_in_ci.md (100%) diff --git a/active.md b/active.md index ab06b16d..c43d5c96 100644 --- a/active.md +++ b/active.md @@ -24,3 +24,22 @@ separate /repo_cleanup sweep so a destructive branch delete never rides a code diff. - repos: - PyAutoHands: feature/hands-hygiene-leftovers + +## jax-grad-local-vs-ci-assertions +- issue: https://github.com/PyAutoLabs/autolens_workspace_test/issues/260 (issued 2026-08-22) +- session: https://claude.ai/code/session_01VEHLT33XpVcRt5YCJGLRMJ (web-github; no local worktree yet) +- status: workspace-dev +- worktree: ~/Code/PyAutoLabs-wt/jax-grad-local-vs-ci-assertions +- prompt: active/jax_grad_local_assertions_fail_but_pass_in_ci.md +- classification: workspace (single repo) โ€” routes to /start_workspace +- strategy: investigate-first. Bug Agent: severity=critical, scope=single-repo, + type=wrong-result, confidence=LOW. Reproduce and confirm root cause BEFORE patching. + Brain sizing disagreement: declared medium, derived large. +- control: imaging/jax_grad/lp.py is the discriminator โ€” the only script known to PASS in + CI and FAIL locally. Every A/B runs against it first (~41s). +- blocked-on-input: the failing local venv cannot be read from a cloud session. Capture + `pip freeze` + `np.show_config()` from the machine that reproduces the failure before it + drifts โ€” otherwise step 2 has no ground truth. Raised at plan time; not yet supplied. +- out-of-bounds: moving lp.py's evaluation point, adding skip_indices, or widening a + tolerance without a measured basis. All three mask the trap instead of removing it. +- repos: diff --git a/draft/bug/autolens_workspace_test/jax_grad_local_assertions_fail_but_pass_in_ci.md b/active/jax_grad_local_assertions_fail_but_pass_in_ci.md similarity index 100% rename from draft/bug/autolens_workspace_test/jax_grad_local_assertions_fail_but_pass_in_ci.md rename to active/jax_grad_local_assertions_fail_but_pass_in_ci.md diff --git a/dashboard.html b/dashboard.html index 00f7e868..bc6c302d 100644 --- a/dashboard.html +++ b/dashboard.html @@ -38,7 +38,7 @@

๐Ÿ“‹ PyAutoMind Dashboard

Every task the Mind is holding. Tap a task's ๐Ÿ“‹ and its /start_dev command is on your clipboard โ€” paste it into a Claude Code chat to route Claude straight to that task.

-

In flight 2 ยท Parked 3 ยท Planned 6 ยท Backlog 144 ยท markdown version

+

In flight 3 ยท Parked 3 ยท Planned 6 ยท Backlog 143 ยท markdown version

Start here

Highest priority (filed as high) โ€” showing 12 of 18

TRIAGE: needs manual review before routing โ€” medium ยท safe ยท high

@@ -59,6 +59,7 @@

In flight Issued โ€” each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first call โ€” issue #1501 โ€” HOLD โ€” do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

Hands hygiene: expired announcement dead code, unregistered modules โ€” issue #249 โ€” library-dev

+

jax_grad scripts fail assertions locally that PASS in CI โ€” issue #260 โ€” workspace-dev

Parked markdown version

3 task(s) @@ -77,7 +78,7 @@

Planned

latent-nan-guard-honest-run

Backlog markdown version

-

144 filed prompts, not started โ€” sorted most-pickable first (priority, then size). 23 of them belong to an epic and are listed only under Epics below.

+

143 filed prompts, not started โ€” sorted most-pickable first (priority, then size). 23 of them belong to an epic and are listed only under Epics below.

feature โ€” 28

Numba CPU likelihood phase 1: batched MGE convolution + operated-matrix โ€” autoarray ยท medium ยท supervised ยท high

@@ -110,7 +111,7 @@

Backlog

Teach repos_sync --write to stamp organ config surfaces โ€” pyautomind ยท hard ยท supervised ยท low

-bug โ€” 32 +bug โ€” 31

Fix release JAX runtime compatibility and likelihood parity โ€” health_fixes ยท too-large ยท supervised ยท high

Fix JIT quick-update visualization output regressions โ€” health_fixes ยท too-large ยท supervised ยท high

Fix release-profile numerical inversion failures โ€” health_fixes ยท too-large ยท supervised ยท high

@@ -118,7 +119,6 @@

Backlog

pixel_scales given as an int (or np.float64) is never widened โ€” autoarray ยท small ยท supervised ยท medium

jax 0.11 breaks beta/gamma message log_partition under jit ('tuple' object โ€” autofit ยท small ยท supervised ยท medium

Heart script_timing baselines are orphaned by path moves and filled โ€” pyautoheart ยท small ยท supervised ยท medium

-

jax_grad scripts fail assertions locally that PASS in CI โ€” autolens_workspace_test ยท medium ยท supervised ยท medium

Numba PSF gathers derive the y/x kernel shifts from the โ€” autoarray ยท low ยท supervised ยท medium

PyNUFFT dev extra is incompatible with current SciPy on Python โ€” autoarray ยท small ยท supervised ยท normal

LogGaussianPrior misreports its own support as (-inf, inf) โ€” autofit ยท small ยท supervised ยท normal

diff --git a/dashboard.md b/dashboard.md index d1644d8e..e9a1d7a9 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,10 +8,10 @@ Every task the Mind is holding, on one page: what is in flight, what is parked, | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 2 | +| [In flight](#in-flight) (`active/`) | 3 | | [Parked](#parked) (`parked.md`) | 3 | | [Planned](#planned) (`planned.md`) | 6 | -| [Backlog](#backlog) (`draft/`) | 144 | +| [Backlog](#backlog) (`draft/`) | 143 | ## Start here @@ -137,6 +137,14 @@ Issued โ€” each has an open GitHub issue and usually a branch. The full record f

+
๐Ÿ“‹ jax_grad scripts fail assertions locally that PASS in CI โ€” issue #260 โ€” workspace-dev + +``` +/start_dev active/jax_grad_local_assertions_fail_but_pass_in_ci.md +``` + +
+ ## Parked Started or scoped, not currently in flight โ€” resume by moving the row back to `active.md`. Full detail in [`parked.md`](parked.md). @@ -229,7 +237,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**144** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **23** of them belong to an epic and are listed only under [Epics](#epics) below. +**143** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **23** of them belong to an epic and are listed only under [Epics](#epics) below.
feature โ€” 28 @@ -461,7 +469,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-bug โ€” 32 +bug โ€” 31
๐Ÿ“‹ Fix release JAX runtime compatibility and likelihood parity โ€” health_fixes ยท too-large ยท supervised ยท high @@ -519,14 +527,6 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-
๐Ÿ“‹ jax_grad scripts fail assertions locally that PASS in CI โ€” autolens_workspace_test ยท medium ยท supervised ยท medium - -``` -/start_dev draft/bug/autolens_workspace_test/jax_grad_local_assertions_fail_but_pass_in_ci.md -``` - -
-
๐Ÿ“‹ Numba PSF gathers derive the y/x kernel shifts from the โ€” autoarray ยท low ยท supervised ยท medium ``` From ccf43b95af339ca9426dda46151df2a235acc065 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 22 Aug 2026 13:46:23 +0000 Subject: [PATCH 2/3] =?UTF-8?q?prompt:=20jax-grad-local-vs-ci-assertions?= =?UTF-8?q?=20(#260)=20=E2=80=94=20root=20cause=20found,=20numpy=20falsifi?= =?UTF-8?q?ed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- active.md | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/active.md b/active.md index c43d5c96..16fcc20a 100644 --- a/active.md +++ b/active.md @@ -37,9 +37,26 @@ Brain sizing disagreement: declared medium, derived large. - control: imaging/jax_grad/lp.py is the discriminator โ€” the only script known to PASS in CI and FAIL locally. Every A/B runs against it first (~41s). -- blocked-on-input: the failing local venv cannot be read from a cloud session. Capture - `pip freeze` + `np.show_config()` from the machine that reproduces the failure before it - drifts โ€” otherwise step 2 has no ground truth. Raised at plan time; not yet supplied. +- ROOT CAUSE FOUND 2026-08-22 (no laptop needed; findings on issue #260): + PyAutoArray `util/dataset_util.py:72` `should_simulate()` is existence-only and asymmetric โ€” + it force-regenerates under PYAUTO_SMALL_DATASETS=1 but never under full_datasets. `dataset/**` + is gitignored, so CI always simulates fresh and CANNOT hit this; locally the dir persists and + is never refreshed. Any prior smoke run (SMALL_DATASETS=1 is the default for every OTHER + script) rewrites the FITS at 15x15, and the next jax_grad run silently loads them under + full-resolution settings. All three failures reproduced exactly from a clean checkout + (pixelization eager/jit matches the report to 11 s.f.; regularization's tolerance vector + matches exactly). Full chain proven: fresh=PASS -> one SMALL run -> stale=FAIL -> rm -rf =PASS. +- numpy FALSIFIED: lp.py byte-identical across numpy 2.2.6 / 2.4.6 / 2.5.2 (and across + 1-core vs 4-core). The likelihood runs through JAX/XLA; numpy only does the FD bookkeeping. +- NO tolerance change is warranted โ€” all three asserts did their job on a genuinely invalid + dataset. assert_eager_jit_consistent's rtol=1e-10 is vindicated, not under-specified. +- RECLASSIFY: the fix belongs upstream in PyAutoArray, not this workspace โ€” every PyAuto + workspace using the auto-simulate pattern inherits the trap. Awaiting a human call between + (1) a regime marker and (2) symmetric regeneration; that call moves this to library-dev. +- follow-up (separate defect, not this bug): autolens_workspace_test + `.github/scripts/smoke_install.sh:9` `pip install "jax<0.7" "jaxlib<0.7"` downgrades jax to + 0.6.2 and conflicts with autonerves' jax>=0.7,<0.11; the run only lands on the intended + 0.10.2 because the next line's [optional] extras pull it back up. CI is right by accident. - out-of-bounds: moving lp.py's evaluation point, adding skip_indices, or widening a tolerance without a measured basis. All three mask the trap instead of removing it. - repos: From 2cc2a84f9fc1d42e2b84a333374563775506c9c8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 22 Aug 2026 14:20:50 +0000 Subject: [PATCH 3/3] =?UTF-8?q?prompt:=20jax-grad-local-vs-ci-assertions?= =?UTF-8?q?=20(#260)=20=E2=80=94=20fix=20on=20PyAutoArray=20branch,=202=20?= =?UTF-8?q?follow-ups=20filed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- active.md | 25 +++++++++++++++++++++---- dashboard.html | 2 +- dashboard.md | 2 +- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/active.md b/active.md index 16fcc20a..e4b15fd1 100644 --- a/active.md +++ b/active.md @@ -28,7 +28,8 @@ ## jax-grad-local-vs-ci-assertions - issue: https://github.com/PyAutoLabs/autolens_workspace_test/issues/260 (issued 2026-08-22) - session: https://claude.ai/code/session_01VEHLT33XpVcRt5YCJGLRMJ (web-github; no local worktree yet) -- status: workspace-dev +- status: library-dev (RECLASSIFIED from workspace-dev โ€” the fix is upstream in PyAutoArray; + autolens_workspace_test needs no change and no tolerance change) - worktree: ~/Code/PyAutoLabs-wt/jax-grad-local-vs-ci-assertions - prompt: active/jax_grad_local_assertions_fail_but_pass_in_ci.md - classification: workspace (single repo) โ€” routes to /start_workspace @@ -50,9 +51,24 @@ 1-core vs 4-core). The likelihood runs through JAX/XLA; numpy only does the FD bookkeeping. - NO tolerance change is warranted โ€” all three asserts did their job on a genuinely invalid dataset. assert_eager_jit_consistent's rtol=1e-10 is vindicated, not under-specified. -- RECLASSIFY: the fix belongs upstream in PyAutoArray, not this workspace โ€” every PyAuto - workspace using the auto-simulate pattern inherits the trap. Awaiting a human call between - (1) a regime marker and (2) symmetric regeneration; that call moves this to library-dev. +- FIX SHIPPED to a branch 2026-08-22: PyAutoArray `claude/jax-grad-assertions-ci-hontn3` @ 5ef6eea. + `should_simulate` now also regenerates on the small->full transition, inferring the regime from + data.fits's shape (the cap emits EXACTLY 16x16). Design went through an adversarial review that + corrected three things: no pixel-scale check (not in the FITS header), `==` not `<=` (the cap + cannot emit 12x12), and data.fits BY NAME not first-FITS-in-dir (PSFs are legitimately 11x11 and + dataset/cluster/test/psf.fits is 5760 B โ€” a glob would have deleted real data every run). + Verified: control-tested, 1168 tests pass, 12 new tests over all four regime transitions, and + end-to-end โ€” poison then full-datasets run now PASSES where it raised. Steady state 0.64 ms. + NO PR opened (not requested); branch is pushed and ready. +- SCOPE: imaging manifestation only. Point-source/weak-lensing (JSON, no FITS) and interferometer + (shape-invariant under the cap, so it fails SILENTLY) remain exposed; stated in the docstring. +- follow-ups filed (not absorbed): + - PyAutoNerves#153 โ€” stamp the regime at the single FITS writer funnel (fitsable.py:89); the only + discriminant that can catch the silent interferometer case. Kept out of the fix deliberately: it + changes a header card on every FITS the stack writes (round-trip tests, file-hash pins). + - PyAutoArray#470 โ€” separate live defect: the small-datasets branch rmtree's + dataset/point_source/simple, which is COMMITTED and allowlisted in .gitignore:13, replacing it + with capped-solver garbage. Recoverable via git checkout but violates the allowlist invariant. - follow-up (separate defect, not this bug): autolens_workspace_test `.github/scripts/smoke_install.sh:9` `pip install "jax<0.7" "jaxlib<0.7"` downgrades jax to 0.6.2 and conflicts with autonerves' jax>=0.7,<0.11; the run only lands on the intended @@ -60,3 +76,4 @@ - out-of-bounds: moving lp.py's evaluation point, adding skip_indices, or widening a tolerance without a measured basis. All three mask the trap instead of removing it. - repos: + - PyAutoArray: claude/jax-grad-assertions-ci-hontn3 diff --git a/dashboard.html b/dashboard.html index bc6c302d..243c1930 100644 --- a/dashboard.html +++ b/dashboard.html @@ -59,7 +59,7 @@

In flight Issued โ€” each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first call โ€” issue #1501 โ€” HOLD โ€” do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

Hands hygiene: expired announcement dead code, unregistered modules โ€” issue #249 โ€” library-dev

-

jax_grad scripts fail assertions locally that PASS in CI โ€” issue #260 โ€” workspace-dev

+

jax_grad scripts fail assertions locally that PASS in CI โ€” issue #260 โ€” library-dev (RECLASSIFIED from workspace-dev โ€” the fix is upstream in PyAutoArray;

Parked markdown version

3 task(s) diff --git a/dashboard.md b/dashboard.md index e9a1d7a9..86183dcd 100644 --- a/dashboard.md +++ b/dashboard.md @@ -137,7 +137,7 @@ Issued โ€” each has an open GitHub issue and usually a branch. The full record f
-
๐Ÿ“‹ jax_grad scripts fail assertions locally that PASS in CI โ€” issue #260 โ€” workspace-dev +
๐Ÿ“‹ jax_grad scripts fail assertions locally that PASS in CI โ€” issue #260 โ€” library-dev (RECLASSIFIED from workspace-dev โ€” the fix is upstream in PyAutoArray; ``` /start_dev active/jax_grad_local_assertions_fail_but_pass_in_ci.md