From 66557b97b196328d81e7fbf567713f1796921c9e Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Mon, 27 Jul 2026 19:08:33 +0100 Subject: [PATCH] fix: close the spawn spec gaps taking spawn_drift red MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three separate defects, all surfaced by the nightly Spawn Drift job: 1. issued/remove_pulse_compat.md — the sole survivor of the retired flat issued/ pile. The lifecycle split (#71/#72) repointed spawn.py's ('issued/*', 'DROP') rule to 'active/*', leaving this one file with no matching rule -> UNMATCHED -> exit 1. The work it describes is done (PyAutoPulse gone, no pulse refs in PyAutoHeart) and already recorded at complete/2026/07/remove-pulse-compat.md, so it is a pure orphan. Deleted. 2. AI_POLICY.md — added org-wide today, with no spec rule, so it became the next UNMATCHED the moment (1) was cleared. It names the owning org twice, so it takes KEEP_SUB (PyAutoLabs -> YOURORG), not a verbatim KEEP. 3. CONTRIBUTING.md — was KEEP verbatim. Today's centralisation rewrote it to 'Contributing to PyAutoLabs', so a republish would have stamped PyAutoLabs branding into a fresh-slate template spawned for another org. Same defect class as (2); moved to KEEP_SUB in both rule sets. Also repoints planned.md's jax-point-source-point-smoke-sentinel entry, which pointed at issued/jax_point_source_point_smoke_sentinel.md — a 130-line diagnostic prompt dropped as 'legacy' by the lifecycle migration while the task stayed planned and unfixed. Restored verbatim to draft/bug/autolens/. spawn.py and spawn_spec.md are changed together (the spec is the source of record and the code mirrors it). After this, --check reports 'unmatched: none' + 'canary scan: clean' for both templates; the remaining drift is content staleness that a republish clears. Co-Authored-By: Claude Opus 5 --- ...i_squared_paper_variants_phase_1_design.md | 0 docs/pyautobrain/spawn_spec.md | 6 +- .../jax_point_source_point_smoke_sentinel.md | 149 ++++++++++++++++++ issued/remove_pulse_compat.md | 11 -- planned.md | 2 +- scripts/spawn.py | 12 +- 6 files changed, 164 insertions(+), 16 deletions(-) rename {draft/feature/autolens => active}/point_source_chi_squared_paper_variants_phase_1_design.md (100%) create mode 100644 draft/bug/autolens/jax_point_source_point_smoke_sentinel.md delete mode 100644 issued/remove_pulse_compat.md diff --git a/draft/feature/autolens/point_source_chi_squared_paper_variants_phase_1_design.md b/active/point_source_chi_squared_paper_variants_phase_1_design.md similarity index 100% rename from draft/feature/autolens/point_source_chi_squared_paper_variants_phase_1_design.md rename to active/point_source_chi_squared_paper_variants_phase_1_design.md diff --git a/docs/pyautobrain/spawn_spec.md b/docs/pyautobrain/spawn_spec.md index 1353cc2e..dd29a6a3 100644 --- a/docs/pyautobrain/spawn_spec.md +++ b/docs/pyautobrain/spawn_spec.md @@ -32,8 +32,9 @@ deliberately, never silently shipped into a template. | # | Pattern | Action | |---|---------|--------| | 1 | `scripts/**` | KEEP verbatim (generic tooling: repos_sync, prompt_sync, status, spawn itself) | -| 2 | `REFERENCE.md`, `AGENTS.md`, `CLAUDE.md`, `LICENSE`, `CONTRIBUTING.md`, `ROUTING.md`, `.gitignore` | KEEP verbatim | +| 2 | `REFERENCE.md`, `AGENTS.md`, `CLAUDE.md`, `LICENSE`, `ROUTING.md`, `.gitignore` | KEEP verbatim | | 3 | `README.md` | KEEP verbatim (already generic post-Phase-1) | +| 3b | `AI_POLICY.md`, `CONTRIBUTING.md` | KEEP with owner substitution — both are org-wide pointer docs: generic prose that names the owning org and links the canonical copy inside that org's `PyAutoScientist`. They take the same `PyAutoLabs` → `YOURORG` substitution as rule 9 rather than a verbatim KEEP; verbatim would stamp a literal "Contributing to PyAutoLabs" into a template spawned for another org | | 4 | `repos.yaml` | SUBSTITUTE → the **template body map**: the five organ rows kept with `github:` owner replaced by `YOURORG`; all live satellite rows replaced by the PyAutoProject family rows (`PyAutoProject` category `library`, `autoproject_workspace` category `workspace`, `autoproject_workspace_test` category `workspace_test`) + a commented-out `autoproject_assistant` row ("uncomment when the clone agent seeds it") | | 5 | `active.md`, `planned.md`, `parked.md`, `condemned.md`, `ideas.md`, `queue.md`, `autonomy_log.md` | EMPTY → header line + schema pointer comment only (e.g. `# Active Tasks` + ``); autonomy_log keeps its schema header rows | | 6 | `draft/**` (the work-type dirs `feature/ bug/ refactor/ docs/ test/ release/ maintenance/ research/ experiment/ triage/` now live under `draft/`) | SKELETON → keep a single `draft/.gitkeep`; drop all draft prompts and their work-type/target subdirs (a fresh Mind starts with an empty `draft/`; intake recreates the work-type subdirs on demand) | @@ -47,7 +48,8 @@ deliberately, never silently shipped into a template. | # | Pattern | Action | |---|---------|--------| -| 1 | `bibliography/*.py`, `bibliography/README.md`, `scripts/`, `tests/`, `Makefile`, `LICENSE`, `CONTRIBUTING.md`, `AGENTS.md`, `CLAUDE.md`, `.gitignore` | KEEP verbatim (tooling + schema + policy — scripts/tests are the citation-validation tooling the Makefile drives) | +| 1 | `bibliography/*.py`, `bibliography/README.md`, `scripts/`, `tests/`, `Makefile`, `LICENSE`, `AGENTS.md`, `CLAUDE.md`, `.gitignore` | KEEP verbatim (tooling + schema + policy — scripts/tests are the citation-validation tooling the Makefile drives) | +| 1b | `AI_POLICY.md`, `CONTRIBUTING.md` | KEEP with owner substitution — same org-wide pointer docs as the Mind table's rule 3b | | 2 | `bibliography/*.bib` and any bibliography data files | EMPTY → file kept with header comment ("populated by your literature") | | 3 | `wiki/CLAUDE.md` (the shared schema) | KEEP verbatim — since the wiki/ restructure (PyAutoMemory#24) the schema is a single domain-neutral file at `wiki/CLAUDE.md`, kept canary-clean at the source (its examples carry no instance tokens), so spawn no longer maintains a duplicate schema asset | | 4 | `wiki/*` (all live sub-wikis, all pages) | DROP; generate instead ONE `wiki/example/` containing a slim scope-only `CLAUDE.md` (the schema is inherited from rule 3, not copied), an `index.md` skeleton listing zero sources, and one `sources/EXAMPLE_stub.md` demonstrating the stub format (hand-written once, stored inside spawn as a heredoc/template asset — not copied from live content) | diff --git a/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md b/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md new file mode 100644 index 00000000..4f0a1e71 --- /dev/null +++ b/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md @@ -0,0 +1,149 @@ +# JAX point-source smoke sentinel: point.py returns -1e99 instead of -83.38 + +Type: bug +Target: autolens +Repos: +- @PyAutoLens +- @PyAutoGalaxy +- @PyAutoArray +- @autolens_workspace_test +Difficulty: medium +Autonomy: supervised +Priority: normal +Status: draft + +> Restored 2026-07-27. This prompt file was dropped from the flat `issued/` pile +> during the prompt-lifecycle migration (PR #71/#72) as "legacy", but the task was +> never done — its `planned.md` entry stayed live and pointed at the deleted path. +> Body below is the original 2026-05-21 content, verbatim. + +Smoke regression surfaced during the `fast-viz-zero-contour-perf` task +(workspace PR https://github.com/PyAutoLabs/autolens_workspace_test/pull/111). + +`autolens_workspace_test/scripts/jax_likelihood_functions/point_source/point.py` +fails its hardcoded `assert_allclose(np.array(result), -83.38049778, rtol=1e-4)` +check on canonical `main` of all three libraries — `fitness._vmap(parameters)` +returns the `-1e99` sentinel (the chi-squared "non-finite likelihood" reject +value used by `FitPositionsImagePairAll`) instead of the expected -83.38. + +Reproduces identically on canonical main with no workspace-PR changes applied, +so this is a library-side regression, not anything from the current task. + +## Observed symptom + +| Script | Expected (hardcoded) | Actual (vmap) | Notes | +|---|---|---|---| +| `jax_likelihood_functions/point_source/point.py` (L234-239) | `-83.38049778` | `-1e99` (sentinel) | Pre-existing on `main` 2026-05-21 | + +The -1e99 value is the standard `FitPositionsImagePairAll` sentinel for +"position-pairing rejected this model" — the JAX vmap path is treating every +sampled parameter set as a reject, which means either every position is +failing to be paired or every solver call is returning `inf`/`nan` rows. + +## Last known good + +The literal `-83.38049778` was set on **2026-05-08** in +`autolens_workspace_test@362cfa8` ("rebaseline JAX point-source likelihood +literals after noise-scale change"). At that commit the smoke passed +end-to-end against the libraries-of-the-day. Something in PyAutoLens / +PyAutoGalaxy / PyAutoArray between 2026-05-08 and 2026-05-21 has broken the +JAX vmap path through `FitPositionsImagePairAll` for the seed point-source +dataset committed under `dataset/point_source/simple/`. + +## Relationship to PyAutoLens#514 + +`PyAutoPrompt/issued/jit_regression_constant_drift.md` already tracks a +drift in `autolens_workspace_developer/jax_profiling/jit/point_source/image_plane.py` +filed as https://github.com/PyAutoLabs/PyAutoLens/issues/514. That ticket +exercises the **same JAX code path** (`PointSolver` + `FitPositionsImagePairAll`) +but a different symptom (constant drifted 0.07 → -362, not a -1e99 sentinel) +on a different file (the profiling script in `_developer`, not the smoke +script in `_test`). + +Two hypotheses worth holding in tension during triage: + +1. **Same root cause, different manifestation.** A change in `PointSolver` + triangle refinement or `FitPositionsImagePairAll` position-pairing could + produce a drifted-but-finite value on `image_plane.py`'s seeded inputs + and a clean-reject sentinel on `point.py`'s seeded inputs, if the latter + is closer to a pairing-threshold edge case. If so this prompt and #514 + resolve together. +2. **Two independent regressions.** The -1e99 sentinel is qualitatively + different from a sign-flip drift — it could be a separate JAX-path bug + (e.g. a tracer-typed quantity being compared with a NumPy threshold, + forcing the whole batch into the reject branch). If so this needs its + own library-side fix. + +Either way the symptom is severe enough to warrant a separate triage line. + +## Task + +1. **Reproduce** on canonical main of all three libraries: + ```bash + cd ~/Code/PyAutoLabs/autolens_workspace_test + git checkout main && git pull --ff-only + git -C ../PyAutoLens checkout main && git -C ../PyAutoLens pull --ff-only + git -C ../PyAutoGalaxy checkout main && git -C ../PyAutoGalaxy pull --ff-only + git -C ../PyAutoArray checkout main && git -C ../PyAutoArray pull --ff-only + python scripts/jax_likelihood_functions/point_source/point.py + ``` + Confirm the AssertionError and capture the actual vmap output. + +2. **Bisect across the three libraries** since 2026-05-08: + ```bash + git -C ../PyAutoLens log --oneline --since=2026-05-08 -- autolens/point/ + git -C ../PyAutoLens log --oneline --since=2026-05-08 -- autolens/analysis/ + git -C ../PyAutoGalaxy log --oneline --since=2026-05-08 + git -C ../PyAutoArray log --oneline --since=2026-05-08 + ``` + Prime suspects (matching the #514 candidate-list): + - `PointSolver.solve` and the triangle-refinement loop + - `FitPositionsImagePairAll` chi-squared assembly + - Anything touching `positions_noise_map` application or `xp` + propagation through the point-source stack + +3. **Diagnose where -1e99 enters.** Drop a `jax.debug.print` (or eager + NumPy-path run) inside `FitPositionsImagePairAll.log_likelihood_function` + to see whether: + - every pair distance is `inf`/`nan` (solver returning bad positions); + - the pair-permutation `argmin` is selecting the wrong index; + - the χ² → log-likelihood reduction is hitting an inf/nan that triggers + the sentinel branch. + +4. **Decide per finding**: + - If the new behaviour is **a library bug** → file against the responsible + PyAuto* repo, cross-reference both this prompt and #514, leave the smoke + literal as-is (the failing assertion is load-bearing while the bug is + open). + - If the new behaviour is **a deliberate change** (noise-map convention, + pairing semantics) → rebaseline the literal in `point.py`. Also re-run + `image_plane.py` and `source_plane.py` from `jax_likelihood_functions/point_source/` + since they share the seed dataset and may need matching updates. + +5. **Spot-check the sibling JAX smoke scripts** in + `autolens_workspace_test/scripts/jax_likelihood_functions/point_source/` + (`image_plane.py`, `source_plane.py`) — they share the seed dataset and + may exhibit the same regression, in which case the fix is unified. + +## Out of scope + +- Re-running the full `autolens_workspace_test` smoke suite — only the + three point-source JAX scripts need verifying for this triage. +- The constant drift in `autolens_workspace_developer/jax_profiling/jit/point_source/image_plane.py` + tracked by #514 — share findings, but don't expand this prompt into that + one. Update #514 with whatever the bisect surfaces. +- Regenerating the seed dataset under `dataset/point_source/simple/` — the + files are committed, the literals were set against them on 2026-05-08, so + the dataset is *not* the moving variable here. Do not run + `PYAUTO_SMALL_DATASETS=1` during triage — it would delete the committed + seed data. + +## Pre-existing context + +- Last known-good rebaseline: `autolens_workspace_test@362cfa8` (2026-05-08) +- Related JIT profiling drift: `PyAutoPrompt/issued/jit_regression_constant_drift.md` → PyAutoLens#514 +- Smoke philosophy: workspace `scripts/CLAUDE.md` documents that the + hardcoded literals in `jax_likelihood_functions/` are intentional + absolute-regression markers — don't relational-rewrite them away. +- Surfaced by: workspace PR https://github.com/PyAutoLabs/autolens_workspace_test/pull/111 + (fast-viz-zero-contour-perf), noted in that PR body. diff --git a/issued/remove_pulse_compat.md b/issued/remove_pulse_compat.md deleted file mode 100644 index 2c2fe3af..00000000 --- a/issued/remove_pulse_compat.md +++ /dev/null @@ -1,11 +0,0 @@ -# Remove PyAutoPulse Compatibility Names - -## Original Request - -We renamed PyAutoPulse to PyAutoHeart, but the folder still has PyAutoPulse and there is PyautoHeart/autopulse, is it safe to remove these pulse things and if so do it - -## Notes - -- Remove the old top-level `PyAutoPulse` symlink if it is only an alias to `PyAutoHeart`. -- Remove tracked `pulse` / `pyautopulse` compatibility wrappers from `PyAutoHeart`. -- Update packaging and tests to use canonical `heart` / `pyautoheart` paths only. diff --git a/planned.md b/planned.md index 1e605dd5..24df15c4 100644 --- a/planned.md +++ b/planned.md @@ -133,7 +133,7 @@ - PyAutoBuild ## jax-point-source-point-smoke-sentinel -- prompt: PyAutoMind/issued/jax_point_source_point_smoke_sentinel.md +- prompt: draft/bug/autolens/jax_point_source_point_smoke_sentinel.md - status: planned - filed: 2026-05-21 - classification: library (triage; routing TBD by bisect) diff --git a/scripts/spawn.py b/scripts/spawn.py index 0c203d6a..2952968f 100644 --- a/scripts/spawn.py +++ b/scripts/spawn.py @@ -43,9 +43,15 @@ MIND_RULES = [ ("scripts/*", "KEEP"), ("REFERENCE.md", "KEEP"), ("AGENTS.md", "KEEP"), ("CLAUDE.md", "KEEP"), - ("LICENSE", "KEEP"), ("CONTRIBUTING.md", "KEEP"), ("ROUTING.md", "KEEP"), + ("LICENSE", "KEEP"), ("ROUTING.md", "KEEP"), (".gitignore", "KEEP"), ("README.md", "KEEP"), + # Org-wide pointer docs. Generic prose, but each names the owning org and + # links the canonical copy in that org's PyAutoScientist — so they take the + # same owner substitution .github/** does rather than a verbatim KEEP. + # Verbatim would stamp "Contributing to PyAutoLabs" into a fresh-slate + # template spawned for somebody else's org. + ("AI_POLICY.md", "KEEP_SUB"), ("CONTRIBUTING.md", "KEEP_SUB"), ("repos.yaml", "SPECIAL:body_map"), ("active.md", "EMPTY"), ("planned.md", "EMPTY"), ("parked.md", "EMPTY"), ("condemned.md", "EMPTY"), ("ideas.md", "EMPTY"), ("queue.md", "EMPTY"), @@ -72,8 +78,10 @@ MEMORY_RULES = [ ("bibliography/*.py", "KEEP"), ("bibliography/README.md", "KEEP"), ("scripts/*", "KEEP"), ("tests/*", "KEEP"), - ("Makefile", "KEEP"), ("LICENSE", "KEEP"), ("CONTRIBUTING.md", "KEEP"), + ("Makefile", "KEEP"), ("LICENSE", "KEEP"), ("AGENTS.md", "KEEP"), ("CLAUDE.md", "KEEP"), (".gitignore", "KEEP"), + # Same org-wide pointer docs as MIND_RULES — owner substitution. + ("AI_POLICY.md", "KEEP_SUB"), ("CONTRIBUTING.md", "KEEP_SUB"), ("bibliography/*", "EMPTY"), (".github/*", "KEEP_SUB"), # The shared wiki schema is template content; the sub-wikis are instance