Skip to content

lifecycle: validate the registry files in check, reconcile planned.md #158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jammy2211 merged 2 commits into from
Aug 8, 2026
Merged

lifecycle: validate the registry files in check, reconcile planned.md #158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
506c2b4
prompt: file registry-integrity-check (maintenance/pyautomind)
claude Aug 8, 2026
ebbc3c1
lifecycle: validate the registry files in `check`, reconcile planned.md
claude Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions draft/maintenance/pyautomind/registry_integrity_check.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Teach `lifecycle.py check` to validate the registry files

Type: maintenance
Target: PyAutoMind
Repos:
- PyAutoMind
Difficulty: small
Autonomy: supervised
Priority: high
Status: formalised

`scripts/lifecycle.py check` reports **OK** on a `planned.md` in which 8 of 12
entries are wrong. The registry is the first thing any task-selection pass
reads, so the rot is not cosmetic — it actively costs sessions. This task closes
that gap: extend `cmd_check` to validate the registry files, then fix what it
finds.

## Why now (measured, 2026-08-08)

A task-selection pass picked the two highest-leverage `planned.md` entries and
spent most of a session discovering **both were already fixed on main**:

- **`notebook-kernel-cwd-auto-simulate`** (PyAutoHands#204) — FIXED.
`autohands/build_util.py:302` no longer shells out to
`jupyter nbconvert --execute`; it runs `autohands/run_notebook.py`, which sets
`resources['metadata']['path']` through the Python API so the kernel starts at
the repo root. The in-code comment describes this exact bug. Verified on
PyAutoHands main at `a5bac76`.
- **`auto-simulate-guard-wrong-simulator-target`** (autolens_workspace#359) —
FIXED. All 246 `should_simulate` guards in autolens_workspace were audited
against the 54 simulators' declared `dataset_type`/`dataset_name` outputs:
**236 resolvable, 0 mismatches.** The 10 unresolved are benign — a simulator
matching its own glob, `guides/hpc/example_cpu_and_gpu.py` (path built from an
`hpc_dataset_path` variable), and the `guides/results/` scripts whose first
subprocess target is the `_quick_fit.py` helper rather than a simulator.

Neither entry had a prompt file, so neither was reachable through the normal
`$start-dev` path — the staleness was only discoverable by reading upstream
code.

## The drift, classified

`planned.md` holds 12 entries. Resolving each `prompt:` path through the
fallback chain that `AGENTS.md` documents (`draft/<rel>`, bare `<rel>`,
`active/<name>`):

| Class | N | Entries |
|---|---|---|
| Prompt file never existed in git history | 2 | `notebook-kernel-cwd-auto-simulate`, `auto-simulate-guard-wrong-simulator-target` |
| Prompt file truly missing | 3 | `heart-ci-linkage`, `heart-release-validation`, `heart-release-profile-wheel-integration` |
| Legacy `PyAutoMind/<type>/<target>/` path, resolves only via fallback | 3 | `samples-parameter-paths`, `nfw-truncated-potential-accuracy`, `piemass-potential` |
| State contradiction | 1 | `build-testpypi-rehearsal-mode` — `status: planned`, but its prompt is in `active/` |

Only 4 of 13 entries had an exactly-correct prompt path.

**Six of the thirteen were work that had already shipped.** Chasing each of the
five missing-prompt entries to its upstream repo found the capability live on
main in every case:

| Entry | Milestone | Shipped as |
|---|---|---|
| `notebook-kernel-cwd-auto-simulate` | — | PyAutoHands `build_util.py:302` → `run_notebook.py`, setting `resources['metadata']['path']` |
| `auto-simulate-guard-wrong-simulator-target` | — | autolens_workspace: 236 resolvable guards, 0 mismatches |
| `build-testpypi-rehearsal-mode` | M1 | PyAutoHands `release.yml` — `rehearsal` dispatch input, `resolve_mode` job, downstream jobs gated `if: rehearsal != 'true'`, dev-segment version output. Entry targeted "PyAutoBuild", which is now PyAutoHands |
| `heart-ci-linkage` | M0 | PyAutoHeart `heart/checks/ci_status.{sh,py}` + `tests/test_ci_status.py`; the script's own comment says it "replaces the old `gh run list --limit 1`" — verbatim the defect the entry described |
| `heart-release-validation` | M2 | PyAutoHeart `pyauto-heart validate --ingest` → `heart/validate.py`, `validation_report.json`, `.github/workflows/release-integrate.yml` |
| `heart-release-profile-wheel-integration` | M3 | PyAutoHeart `heart/validate.py` carries the named `release` profile and gates fidelity on `profile == release`; TestPyPI wheel install in `heart/checks/verify_install.sh` |

The whole M0–M3 release-validation milestone chain shipped without a single
registry entry being retired. That is the cost this check exists to prevent.

## Why `check` misses all of it

`cmd_check` (`scripts/lifecycle.py:376`) validates exactly two conditions:

1. an `active.md` slug that also has a `complete/` record;
2. a filename present in both `active/` and `complete/`.

It never opens `planned.md` or `parked.md`, and never resolves a `prompt:` path
in any registry file. `scripts/lifecycle.py` also has **no test** —
`tests/` holds only `test_repos_sync_hygiene_coverage.py`,
`test_spawn_privacy.py`, `test_spawn_template_contract.py`.

## Scope

Two legs, one PR — the data fix is required for the new check to land green.

**Leg 1 — the check.** In `cmd_check`, parse `## <slug>` entries and their
`- key: value` fields from `active.md`, `planned.md` and `parked.md`, and add:

- **Prompt resolution** — every `prompt:` path resolves through the documented
fallback chain, else `DRIFT`. Report the resolved location when it differs
from the literal path, so legacy paths are visible rather than silently
absorbed.
- **State contradiction** — a `planned.md`/`parked.md` entry whose prompt lives
in `active/` (it is issued) or under `complete/` (it shipped) is drift. This
is the check that would have caught `build-testpypi-rehearsal-mode`.
- **Slug uniqueness** — a slug must not appear in two registries at once.

Match the existing `problems` list + `lifecycle check: DRIFT` output shape; do
not change the exit-code contract (`0` OK, `1` drift).

**Leg 2 — reconcile the 8 entries.**

- Rewrite the 3 legacy paths to their exact `draft/...` form.
- Remove the 6 verified-shipped entries, citing the evidence table above. The
M0–M3 chain needed no reconstruction from its `summary:` blocks after all —
every milestone was already live upstream.
- Removal is the right disposal, not a fabricated `complete/` record: these
shipped under other tasks' PRs, and inventing dated records with merge
evidence nobody verified would put a worse lie in a more trusted place.

**Leg 3 — `tests/test_lifecycle_check.py`.** Drive the real `cmd_check` against
fixture registry trees (tmp_path with `draft/`, `active/`, `complete/` and
synthetic registry files) — one case per new condition, plus a clean tree
asserting `OK`. Follow `test_repos_sync_hygiene_coverage.py` for style.

## Explicitly out of scope

- **Online issue cross-checking.** A local check catches the two shipped entries
only because their prompt files are missing; had the files existed, nothing
offline would flag "this shipped upstream". Catching that class needs each
entry's `issue:` cross-checked against GitHub, which makes `check`
non-hermetic and credentialed. Decided 2026-08-08 to keep `check` offline;
filed as a follow-up idea instead.
- **Closing the upstream issues.** PyAutoHands#204 and autolens_workspace#359
are still open on trackers outside this task's repo. Flagged here for a later
`/issue_cleanup` run; this PR touches PyAutoMind only.
- **The orphaned `active/` prompt.** Removing `build-testpypi-rehearsal-mode`
from planned.md leaves `active/release_yml_testpypi_rehearsal_mode.md` sitting
in `active/` with no registry entry — shipped work whose prompt was never
advanced to `complete/`. `check` does not look for prompts that no registry
claims, so this is invisible to it. Two follow-ups, both deliberately not
taken here: give that prompt a proper `complete/` record via the ship path,
and add an orphan-prompt check (every `active/*.md` is claimed by an
`active.md` or `parked.md` entry) — the mirror of the checks added here.
- `condemned.md`, `queue.md` and `ideas.md` — different schemas, no `prompt:`
field. Not covered.

## Acceptance

- `python3 scripts/lifecycle.py check` exits `0` on the reconciled tree, and
exits `1` with a named problem for each of the three new conditions when
seeded with a fixture that violates it.
- `pytest tests/test_lifecycle_check.py` green.
- No entry left in `planned.md` whose prompt path does not resolve exactly.
122 changes: 3 additions & 119 deletions planned.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
- autolens_assistant

## samples-parameter-paths
- prompt: PyAutoMind/bug/health_fixes/samples_parameter_paths.md
- prompt: draft/bug/health_fixes/samples_parameter_paths.md
- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1327 (open, parked)
- status: parked
- filed: 2026-07-08
Expand All @@ -47,88 +47,6 @@
Full trail: PyAutoFit#1327 comments.
- affected-repos:

## heart-ci-linkage
- prompt: PyAutoMind/feature/pyautoheart/ci_linkage.md
- status: planned
- filed: 2026-06-30
- classification: organism (PyAutoHeart CI signal + registry)
- suggested-branch: feature/heart-ci-linkage
- milestone: M0 (foundational — release-validation gate builds on a trustworthy CI signal)
- summary: |
Final-review finding: Heart's CI signal is too coarse/narrow to gate a
release. ci_status reads `gh run list --limit 1` (newest run, any workflow,
any branch) but workspaces gate on 3 workflows × 2 Pythons; readiness gates
only the 5 libraries' CI (workspace CI observed but never gated); and the
signal should come from the Actions server (mobile-reachable via MCP) with
report.json as enrichment, not a hard dependency. Plus repos.yaml is stale
(PyAutoPrompt→Mind, PyAutoPaper→Memory; organism repos unpolled). Rework
ci_status to per-required-workflow-on-main, gate workspace CI, make the run
conclusion the primary test_run signal, refresh the registry.
- affected-repos:
- PyAutoHeart

## heart-release-validation
- prompt: PyAutoMind/feature/pyautoheart/release_validation.md
- status: planned
- filed: 2026-06-30
- classification: organism (PyAutoHeart deep validation + report + readiness gate)
- suggested-branch: feature/heart-release-validation
- milestone: M2 (depends on M1 = build-testpypi-rehearsal-mode)
- boundary: |
Heart never mutates a repo and never triggers a build. The Brain Release
Agent dispatches the rehearsal + validation workflows and awaits them; Heart's
`validate` is ingest-and-judge only; the Health Agent (read-only) reports the
verdict. Heart and Build never call each other.

## heart-release-profile-wheel-integration
- prompt: PyAutoMind/feature/pyautoheart/release_profile_and_wheel_integration.md
- status: planned
- filed: 2026-06-30
- classification: organism (validation fidelity — wheels + release env profile)
- suggested-branch: feature/heart-release-profile-wheel-integration
- milestone: M3 (depends on M1 + M2; closes Gaps A & B)
- summary: |
Make the validation run install the TestPyPI wheels (no source on PYTHONPATH,
scripts run from inside the workspace checkout so autoconf resolves workspace
config/) and run at release fidelity via a named `release` env profile
(user workspaces TEST_MODE=1+small+fast; *_test TEST_MODE=0, full-res),
mirroring release.yml's tier split. Env-var profile only — does not touch
config/general.yaml test:/version: toggles.
- affected-repos:
- PyAutoHeart
- PyAutoBuild
- autolens_workspace_test / autogalaxy_workspace_test / autofit_workspace_test
- autolens_workspace / autogalaxy_workspace / autofit_workspace
- summary: |
New third Heart tier: a release-grade `pyauto-heart validate` that composes
a TestPyPI build rehearsal + unit tests + the full workspace/workspace_test
integration surface, ingests the run reports into a tracked
`validation_report.json`, and hard-gates `readiness` GREEN on a fresh pass
for the current source SHAs. Driven from mobile via the Brain health agent
(GitHub dispatch/poll via MCP; Heart stays credential-free). Bakes in two
verified gaps the current `workspace-validation.yml` has: it tests source
not wheels (PYTHONPATH-shadow), and it runs the smoke profile
(PYAUTO_TEST_MODE=2 + PYAUTO_SMALL_DATASETS=1) not a release-fidelity profile.
- affected-repos:
- PyAutoHeart
- PyAutoBrain
- PyAutoBuild

## build-testpypi-rehearsal-mode
- prompt: PyAutoMind/feature/pyautobuild/release_yml_testpypi_rehearsal_mode.md
- status: planned
- filed: 2026-06-30
- classification: organism (PyAutoBuild executor capability)
- suggested-branch: feature/build-testpypi-rehearsal-mode
- milestone: M1 (prerequisite for M2 = heart-release-validation)
- summary: |
Add a TestPyPI-only "rehearsal" dispatch mode to release.yml: build current
source, publish to TestPyPI, emit the version string, and STOP before
PyPI/tag/notebook steps — so Heart can install and validate the actual wheels
before any release. Small, isolated, highest-value first piece.
- affected-repos:
- PyAutoBuild

## jax-point-source-point-smoke-sentinel
- prompt: draft/bug/autolens/jax_point_source_point_smoke_sentinel.md
- status: planned
Expand All @@ -155,7 +73,7 @@
source_plane.py in the same dir — they share the seed dataset.

## nfw-truncated-potential-accuracy
- prompt: PyAutoMind/bug/autogalaxy/nfw_truncated_potential_accuracy.md
- prompt: draft/bug/autogalaxy/nfw_truncated_potential_accuracy.md
- status: planned
- filed: 2026-06-05
- classification: library (accuracy bug)
Expand All @@ -172,7 +90,7 @@


## piemass-potential
- prompt: PyAutoMind/feature/autogalaxy/piemass_potential.md
- prompt: draft/feature/autogalaxy/piemass_potential.md
- status: planned
- filed: 2026-06-05
- classification: library (missing feature)
Expand All @@ -196,37 +114,3 @@
- affected-repos:
- autolens_workspace_test
- note: latent/latent_nan_robustness.py PASSES but VACUOUSLY under the smoke profile — TEST_MODE=2 yields only 4 bypass samples, and DISABLE_JAX=1 silently flips its deliberate AnalysisImaging(use_jax=True) to False (PyAutoLens analysis/analysis/dataset.py:89), so the JAX column-masking branch the guard exists to catch is never taken. MultiStartAdam/BlackJAXNUTS precedent. Work = (1) config/build/env_vars.yaml override for `latent/latent_nan_robustness` with unset: [PYAUTO_TEST_MODE, PYAUTO_DISABLE_JAX]; (2) trim the script under the 300s cap. MEASURED: honest run = 412s; PYAUTO_TEST_MODE=1 does NOT help (455s) — Nautilus is NOT the bottleneck (~136s post-fit results update + ~56s latent compute on 100 samples), so the lever is sample count. Script is in the curated smoke_tests.txt, which DOES read env_vars.yaml, so this lands in the per-PR gate. Adjacent to the blocker's own follow-up ("re-time the SLOW siblings"). NOT bugs, verified passing from clean output, no change needed: imaging/model_fit.py and latent/latent_variables_smoke.py.

## notebook-kernel-cwd-auto-simulate
- issue: https://github.com/PyAutoLabs/PyAutoHands/issues/204
- prompt: draft/bug/workspaces/notebook_kernel_cwd_breaks_auto_simulate.md
- status: planned
- filed: 2026-07-27
- classification: library+workspace (PyAutoHands runner vs per-script path resolution — fix option is a HUMAN CALL, see prompt)
- suggested-branch: feature/notebook-kernel-cwd-auto-simulate
- summary: |
jupyter nbconvert runs the kernel in the NOTEBOOK'S OWN directory, but the
auto-simulate guard shells out to a workspace-root-relative simulator path.
So every notebook that auto-simulates dies with exit status 2 ("can't open
file"). Proven empirically: launcher cwd .../cwdtest, kernel cwd
.../cwdtest/notebooks/sub. Accounts for ~20 of the 29 failing jobs in
workspace-validation run 30242158468. Scripts are unaffected (they do run
from the root), which is why run_scripts mostly passes and run_notebooks
mostly fails.

## auto-simulate-guard-wrong-simulator-target
- issue: https://github.com/PyAutoLabs/autolens_workspace/issues/359
- prompt: draft/bug/autolens_workspace/auto_simulate_guard_wrong_simulator_target.md
- status: planned
- filed: 2026-07-27
- classification: workspace
- suggested-branch: feature/auto-simulate-guard-wrong-simulator-target
- summary: |
likelihood_function.py scripts load dataset/imaging/simple but their
auto-simulate guard runs no_lens_light/simulator.py, which writes
simple__no_lens_light — so the guard fires and the load still fails. Guard
target dates to 1f39244f; surfaced now because #354 swapped the raw
path-exists check for should_simulate. FIRST establish whether the target
was always wrong or should_simulate changed the predicate (the latter would
be a much wider bug), THEN sweep all 116 migrated guards for the same
mismatch.
Loading
Loading