From 6d1979b967de36959c6fc83c4fc31ae5e13afc81 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 12:25:04 +0000 Subject: [PATCH] mind: cover the gh fetcher, record that the point-smoke prompt is overtaken MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two loose ends from the first live run of `lifecycle.py issues`. **`_gh_issue_states` had never executed.** Every online-leg test injects `fetch`, which is what keeps them hermetic — but it also meant the real shim, the one that runs on a machine with gh, had zero coverage, and its first real invocation would have been against production. Six tests drive it with `subprocess.run` stubbed: the `gh api … --jq .state` argv, the strip of gh's trailing newline, FileNotFoundError -> GhUnavailable (the exception, not a state string, so "could not run" never reads as "no findings"), a non-zero exit -> `unreadable: `, the `or ["error"]` fallback on empty stderr, and non-URL entries costing no subprocess call. Each was confirmed to fail under a matching mutation of the shim. **The point-source smoke prompt is overtaken by events.** `issues --drafts` flagged it for citing closed PyAutoLens#514; chasing that found three things against upstream main: the target file moved (`scripts/jax_likelihood_functions/point_source/` -> `scripts/point_source/jax_likelihood/`), the reported mechanism — `FitPositionsImagePairAll` returning NaN, which fitness.py converts to the -1e99 `resample_figure_of_merit` — is what PyAutoLens 2a3f1a63 (PR #662) fixed on 2026-07-28 as phase 1 of the rhayes audit epic, and d838ca59 then changed `AnalysisPoint`'s default fit class out from under the script. Recorded on the prompt rather than closed: the surviving `-83.38049778` literal proves nothing either way (the prompt's own doctrine leaves a failing literal in place while the bug is open), so it needs one laptop run to settle. Incidental repair nobody folded back into the Mind is precisely the drift this tooling exists to catch. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01XqCY1yLfdB4SR97SXjTutq --- .../jax_point_source_point_smoke_sentinel.md | 55 +++++++- tests/test_lifecycle_check.py | 122 ++++++++++++++++++ 2 files changed, 176 insertions(+), 1 deletion(-) diff --git a/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md b/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md index 4f0a1e71..c9cd37cc 100644 --- a/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md +++ b/draft/bug/autolens/jax_point_source_point_smoke_sentinel.md @@ -10,13 +10,66 @@ Repos: Difficulty: medium Autonomy: supervised Priority: normal -Status: draft +Status: draft — NEEDS RE-VERIFICATION before any work (see 2026-08-09 note) > Restored 2026-07-27. This prompt file was dropped from the flat `issued/` pile > during the prompt-lifecycle migration (PR #71/#72) as "legacy", but the task was > never done — its `planned.md` entry stayed live and pointed at the deleted path. > Body below is the original 2026-05-21 content, verbatim. +## 2026-08-09 — DO NOT execute the task below as written + +Surfaced by `lifecycle.py issues --drafts`, which flagged this prompt for citing +a now-closed issue (PyAutoLens#514, closed 2026-05-16). Chasing that flag against +upstream `main` found the prompt has been overtaken on three separate axes. None +of this was checked by running the script — see "What is still unknown" below. + +**1. The target file moved.** The path this prompt names throughout, +`scripts/jax_likelihood_functions/point_source/point.py`, is a 404 on +`autolens_workspace_test@main`. It is now +`scripts/point_source/jax_likelihood/point.py` (siblings `image_plane.py`, +`source_plane.py` and a new `fluxes_time_delays.py` moved with it). Every path in +the "Task" and "Pre-existing context" sections below needs rewriting before use. + +**2. The root cause looks fixed.** PyAutoLens `2a3f1a63` (2026-07-28, PR +[#662](https://github.com/PyAutoLabs/PyAutoLens/pull/662)) — *"give +`FitPositionsImagePairAll` a no-image floor; recover from zero images"* — +describes exactly the mechanism this prompt reports: + +> `FitPositionsImagePairAll.chi_squared` returned NaN with no model positions +> […] `fitness.py` converts a NaN log-likelihood into `resample_figure_of_merit`, +> so the model was silently resampled instead of scored […] `FitPositionsImagePair` +> and `FitPositionsImagePairRepeat` both already applied that floor; `PairAll` was +> the one sibling that did not. + +`resample_figure_of_merit` **is** the `-1.0e99` sentinel this prompt observes, and +`PairAll` is the fit class it exercises. That commit landed as phase 1 of the +@rhayes777 audit epic (PyAutoArray#415), not from this prompt — so this is +incidental repair, which is exactly the class of drift the Mind keeps missing. + +**3. The fit class under test changed default.** PyAutoLens `d838ca59` +(2026-08-01, breaking) switched `AnalysisPoint` to default to +`FitPositionsImagePairAllSolved`. The script constructs `al.AnalysisPoint(...)` +without an explicit `fit_positions_cls`, so it no longer exercises the class this +prompt is about. Consistent with that, `point.py` on main has gained a second +assertion block pinned at `EXPECTED_VMAP_LOG_LIKELIHOOD_POINT_ALL_SOLVED = +-82.33883111` — a finite value, not a sentinel. + +**What is still unknown.** The original `-83.38049778` assertion survives verbatim +on main (line ~233). That proves nothing either way: this prompt's own doctrine is +to leave a failing literal in place while the bug is open, so its presence is +equally consistent with "still broken" and with "fixed but never re-run". Settling +it needs one laptop run of the relocated script against current library `main` — +which a cloud session cannot do (needs the JAX stack plus the committed seed +dataset; note the standing "do not run under `PYAUTO_SMALL_DATASETS=1`" warning +below, which would delete that seed data). + +**Next step:** run it. Then either close this prompt out as shipped-by-#662 and +rebaseline the literal, or rewrite the paths and the fit-class assumption and +keep it open against whatever actually reproduces. + +--- + Smoke regression surfaced during the `fast-viz-zero-contour-perf` task (workspace PR https://github.com/PyAutoLabs/autolens_workspace_test/pull/111). diff --git a/tests/test_lifecycle_check.py b/tests/test_lifecycle_check.py index 4e4d9511..0a98825f 100644 --- a/tests/test_lifecycle_check.py +++ b/tests/test_lifecycle_check.py @@ -356,6 +356,128 @@ def test_unreadable_issue_state_is_reported_not_swallowed(tmp_path): assert "could not read issue state" in problems[0] +# --------------------------------------------------------------------------- # +# the default fetcher itself +# +# Everything above injects `fetch`, which is what keeps those tests hermetic — +# but it also means the real `_gh_issue_states` shim, the thing that runs on a +# machine that HAS gh, was never executed by the suite. These tests drive it +# with `subprocess.run` stubbed, so the argv, the parsing and both failure +# modes are pinned without a network or a `gh` binary. +# +# `_gh_issue_states` does `import subprocess` inside the function body, which +# rebinds the same module object from sys.modules — so patching the attribute +# on the real module reaches it. +# --------------------------------------------------------------------------- # +class _Completed: + """Stand-in for subprocess.CompletedProcess.""" + + def __init__(self, returncode=0, stdout="", stderr=""): + self.returncode = returncode + self.stdout = stdout + self.stderr = stderr + + +def _stub_run(monkeypatch, handler): + """Patch subprocess.run, recording every argv the shim builds.""" + import subprocess + + calls = [] + + def fake_run(argv, **kwargs): + calls.append(argv) + return handler(argv) + + monkeypatch.setattr(subprocess, "run", fake_run) + return calls + + +def test_default_fetcher_builds_the_gh_argv_and_parses_state(monkeypatch): + """The happy path: one `gh api` call per URL, `.state` jq-extracted, and + the trailing newline gh emits stripped off.""" + calls = _stub_run(monkeypatch, lambda argv: _Completed(stdout="open\n")) + + states = lifecycle._gh_issue_states([GHOST_ISSUE]) + + assert states == {GHOST_ISSUE: "open"} + assert calls == [ + [ + "gh", + "api", + "repos/FictionalOrg/FlywheelRepo/issues/17", + "--jq", + ".state", + ] + ] + + +def test_default_fetcher_raises_gh_unavailable_when_gh_is_missing(monkeypatch): + """The one error the command turns into "could not run" rather than a + finding — so it must be the exception type, not a state string.""" + import pytest + + def _missing(argv): + raise FileNotFoundError(2, "No such file or directory: 'gh'") + + _stub_run(monkeypatch, _missing) + + with pytest.raises(lifecycle.GhUnavailable): + lifecycle._gh_issue_states([GHOST_ISSUE]) + + +def test_default_fetcher_reports_a_failed_call_as_unreadable(monkeypatch): + """gh ran and said no (404, revoked token, rate limit). That is a finding, + not a crash — and `issue_problems` grades any non-'open' state, so the + string it stores must not be mistaken for 'closed'.""" + _stub_run( + monkeypatch, + lambda argv: _Completed( + returncode=1, + stderr="gh: Not Found (HTTP 404)\n", + ), + ) + + states = lifecycle._gh_issue_states([GHOST_ISSUE]) + + assert states[GHOST_ISSUE].startswith("unreadable: ") + assert "HTTP 404" in states[GHOST_ISSUE] + assert states[GHOST_ISSUE] != "closed" + + +def test_default_fetcher_survives_a_failure_with_no_stderr(monkeypatch): + """The `or ["error"]` fallback: a non-zero exit with empty stderr must not + IndexError its way out of the whole check.""" + _stub_run(monkeypatch, lambda argv: _Completed(returncode=1, stderr=" \n")) + + assert lifecycle._gh_issue_states([GHOST_ISSUE]) == {GHOST_ISSUE: "unreadable: error"} + + +def test_default_fetcher_skips_anything_that_is_not_an_issue_url(monkeypatch): + """Guards the loop's `if not m: continue` — a malformed entry costs no + subprocess call and contributes no state, rather than querying nonsense.""" + calls = _stub_run(monkeypatch, lambda argv: _Completed(stdout="open\n")) + + states = lifecycle._gh_issue_states(["(no issue — a release drive)"]) + + assert states == {} + assert calls == [] + + +def test_default_fetcher_reads_each_url_in_a_mixed_batch(monkeypatch): + """Two URLs, different answers — the shim must key states by URL rather + than collapsing or reusing the last result.""" + + def _by_number(argv): + return _Completed(stdout="closed\n" if argv[2].endswith("/18") else "open\n") + + _stub_run(monkeypatch, _by_number) + + assert lifecycle._gh_issue_states([GHOST_ISSUE, OTHER_ISSUE]) == { + GHOST_ISSUE: "open", + OTHER_ISSUE: "closed", + } + + # --------------------------------------------------------------------------- # # the mirror direction — active/ prompts no registry claims # --------------------------------------------------------------------------- #