Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 54 additions & 1 deletion draft/bug/autolens/jax_point_source_point_smoke_sentinel.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,66 @@ Repos:
Difficulty: medium
Autonomy: supervised
Priority: normal
Status: draft
Status: draft — NEEDS RE-VERIFICATION before any work (see 2026-08-09 note)

> Restored 2026-07-27. This prompt file was dropped from the flat `issued/` pile
> during the prompt-lifecycle migration (PR #71/#72) as "legacy", but the task was
> never done — its `planned.md` entry stayed live and pointed at the deleted path.
> Body below is the original 2026-05-21 content, verbatim.

## 2026-08-09 — DO NOT execute the task below as written

Surfaced by `lifecycle.py issues --drafts`, which flagged this prompt for citing
a now-closed issue (PyAutoLens#514, closed 2026-05-16). Chasing that flag against
upstream `main` found the prompt has been overtaken on three separate axes. None
of this was checked by running the script — see "What is still unknown" below.

**1. The target file moved.** The path this prompt names throughout,
`scripts/jax_likelihood_functions/point_source/point.py`, is a 404 on
`autolens_workspace_test@main`. It is now
`scripts/point_source/jax_likelihood/point.py` (siblings `image_plane.py`,
`source_plane.py` and a new `fluxes_time_delays.py` moved with it). Every path in
the "Task" and "Pre-existing context" sections below needs rewriting before use.

**2. The root cause looks fixed.** PyAutoLens `2a3f1a63` (2026-07-28, PR
[#662](https://github.com/PyAutoLabs/PyAutoLens/pull/662)) — *"give
`FitPositionsImagePairAll` a no-image floor; recover from zero images"* —
describes exactly the mechanism this prompt reports:

> `FitPositionsImagePairAll.chi_squared` returned NaN with no model positions
> […] `fitness.py` converts a NaN log-likelihood into `resample_figure_of_merit`,
> so the model was silently resampled instead of scored […] `FitPositionsImagePair`
> and `FitPositionsImagePairRepeat` both already applied that floor; `PairAll` was
> the one sibling that did not.

`resample_figure_of_merit` **is** the `-1.0e99` sentinel this prompt observes, and
`PairAll` is the fit class it exercises. That commit landed as phase 1 of the
@rhayes777 audit epic (PyAutoArray#415), not from this prompt — so this is
incidental repair, which is exactly the class of drift the Mind keeps missing.

**3. The fit class under test changed default.** PyAutoLens `d838ca59`
(2026-08-01, breaking) switched `AnalysisPoint` to default to
`FitPositionsImagePairAllSolved`. The script constructs `al.AnalysisPoint(...)`
without an explicit `fit_positions_cls`, so it no longer exercises the class this
prompt is about. Consistent with that, `point.py` on main has gained a second
assertion block pinned at `EXPECTED_VMAP_LOG_LIKELIHOOD_POINT_ALL_SOLVED =
-82.33883111` — a finite value, not a sentinel.

**What is still unknown.** The original `-83.38049778` assertion survives verbatim
on main (line ~233). That proves nothing either way: this prompt's own doctrine is
to leave a failing literal in place while the bug is open, so its presence is
equally consistent with "still broken" and with "fixed but never re-run". Settling
it needs one laptop run of the relocated script against current library `main` —
which a cloud session cannot do (needs the JAX stack plus the committed seed
dataset; note the standing "do not run under `PYAUTO_SMALL_DATASETS=1`" warning
below, which would delete that seed data).

**Next step:** run it. Then either close this prompt out as shipped-by-#662 and
rebaseline the literal, or rewrite the paths and the fit-class assumption and
keep it open against whatever actually reproduces.

---

Smoke regression surfaced during the `fast-viz-zero-contour-perf` task
(workspace PR https://github.com/PyAutoLabs/autolens_workspace_test/pull/111).

Expand Down
122 changes: 122 additions & 0 deletions tests/test_lifecycle_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,128 @@ def test_unreadable_issue_state_is_reported_not_swallowed(tmp_path):
assert "could not read issue state" in problems[0]


# --------------------------------------------------------------------------- #
# the default fetcher itself
#
# Everything above injects `fetch`, which is what keeps those tests hermetic —
# but it also means the real `_gh_issue_states` shim, the thing that runs on a
# machine that HAS gh, was never executed by the suite. These tests drive it
# with `subprocess.run` stubbed, so the argv, the parsing and both failure
# modes are pinned without a network or a `gh` binary.
#
# `_gh_issue_states` does `import subprocess` inside the function body, which
# rebinds the same module object from sys.modules — so patching the attribute
# on the real module reaches it.
# --------------------------------------------------------------------------- #
class _Completed:
"""Stand-in for subprocess.CompletedProcess."""

def __init__(self, returncode=0, stdout="", stderr=""):
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr


def _stub_run(monkeypatch, handler):
"""Patch subprocess.run, recording every argv the shim builds."""
import subprocess

calls = []

def fake_run(argv, **kwargs):
calls.append(argv)
return handler(argv)

monkeypatch.setattr(subprocess, "run", fake_run)
return calls


def test_default_fetcher_builds_the_gh_argv_and_parses_state(monkeypatch):
"""The happy path: one `gh api` call per URL, `.state` jq-extracted, and
the trailing newline gh emits stripped off."""
calls = _stub_run(monkeypatch, lambda argv: _Completed(stdout="open\n"))

states = lifecycle._gh_issue_states([GHOST_ISSUE])

assert states == {GHOST_ISSUE: "open"}
assert calls == [
[
"gh",
"api",
"repos/FictionalOrg/FlywheelRepo/issues/17",
"--jq",
".state",
]
]


def test_default_fetcher_raises_gh_unavailable_when_gh_is_missing(monkeypatch):
"""The one error the command turns into "could not run" rather than a
finding — so it must be the exception type, not a state string."""
import pytest

def _missing(argv):
raise FileNotFoundError(2, "No such file or directory: 'gh'")

_stub_run(monkeypatch, _missing)

with pytest.raises(lifecycle.GhUnavailable):
lifecycle._gh_issue_states([GHOST_ISSUE])


def test_default_fetcher_reports_a_failed_call_as_unreadable(monkeypatch):
"""gh ran and said no (404, revoked token, rate limit). That is a finding,
not a crash — and `issue_problems` grades any non-'open' state, so the
string it stores must not be mistaken for 'closed'."""
_stub_run(
monkeypatch,
lambda argv: _Completed(
returncode=1,
stderr="gh: Not Found (HTTP 404)\n",
),
)

states = lifecycle._gh_issue_states([GHOST_ISSUE])

assert states[GHOST_ISSUE].startswith("unreadable: ")
assert "HTTP 404" in states[GHOST_ISSUE]
assert states[GHOST_ISSUE] != "closed"


def test_default_fetcher_survives_a_failure_with_no_stderr(monkeypatch):
"""The `or ["error"]` fallback: a non-zero exit with empty stderr must not
IndexError its way out of the whole check."""
_stub_run(monkeypatch, lambda argv: _Completed(returncode=1, stderr=" \n"))

assert lifecycle._gh_issue_states([GHOST_ISSUE]) == {GHOST_ISSUE: "unreadable: error"}


def test_default_fetcher_skips_anything_that_is_not_an_issue_url(monkeypatch):
"""Guards the loop's `if not m: continue` — a malformed entry costs no
subprocess call and contributes no state, rather than querying nonsense."""
calls = _stub_run(monkeypatch, lambda argv: _Completed(stdout="open\n"))

states = lifecycle._gh_issue_states(["(no issue — a release drive)"])

assert states == {}
assert calls == []


def test_default_fetcher_reads_each_url_in_a_mixed_batch(monkeypatch):
"""Two URLs, different answers — the shim must key states by URL rather
than collapsing or reusing the last result."""

def _by_number(argv):
return _Completed(stdout="closed\n" if argv[2].endswith("/18") else "open\n")

_stub_run(monkeypatch, _by_number)

assert lifecycle._gh_issue_states([GHOST_ISSUE, OTHER_ISSUE]) == {
GHOST_ISSUE: "open",
OTHER_ISSUE: "closed",
}


# --------------------------------------------------------------------------- #
# the mirror direction — active/ prompts no registry claims
# --------------------------------------------------------------------------- #
Expand Down
Loading