prompt: record multistart-gradient-resume-fom-sanity-check as shipped - #187
Merged
Merged
Conversation
PyAutoFit#1474 merged (a0af8574 -> main): `Fitness.check_log_likelihood` compared a stored log likelihood against the search's own figure of merit, so resuming any non-log-likelihood search failed on an unchanged likelihood function. Advances the prompt draft/bug/autofit/ -> complete/2026/08/, regenerates complete/index.md and dashboard.md. The record carries three traps worth keeping: the test config disables `check_likelihood_function` (so the unit suite cannot catch a regression by default), a GaussianPrior has log prior 0.0 at its mean (so a test evaluated there passes against the unfixed code), and the GitHub `get_check_runs` API serves stale data (a healthy CI run was cancelled on the strength of it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNbe6eLQbxUtY52EGqej5o
The counters accumulate correctly across a killed mid-run resume — no bug. Covered permanently by a new autofit_workspace_test script. Records two things worth reusing: `_broad_starts` rejects non-finite draws, so a NaN trap at the edges of the prior is never reached and must sit on the descent path; and the load-bearing assertion is equality with an uninterrupted reference run, since a `>=` check accepts the injected regression. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNbe6eLQbxUtY52EGqej5o
The end-to-end NaN-counter cover now has a PR to point at. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNbe6eLQbxUtY52EGqej5o
autofit_workspace_test#85 merged as 69ee1305 -> main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNbe6eLQbxUtY52EGqej5o
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lifecycle bookkeeping for PyAutoFit#1474, merged as
a0af8574->main.Fitness.check_log_likelihoodcompared the log likelihood stored in a previous run's samples summary againstfitness(parameters)— the figure of merit in the search's own convention. Those agree only whenfom_is_log_likelihood=Trueandconvert_to_chi_squared=False, so resuming any other search failed its sanity check on a completely unchanged likelihood function, making the multi-start resume path unusable.What this PR does
draft/bug/autofit/multistart_gradient_resume_fom_sanity_check.md->complete/2026/08/multistart-gradient-resume-fom-sanity-check.md, folding the original prompt into the recordcomplete/index.md(991 records) anddashboard.mdscripts/lifecycle.py checkpassesNo registry entry was removed because none existed — the task shipped straight from
draft/in a single remote session without a tracking issue, which the record states explicitly rather than leaving implicit.Traps the record captures
The record is deliberately heavy on the things that would cost someone else time:
test_autofit/config/general.yamlsetscheck_likelihood_function: false. That is why the library suite never caught this bug and cannot catch a regression of it by default — any test touching the resume sanity check has to flip the flag on via fixture.GaussianPriorhas log prior exactly 0.0 at its mean. A test parametrised over figure-of-merit conventions that evaluates at the prior means collapses the log-posterior conventions onto the log-likelihood one and passes against the unfixed code. The first draft of the test made exactly this mistake._broad_startsrejects any draw whose objective or gradient is non-finite, so a NaN trap placed at the edges of the prior is never reached — three verification runs reported counters of exactly 0 before this was spotted.get_check_runsAPI serves stale cached data. It froze all three CI jobs atin_progresslong after they had finished, and acting on that reading a healthy mid-suite run was cancelled and had to be re-run.list_workflow_jobs/get_workflow_runare accurate, andget_workflow_run_usagegaining arun_duration_mskey is a reliable completion signal.Follow-up, also closed
PyAutoFit#1472's
n_value_nan_lane_steps/n_grad_nan_lane_stepscounters were never demonstrable end-to-end because this bug killed every resume before the fit loop. Now verified — no bug, they accumulate correctly — and covered permanently by autofit_workspace_test#85, cited in the record.Generated by Claude Code