Count multi-start lanes trapped outside a declared model constraint - #1475
Merged
Conversation
…aint `_nan_lane_counts` splits failing lanes into value-NaN and gradient-NaN. A third mode escapes both: a lane whose value and gradient are finite but which sits on a saturating plateau with no gradient along the saturated direction, so it can never leave. The reference case is the ell_comps magnitude clamp, past which the likelihood is exactly constant radially. Such a lane is finite, differentiable and permanently wrong, and its flat figure of merit is indistinguishable from convergence. Detecting it from the gradient does not work: the clamp kills only the radial derivative while the angle stays differentiable, so in general position both components carry a large non-zero gradient and their radial projection is zero only to floating-point residue. A component-wise `== 0` test finds nothing. Instead a class declares its own valid region via `__model_constraint__`, returning a traced non-negative violation measure. This is the differentiable sibling of `add_assertion`: declared on the class rather than attached per model, and evaluated as a traced value rather than raised, because a `raise` on a traced condition gives TracerBoolConversionError and cannot work under jit. A measure rather than a bool so a later penalty term can reuse it unchanged. Nothing flows up through the likelihood — the constraint is a pure function of the parameter vector, evaluated beside `fitness.call`, which is untouched. It rides the fused value_and_grad as a fourth output, on the device->host sync that already happens each step; cost is a fixed ~20 HLO lines independent of likelihood cost, plus one array per step. Measurement only: it never gates a redraw and never changes stepping, matching the existing gradient-NaN counter. Models declaring no constraint short-circuit and are bit-identical. Verified end to end on a MultiStartProdigy run over a likelihood carrying the real clamp: 15-16 of 32 lanes trapped and reported, while the existing counters showed `alive 30/32` and zero gradient-NaN. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GAFoogitLceTsgA7bfB4k
Jammy2211
added a commit
that referenced
this pull request
Aug 16, 2026
The Clipper (#1477) counts how often it fires -- n_clipped_lane_steps is accumulated per-lane in AbstractMultiStartGradient and written to search_internal -- but the count never reached the artefact a user actually reads to find out what a search did. search.summary said nothing about clipping at all. The channel already existed: search_summary_from_samples reads samples.samples_info and emits the NaN counters and their rates, guarded on the key so searches without them are unaffected. Two things were missing from it rather than a mechanism: - n_clipped_lane_steps never reached samples_info. It stopped at search_internal, so nothing downstream could see it. - n_constrained_lane_steps did reach samples_info but was never emitted, so the trapped-lane counter from #1475 has been invisible in search.summary since it shipped. Now reported. Clipping is reported in three cases, and the distinction between the last two is the point: - No clipper (ClipperNone, or a search predating the Clipper): emits nothing. The default path's summary is unchanged, which matters because this file is read by tooling and sits in every archived run's output. - Clipped and counted (MultiStartGradient): it enforces the constraint itself every step via Clipper.project, so it knows how often it fired and reports the count and the rate, denominated by n_starts * total_steps like the NaN rates beside it. - Clipped but not observable (LBFGS and the bound-supporting scipy methods): declarative, handing optimize.Bounds to scipy and letting scipy enforce, so project is never called and no mask exists. Reporting 0 there would read as "the clipper never fired" when it means "this search cannot know", so it says "not measured (bounds enforced by scipy)" instead. The clipper is published as its class NAME rather than a bool, so the summary can say which strategy ran and a later strategy needs no schema change. The count is per-LANE, not per-coordinate -- a lane clipped in three parameters on one step is one clipped lane-step -- matching how the counters beside it read, so all four stay directly comparable. It is restored from search_internal as a lifetime total, so a resumed run reports the whole run's clipping rather than the current process's share. Verified end-to-end against the search.summary files four real searches wrote, not just the formatting helper in isolation: LBFGS default (no clipping lines), LBFGS clipped ("not measured"), MultiStart default (no clipping lines), MultiStart clipped (Clipped Lane-Steps = 414, rate 0.958). One behaviour change to note: multi-start summaries gain a Constrained Lane-Steps line they did not have before. Everything else is additive and gated. Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
_nan_lane_countssplits failing multi-start lanes into value-NaN (likelihood undefined) and gradient-NaN (defined but not differentiable). A third mode escapes both: a lane whose value and gradient are finite, but which sits on a saturating plateau with no gradient along the saturated direction, so it can never leave.The reference case is the
ell_compsmagnitude clamp (jax.lax.min(fac, 0.999)in PyAutoGalaxy'sconvert.py). Past|ell_comps| >= 0.999the axis ratio pins and the likelihood is exactly constant radially — measured identical to 10 significant figures at magnitude 1.0, 1.2 and 5.0. Such a lane is finite, differentiable, permanently wrong, and its flat figure of merit is indistinguishable from convergence.This adds a third disjoint counter for it, plus the mechanism it needs.
Why not detect it from the gradient
Because that does not work, and it is worth recording why. The clamp kills only the radial derivative; the angle still comes from an unclamped
arctan2, so in general position bothell_compscomponents carry a large non-zero gradient while their radial projection is zero only to floating-point residue. Scored against 17 genuinely trapped lanes in a JAX reproduction:Per-parameter prior limits score well only because the overshoot is gross; they provably cannot see the corner region (both components inside
(-1, 1), magnitude above 1 — the whole4 - piarea between the unit disc and unit square). The declared constraint states the valid region exactly rather than inferring it.The mechanism
A class declares its own valid region via
__model_constraint__, returning a traced non-negative violation measure:This is the differentiable sibling of
add_assertion, with two changes: declared on the class rather than attached per model, and evaluated as a traced value rather than raised. The second is forced — araiseon a traced condition givesTracerBoolConversionError, which is exactly why guards likevalidate_ell_compsreturn early for non-concrete scalars rather than raising.A measure rather than a bool is deliberate: a bool suffices for counting, but a magnitude is what a penalty term would need later and carries a usable gradient back into the valid region.
Cost
Nothing flows up through the likelihood. The constraint is a pure function of the parameter vector, evaluated beside
fitness.call, which is untouched. It rides the fusedvalue_and_gradas a fourth output, on the device→host sync that already happens each step — the same pattern and rationale as the existinggrad_finitereduction.Measured: a flat +20 HLO lines at every scale tested (grid 31→256, i.e. ~1.2 ms→93 ms per call; starts 32→128), plus 128 bytes of transfer. The cost is fixed, not proportional to likelihood cost. Wall-clock overhead was inside the noise band on a shared CPU and is not reported as a number.
Models declaring no constraint short-circuit and are bit-identical.
Scope
Measurement only. It never gates a redraw and never changes stepping, matching the existing gradient-NaN counter — the policy question waits on what the counts show. No penalty term. No change to
resurrect,apply_if_finite, the convergence check, or the clamp.Nothing in PyAutoGalaxy declares a constraint yet, so on real lens models this reads zero until
EllProfileopts in.Verification
End to end on a real
MultiStartProdigyrun over a likelihood carrying the actual clamp:15 of 32 lanes permanently trapped — which the existing instrumentation reports as
alive 30/32with zero gradient-NaN. The surviving lanes still recovered the truth (|ell_comps|0.9001 against 0.90), which is the honest severity: wasted budget rather than a corrupted result, untiln_startsis small or the good basin is rare.Suite: 1745 passed, +15 new tests. Five failures are pre-existing — verified identical on a stashed clean tree (missing
astropy, and a nautilus pool test).Review note
This touches
Model.__init__andAbstractPriorModel, which every model composition path runs through, so the blast radius is wider than the feature. The short-circuit for constraint-free models is the thing most worth a careful look.Formatting was deliberately left alone:
blackwants to reformat all three modified files atmaintoo, so running it would bury the change in unrelated churn. The two new files areblack-clean.Generated by Claude Code