From d0f5eaae93821bf27055e4ead34a35436f4055c0 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 18:12:21 +0000 Subject: [PATCH] record: nss-remainline-mainline-blackjax shipped (PyAutoFit#1492 + workspace_test#86) Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_018dV9h5h76hZ6trGZKWJFHe --- active.md | 19 ---- active/nss_remainline_mainline_blackjax.md | 49 ---------- .../08/nss-remainline-mainline-blackjax.md | 98 +++++++++++++++++++ complete/index.md | 3 +- 4 files changed, 100 insertions(+), 69 deletions(-) delete mode 100644 active/nss_remainline_mainline_blackjax.md create mode 100644 complete/2026/08/nss-remainline-mainline-blackjax.md diff --git a/active.md b/active.md index 8d994406..fd9a4176 100644 --- a/active.md +++ b/active.md @@ -1,24 +1,5 @@ # Active Tasks -## nss-remainline-mainline-blackjax -- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1491 -- session: Claude Code cloud/mobile (no local worktree) -- status: pr-open (https://github.com/PyAutoLabs/PyAutoFit/pull/1492) — 17/17 nss tests - pass from checkout; CI + human merge pending; session subscribed to PR activity -- worktree: none — cloud session works on the GitHub clone directly -- repos: - - PyAutoFit: feature/nss-remainline-mainline-blackjax - - autofit_workspace_test: feature/nss-searches-integration-test -- prompt: active/nss_remainline_mainline_blackjax.md -- integration-test-PR: https://github.com/PyAutoLabs/autofit_workspace_test/pull/86 - (scripts/searches/NSS.py — truth recovery + logZ ensemble + chunked bit-parity; - validated against the #1492 branch in direct and TEST_MODE=1 runs; MERGE AFTER - PyAutoFit#1492 — af.NSS absent from main until then) -- note: human-directed early re-mainlining of af.NSS on mainline blackjax >=1.6 - (removal reason — git-fork pins — is gone). Port pre-validated on CPU toy - (chunked bit-identical to unchunked; logZ within 0.6 sigma of analytic). - Gate A still decides baseline adoption; DECISIONS.md entry to follow. - ## stored-sample-reconstruction-guard - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1486 - status: library-dev — WORKSPACE HALF SHIPPED; the PyAutoFit hardening (#1486) is what remains diff --git a/active/nss_remainline_mainline_blackjax.md b/active/nss_remainline_mainline_blackjax.md deleted file mode 100644 index c52ae382..00000000 --- a/active/nss_remainline_mainline_blackjax.md +++ /dev/null @@ -1,49 +0,0 @@ -# Re-mainline af.NSS as a proper PyAutoFit search on mainline blackjax ≥1.6 - -Type: feature -Target: autofit -Repos: -- @PyAutoFit -Difficulty: small -Autonomy: supervised -Priority: high - -Human-directed 2026-08-18 (mobile session): the parked `af.NSS` nested slice -sampler — removed from PyAutoFit on 2026-07-11 (PyAutoFit#1356/#1357) solely -because its dependencies were unshippable git-fork pins — should be -reimplemented as a proper first-class search now that nested sampling is in -**mainline blackjax 1.6** (the whole removal reason is gone). The full -working implementation is preserved verbatim at -`autofit_workspace_developer/searches/nss/` with a re-mainlining checklist -in its README. - -Port surface (validated on CPU 2026-08-18 in the cloud session before any -source change; see autolens_profiling#142 for the environment validation): - -- `from nss.ns import log_weights, finalise` → `blackjax.ns.utils`. -- SliceInfo eval counter: fork `num_steps` → mainline `num_expansions`. -- Chunked GPU-memory path: the fork's `build_kernel(update_strategy=)` seam - is gone — recompose from mainline public helpers (`build_slice_kernel`, - `slice_constrained_step`, `build_adaptive_kernel`, `live_covariance`); - `_chunked_update.py` is already byte-equivalent to mainline's - `update_with_mcmc_take_last` bar the vmap→lax.map swap. -- End-to-end validation on a 2D analytic toy through the full search - pipeline: logZ −4.558 ± 0.078 vs analytic −4.605; **chunked path - bit-identical to unchunked** (same logZ/evals/ESS at fixed seed). - -Task (stash README checklist, PyAutoFit side): -1. Restore `autofit/non_linear/search/nest/nss/` (search.py, samples.py, - _chunked_nss.py, _chunked_update.py, __init__.py) with the mainline port - and fork-era prose removed. -2. Restore `test_autofit/non_linear/search/nest/nss/` (kept verbatim against - library import paths; one ImportError match-string update). -3. `autofit/__init__.py`: re-export `NSS`. -4. `pyproject.toml`: bump the `optional` extra's blackjax floor to ≥1.6.2 - (no cap exists). -5. Run the nss test subset + smoke the search end-to-end from the checkout. - -Note the programme-plan interaction: PROGRAMME §7 gated "af.NSS re-mainlined -on blackjax.nss" behind Gate A — the human has directed early re-mainlining; -Gate A still decides whether NSS becomes the GPU nested-sampling *baseline*. -Record the scope change in the inference DECISIONS.md. The workspace -tutorial restoration (checklist item 5) is a separate follow-up task. diff --git a/complete/2026/08/nss-remainline-mainline-blackjax.md b/complete/2026/08/nss-remainline-mainline-blackjax.md new file mode 100644 index 00000000..8cee2983 --- /dev/null +++ b/complete/2026/08/nss-remainline-mainline-blackjax.md @@ -0,0 +1,98 @@ +Re-mainlined `af.NSS` (nested slice sampling) into PyAutoFit as a proper +first-class search on **mainline blackjax ≥1.6**, with integration-test +coverage restored — human-directed 2026-08-18, executed entirely from a +mobile/cloud session on CPU. + +- **PRs (both merged):** PyAutoFit#1492 (`401e666`, closes #1491) — the + search, tests, `__init__` export, optional-extra blackjax floor + ≥1.2.0 → ≥1.6.2; autofit_workspace_test#86 (`12da16d`) — + `scripts/searches/NSS.py` integration script. +- **Why now:** the search was removed in PyAutoFit#1356/#1357 solely because + its dependencies were unshippable git-fork pins; nested sampling merged + into mainline blackjax 1.6, so the removal reason vanished. Implementation + restored verbatim from the `autofit_workspace_developer/searches/nss/` + stash (whose README's re-mainlining checklist this executes). +- **The mainline port (3 diffs):** `from nss.ns import log_weights, finalise` + → `blackjax.ns.utils`; `SliceInfo.num_steps` → `num_expansions` (eval + counter); the chunked GPU-memory builder recomposed from mainline public + helpers (`build_slice_kernel`/`slice_constrained_step`/ + `build_adaptive_kernel`/`live_covariance`) since mainline `build_kernel` + dropped the fork's `update_strategy=` seam. `_chunked_update.py` needed no + logic change (byte-equivalent to mainline's `update_with_mcmc_take_last` + bar the vmap→lax.map swap). +- **Validation:** port proven on a 2D analytic toy BEFORE any source change + (logZ −4.558 ± 0.078 vs analytic −4.605; **chunked bit-identical to + unchunked at fixed seed**); 17/17 restored unit tests + full CI green + (3.12/3.13/docs); integration script validated in direct and + PYAUTO_TEST_MODE=1 runs against the PR branch (logZ −72.16 ± 0.42, ESS + 417, chunked ≡ unchunked) — merged after #1492 per the library-first gate. +- **Traps for the record:** (1) a quick-update Fitness-wiring convention test + added after the removal (`test_quick_update_wiring.py`) tripped on the + restored search — NSS never samples through Fitness, so the fix is an + EXEMPT entry with reason, not forwarding the kwarg to a dead path. + (2) PyAutoFit#1494 (clipper identifiers) landed on main mid-PR — its + nested-sampler identifier tripwire pins a fixed search list, so NSS is + unaffected; verified by merging main locally and running all 55 affected + tests before pressing merge. (3) blackjax is an optional extra, not a hard + autofit dep — environments must install/upgrade it explicitly. +- **Programme tie-in:** scope-change entry in the inference DECISIONS.md + (autolens_profiling#144): re-mainlined ahead of Gate A per human direction; + Gate A still decides GPU-baseline adoption. Phase 2 can now drive `af.NSS` + directly (the "profiling-local runner first" hedge is obsolete). +- **Follow-ups:** restore the `Search: NSS` tutorial section in + `autofit_workspace/scripts/searches/nest.py` (stash checklist item 5, + user-facing docs); blackjax ≥1.6.2 upgrades in the local venvs + RAL stack + (re-run `autolens_profiling/scripts/misc/searches/nss_smoke.py` there); + Phase 2 GPU re-benchmarking campaign. + +## Original prompt + +# Re-mainline af.NSS as a proper PyAutoFit search on mainline blackjax ≥1.6 + +Type: feature +Target: autofit +Repos: +- @PyAutoFit +Difficulty: small +Autonomy: supervised +Priority: high + +Human-directed 2026-08-18 (mobile session): the parked `af.NSS` nested slice +sampler — removed from PyAutoFit on 2026-07-11 (PyAutoFit#1356/#1357) solely +because its dependencies were unshippable git-fork pins — should be +reimplemented as a proper first-class search now that nested sampling is in +**mainline blackjax 1.6** (the whole removal reason is gone). The full +working implementation is preserved verbatim at +`autofit_workspace_developer/searches/nss/` with a re-mainlining checklist +in its README. + +Port surface (validated on CPU 2026-08-18 in the cloud session before any +source change; see autolens_profiling#142 for the environment validation): + +- `from nss.ns import log_weights, finalise` → `blackjax.ns.utils`. +- SliceInfo eval counter: fork `num_steps` → mainline `num_expansions`. +- Chunked GPU-memory path: the fork's `build_kernel(update_strategy=)` seam + is gone — recompose from mainline public helpers (`build_slice_kernel`, + `slice_constrained_step`, `build_adaptive_kernel`, `live_covariance`); + `_chunked_update.py` is already byte-equivalent to mainline's + `update_with_mcmc_take_last` bar the vmap→lax.map swap. +- End-to-end validation on a 2D analytic toy through the full search + pipeline: logZ −4.558 ± 0.078 vs analytic −4.605; **chunked path + bit-identical to unchunked** (same logZ/evals/ESS at fixed seed). + +Task (stash README checklist, PyAutoFit side): +1. Restore `autofit/non_linear/search/nest/nss/` (search.py, samples.py, + _chunked_nss.py, _chunked_update.py, __init__.py) with the mainline port + and fork-era prose removed. +2. Restore `test_autofit/non_linear/search/nest/nss/` (kept verbatim against + library import paths; one ImportError match-string update). +3. `autofit/__init__.py`: re-export `NSS`. +4. `pyproject.toml`: bump the `optional` extra's blackjax floor to ≥1.6.2 + (no cap exists). +5. Run the nss test subset + smoke the search end-to-end from the checkout. + +Note the programme-plan interaction: PROGRAMME §7 gated "af.NSS re-mainlined +on blackjax.nss" behind Gate A — the human has directed early re-mainlining; +Gate A still decides whether NSS becomes the GPU nested-sampling *baseline*. +Record the scope change in the inference DECISIONS.md. The workspace +tutorial restoration (checklist item 5) is a separate follow-up task. diff --git a/complete/index.md b/complete/index.md index 3d6fed37..38d3718c 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -1014 records across 7 buckets. +1015 records across 7 buckets. ## Highlights @@ -88,6 +88,7 @@ _(curate hard-won records here — survives regeneration.)_ - [nautilus-1core-serial-pool](2026/08/nautilus-1core-serial-pool.md) — corrective for the Heart RED "release validation FAILED (stage - [normalise-auto-simulate-guard-idiom](2026/08/normalise-auto-simulate-guard-idiom.md) - [notebook-setup-notebook-regen-drift](2026/08/notebook-setup-notebook-regen-drift.md) +- [nss-remainline-mainline-blackjax](2026/08/nss-remainline-mainline-blackjax.md) - [nufft-parity-full-datasets](2026/08/nufft-parity-full-datasets.md) - [numerical-hazard-profiling](2026/08/numerical-hazard-profiling.md) — Phase 1 established the reusable numerical-hazard profiling framework, typed risk measurements, reachability a… - [per-parameter-step-scaling](2026/08/per-parameter-step-scaling.md)