diff --git a/dashboard.html b/dashboard.html index 64ac3eea..88b55d6a 100644 --- a/dashboard.html +++ b/dashboard.html @@ -38,7 +38,7 @@

📋 PyAutoMind Dashboard

Every task the Mind is holding. Tap a task's 📋 and its /start_dev command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task.

-

In flight 2 · Parked 3 · Planned 6 · Backlog 146 · markdown version

+

In flight 2 · Parked 3 · Planned 6 · Backlog 148 · markdown version

Start here

Highest priority (filed as high) — showing 12 of 18

TRIAGE: needs manual review before routing — medium · safe · high

@@ -54,7 +54,7 @@

Highest priority (filed as high) — showing 12 of 18

einstein_radius_jit_from: replace static init_guess with a JAX-native seed finder — autogalaxy · too-large · supervised · high

Split lensing regimes: multi_galaxy / group / cluster (epic plan) — autolens · too-large · supervised · high

Quick wins (small enough, and safe enough to run unattended)

-

(none right now)

+

Defer the eager scipy.sparse import in derivative_util (~0.10 s of — libraries · small · safe · normal

In flight markdown version

Issued — each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first call — issue #1501 — HOLD — do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

@@ -77,7 +77,7 @@

Planned

latent-nan-guard-honest-run

Backlog markdown version

-

146 filed prompts, not started — sorted most-pickable first (priority, then size). 23 of them belong to an epic and are listed only under Epics below.

+

148 filed prompts, not started — sorted most-pickable first (priority, then size). 23 of them belong to an epic and are listed only under Epics below.

feature — 28

Numba CPU likelihood phase 1: batched MGE convolution + operated-matrix — autoarray · medium · supervised · high

@@ -146,7 +146,7 @@

Backlog

interferometer/start_here.py OOM in nightly release-validation integrate leg — autolens

-research — 16 +research — 17

Deep research: Can we speed up Delaunay in PyAutoArray? — autoarray · too-large · supervised · high

Census of priors and messages — confirmed bugs + redesign — autofit · too-large · supervised · high

Delaunay-family JAX modules never hit the persistent compilation cache — autoarray · medium · supervised · medium

@@ -156,6 +156,7 @@

Backlog

Re-baseline the slacs0008 acceptance parity after the HAP-dedupe fix — pyautoreduce · small · supervised · normal

Kernel-CDF bandwidth defaults — config-dependent quality, investigate adaptivity — autoarray · medium · supervised · normal

We have lots of examples which profile how long JAX — autolens_workspace_developer · medium · supervised · normal

+

Is Intel macOS a supported platform, and what is the — libraries · medium · supervised · normal

Checkerboard PSF-mismatch residual diagnostic — research + document + ingest — pyautomemory · medium · supervised · normal

autofit_profiling: bootstrap the repo + general PyAutoFit profiling epic — autofit · large · supervised · normal

Multi-band compile census completion — A100/multi-core + hetero GPU rows — autolens_profiling · small · supervised · low

@@ -165,9 +166,10 @@

Backlog

Adopt Python 3.12 as the PyAuto ecosystem minimum — libraries

-maintenance — 18 +maintenance — 20

smoke_install.sh's stale jax<0.7 pin — CI is on the right — ci · low · supervised · medium

autolens_workspace_developer rectangular experiments — Gut stash + rename — autolens_workspace_developer · small · supervised · normal

+

Defer the eager scipy.sparse import in derivative_util (~0.10 s of — libraries · small · safe · normal

Mirror drifted library config keys into the workspace configs — workspaces · small · supervised · normal

Un-park imaging/features/scaling_relation/slam once PyAutoArray#431 merges — workspaces · small · supervised · normal

autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) — autolens_workspace_developer · medium · supervised · normal

@@ -179,6 +181,7 @@

Backlog

autolens_profiling is now a mature project, with a good separation — autolens_profiling · large · supervised · normal

Auto-request GitHub Copilot code review on every PR, org-wide — ci · large · supervised · normal

autolens_workspace — workspaces · too-large · supervised · normal

+

Remove pynufft + legacy TransformerNUFFTPyNUFFT — libraries · low-medium · supervised · normal

dataset/imaging/jwst_lw is untracked because the gitignore was never extended for — autolens_profiling · small · supervised · low

cosmos_web_ring stores boolean masks as float64, wasting ~3.4 MB of — autolens_workspace · small · supervised · low

LaTeX in non-raw docstrings emits SyntaxWarning: invalid escape sequence — autolens_workspace · small · supervised · low

@@ -203,12 +206,11 @@

Backlog

Phase 2: Make workspace READMEs assistant-first — workspaces

-refactor — 6 +refactor — 5

einstein_radius_jit_from: replace static init_guess with a JAX-native seed finder — autogalaxy · too-large · supervised · high

Vendor bessel_kve into autoarray and drop the tensorflow-probability dependency — autoarray · large · supervised · medium

Split Fitness.batch_size into lh_batch_size and latent_batch_size — autofit · small · supervised · normal

Remove the dead EDEN packaging tooling from PyAutoFit — pyautofit · medium · supervised · normal

-

Remove pynufft + legacy TransformerNUFFTPyNUFFT (0.23s of import, one fallback — autoarray · low-medium · supervised · normal

Deduplicate repos_sync.py's check/write pairs — pyautomind · medium · safe · low

diff --git a/dashboard.md b/dashboard.md index 2778aa1a..aefe2999 100644 --- a/dashboard.md +++ b/dashboard.md @@ -11,7 +11,7 @@ Every task the Mind is holding, on one page: what is in flight, what is parked, | [In flight](#in-flight) (`active/`) | 2 | | [Parked](#parked) (`parked.md`) | 3 | | [Planned](#planned) (`planned.md`) | 6 | -| [Backlog](#backlog) (`draft/`) | 146 | +| [Backlog](#backlog) (`draft/`) | 148 | ## Start here @@ -115,7 +115,13 @@ Every task the Mind is holding, on one page: what is in flight, what is parked, **Quick wins** (small enough, and safe enough to run unattended) -- _(none right now)_ +
📋 Defer the eager scipy.sparse import in derivative_util (~0.10 s of — libraries · small · safe · normal + +``` +/start_dev draft/maintenance/libraries/defer_scipy_sparse_import.md +``` + +
## In flight @@ -229,7 +235,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**146** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **23** of them belong to an epic and are listed only under [Epics](#epics) below. +**148** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **23** of them belong to an epic and are listed only under [Epics](#epics) below.
feature — 28 @@ -730,7 +736,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-research — 16 +research — 17
📋 Deep research: Can we speed up Delaunay in PyAutoArray? — autoarray · too-large · supervised · high @@ -804,6 +810,14 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
+
📋 Is Intel macOS a supported platform, and what is the — libraries · medium · supervised · normal + +``` +/start_dev draft/research/libraries/intel_macos_support_policy.md +``` + +
+
📋 Checkerboard PSF-mismatch residual diagnostic — research + document + ingest — pyautomemory · medium · supervised · normal ``` @@ -863,7 +877,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-maintenance — 18 +maintenance — 20
📋 smoke_install.sh's stale jax<0.7 pin — CI is on the right — ci · low · supervised · medium @@ -881,6 +895,14 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
+
📋 Defer the eager scipy.sparse import in derivative_util (~0.10 s of — libraries · small · safe · normal + +``` +/start_dev draft/maintenance/libraries/defer_scipy_sparse_import.md +``` + +
+
📋 Mirror drifted library config keys into the workspace configs — workspaces · small · supervised · normal ``` @@ -969,6 +991,14 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
+
📋 Remove pynufft + legacy TransformerNUFFTPyNUFFT — libraries · low-medium · supervised · normal + +``` +/start_dev draft/maintenance/libraries/remove_pynufft_legacy_transformer.md +``` + +
+
📋 dataset/imaging/jwst_lw is untracked because the gitignore was never extended for — autolens_profiling · small · supervised · low ``` @@ -1129,7 +1159,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-refactor — 6 +refactor — 5
📋 einstein_radius_jit_from: replace static init_guess with a JAX-native seed finder — autogalaxy · too-large · supervised · high @@ -1163,14 +1193,6 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-
📋 Remove pynufft + legacy TransformerNUFFTPyNUFFT (0.23s of import, one fallback — autoarray · low-medium · supervised · normal - -``` -/start_dev draft/refactor/autoarray/remove_pynufft_legacy_transformer.md -``` - -
-
📋 Deduplicate repos_sync.py's check/write pairs — pyautomind · medium · safe · low ``` diff --git a/draft/bug/autoarray/pynufft_scipy_pinv2_dev_extra.md b/draft/bug/autoarray/pynufft_scipy_pinv2_dev_extra.md index a1684d2c..fc805b2f 100644 --- a/draft/bug/autoarray/pynufft_scipy_pinv2_dev_extra.md +++ b/draft/bug/autoarray/pynufft_scipy_pinv2_dev_extra.md @@ -7,7 +7,7 @@ Repos: Difficulty: small Autonomy: supervised Priority: normal -Status: draft +Status: superseded ## Original request @@ -40,3 +40,20 @@ exists, or retire the legacy backend and its tests. Keep this separate from the - The PyNUFFT test either passes against the supported dependency set or is removed together with an explicit backend-retirement decision. - The standard optional-profile suite remains green. + +## Superseded 2026-08-22 + +Resolved by `draft/maintenance/libraries/remove_pynufft_legacy_transformer.md`, +which takes this prompt's third sanctioned remedy — "retire the legacy backend +and its tests". `pynufft` is gone from PyAutoArray's `optional` and `dev` +extras, so there is no longer a dev-extra install that can hit +`scipy.linalg.pinv2`. + +Confirmed on a clean Python 3.13 install (2026-08-22): with +`pynufft==2022.2.2` present, `hasattr(scipy.linalg, "pinv2")` is `False` under +SciPy 1.17.1 — the drift this prompt reported is real, and pre-existing rather +than a Python-floor regression as it said. pynufft 2022.2.2 also emits +`SyntaxWarning: "is" with 'str' literal` on 3.13, i.e. it is unmaintained +against the supported interpreter range. + +Close this out when the removal PRs merge. diff --git a/draft/maintenance/libraries/defer_scipy_sparse_import.md b/draft/maintenance/libraries/defer_scipy_sparse_import.md new file mode 100644 index 00000000..9976d3c9 --- /dev/null +++ b/draft/maintenance/libraries/defer_scipy_sparse_import.md @@ -0,0 +1,89 @@ +# Defer the eager scipy.sparse import in derivative_util (~0.10 s of import) + +Type: maintenance +Target: libraries +Repos: +- @PyAutoArray +Difficulty: small +Autonomy: safe +Priority: normal +Status: in-flight + +## Where this came from + +Found 2026-08-22 while measuring the pynufft removal +(`draft/maintenance/libraries/remove_pynufft_legacy_transformer.md`). That task +assumed removing pynufft would take ~0.23 s off `import autoarray`. It takes +~10 ms. The reason is the real target: + +`autoarray/operators/derivative_util.py:30` does + +```python +from scipy.sparse import csr_matrix +``` + +at module scope. `scipy.sparse` costs **0.106 s cumulative** and is imported on +every `import autoarray`, whether or not anything touches a derivative +operator. pynufft's apparent 0.19 s was ~95 % this same shared subtree — +removing pynufft did not remove it, because `derivative_util` pulls it in +anyway. + +## Evidence (Python 3.13, dev extras, median of 7 runs) + +| | `import autoarray` | +|---|---| +| main, pynufft installed | 369.8 ms | +| pynufft removed | 359.9 ms | + +`python -X importtime` on the pynufft-removed branch still shows +`scipy.sparse` at 0.106 s cumulative. + +## Task + +Defer the `csr_matrix` import into the functions that build the sparse +operators (the same pattern `transformer.py` already uses for `nufftax` via +`_load_nufftax()`). Check for other eager `scipy.sparse` importers before +assuming this is the only one — the win only lands if *no* module-scope import +of it survives on the `import autoarray` path. + +Note the precedent in `transformer.py`: the deferral must survive unpickling in +multiprocessing workers, so hang it off the call sites rather than `__init__` +alone. + +## Acceptance + +- `python -X importtime -c "import autoarray" | grep scipy.sparse` is empty. +- Median `import autoarray` drops by ~0.10 s against the same measurement + method above (record the before/after numbers in the PR). +- Full suite green; sparse-operator behaviour unchanged. + +## Correction + result (implemented 2026-08-22, PyAutoArray#477) + +**This prompt's own premise was wrong, in exactly the way the pynufft one +was.** Deferring `derivative_util.py:30` changed nothing: `scipy.sparse` was +never imported from there. Traced with a `sys.meta_path` hook: + +``` +autoarray/__init__.py:80 + -> inversion/mesh/mesh_geometry/delaunay.py:2 import scipy.spatial + -> scipy/spatial/__init__.py:111 from ._kdtree import * + -> scipy/spatial/_kdtree.py:4 from ._ckdtree import cKDTree +``` + +`scipy.spatial` (134 ms) pulls `scipy.sparse` (154 ms) transitively, so the +`csr_matrix` import was riding on a subtree already paid for. Deferring +`scipy.spatial` as well is what removes both — and is required to satisfy this +prompt's own acceptance criterion. + +The general lesson, now hit twice: **a module's `importtime` cumulative figure +is not its exclusive cost.** Attributing a saving to a dependency requires +checking who else pulls its subtree in. + +Result: `import autoarray` **464.4 ms -> 183.7 ms** (medians of 15 runs, +Python 3.13, dev extras) — a 281 ms saving, ~2.8x this prompt's ~0.10 s +estimate, because `scipy.spatial`'s own cost comes off too. Both greps are +empty. Suites green: autoarray 1179, autogalaxy 1103/1 skipped, +autolens 532/1 skipped. + +Note `delaunay.py` already had local `import scipy.spatial` at two of its three +use sites — this deferral had been started and left half-done. diff --git a/draft/maintenance/libraries/remove_pynufft_legacy_transformer.md b/draft/maintenance/libraries/remove_pynufft_legacy_transformer.md new file mode 100644 index 00000000..5f7a6252 --- /dev/null +++ b/draft/maintenance/libraries/remove_pynufft_legacy_transformer.md @@ -0,0 +1,138 @@ +# Remove pynufft + legacy TransformerNUFFTPyNUFFT + +Type: maintenance +Target: libraries +Repos: +- @PyAutoArray +- @PyAutoGalaxy +- @PyAutoLens +Difficulty: low-medium +Autonomy: supervised +Priority: normal +Status: shipped + +Re-homed 2026-08-22 from `draft/refactor/autoarray/`. The Brain Refactor Agent +refused it there (`SUSPECT-API-CHANGE`, effective autonomy `human-required`): +deleting a public class is not behaviour-preserving, so it cannot be a +`refactor/`. `maintenance/` ("dependency updates, hygiene, cleanup") is the +taxonomy fit for removing a dependency. + +## Original request (verbatim, 2026-08-19) + +> do we even use pynufft? maybe we should file a follow up to remove it I think +> its purely for one function, so make that filing remove it but also include +> this check of import time. + +## Corrections to the original filing (measured 2026-08-22) + +The 2026-08-19 filing had two factual errors, both found during implementation: + +1. **pynufft was never a base dependency.** It sat in the `optional` extra + (`PyAutoArray/pyproject.toml:67`) and the `dev` extra (line 77). A plain + `pip install autoarray` never had it, so `TransformerNUFFTPyNUFFT` already + raised `pynufft_exception()` for most users. +2. **The import-time saving is ~10 ms, not ~230 ms.** Measured as the median of + 7 runs on Python 3.13 with the dev extras: `import autoarray` goes + 369.8 ms → 359.9 ms. `pynufft`'s 0.19 s *cumulative* import is ~95 % + `scipy.sparse` (0.11 s), which `autoarray/operators/derivative_util.py:30` + pulls in eagerly for `csr_matrix` regardless of pynufft. Only ~10 modules and + ~10 ms are exclusive to pynufft. + +The removal is still worth doing — an unmaintained dependency, one dead class, +and a `dev` extra that is broken against SciPy >= 1.17 — but **not** on +import-time grounds. The real 0.10 s win is filed separately as +`draft/maintenance/libraries/defer_scipy_sparse_import.md`. + +## The Intel-macOS decision (settled 2026-08-22) + +The original filing flagged one blocking question: does removing the pynufft +fallback cost Intel-Mac users their interferometer transformer? Verified +against PyPI: + +- `jaxlib`'s **last Intel-macOS (x86_64) wheel is 0.4.38**, uploaded + 2024-12-17. Every release since (through 0.11.1) is `macosx_11_0_arm64` only. +- `jaxlib` has **never published an sdist**, for any version — so there is no + pip fallback that builds from source. +- PyAutoNerves' floor is `jax>=0.7.0` + (`PyAutoNerves/pyproject.toml:38`, markered + `sys_platform != "darwin" or platform_machine == "arm64"`). +- `nufftax` is pure-JAX; even its `xp=np` path calls `nufftax.nufft2d2`. So no + JAX means no `TransformerNUFFT` either. + +So Intel macOS keeps **`TransformerDFT` only** for interferometry — exact, pure +numpy, but O(N_vis x N_pix). Human decision (2026-08-22): accept this; record +it as a release-note line, not a blocker. The broader "is Intel macOS a +supported platform" question is filed as +`draft/research/libraries/intel_macos_support_policy.md`. + +## Work done (branches pushed 2026-08-22) + +All three on `claude/remove-pynufft-6uwt2z`: + +- **@PyAutoArray** — deleted `TransformerNUFFTPyNUFFT`, the `NUFFTPlaceholder` + / `NUFFT_cpu` module-scope try-import and `pynufft_exception()`; dropped the + re-exports from `__init__.py` and `type.py` and its arm of the `Transformer` + union; removed the three `test__nufft_pynufft__*` tests and the `"pynufft"` + arm of the nufftax-absent skip filter in `conftest.py`; dropped `pynufft` + from `optional` and the `pynufft==2022.2.2` pin from `dev`. Rewrote + `nufftax_exception()` and the `use_adjoint_scaling` docstrings, which cited + the deleted class. **1164 passed, 1 skipped.** +- **@PyAutoGalaxy** — dropped the `__init__.py:29` re-export and the `optional` + entry; updated installation docs, the feature overview and the live citation + surface (`files/citations.{md,tex,bib}`, `docs/index.md`) to cite `nufftax`. + **1103 passed, 1 skipped.** +- **@PyAutoLens** — same, minus the citation *addition* (nufftax was already + cited); the PyNUFFT entry was simply dropped. **532 passed, 1 skipped.** + +`paper/` in both downstream repos was deliberately left untouched — published +JOSS records of what the software used at time of publication, not live docs. + +## Remaining + +1. **Workspace tier** (not started, needs the workspace repos): + - `autolens_workspace_test/scripts/interferometer/nufft.py:211` — the one + executable use; drop or replace the PyNUFFT leg of the parity comparison. + - Four prose mentions in autolens_workspace (`start_here.py`, `using_jax.py`, + `simulator.py`, `linear_light_profiles/modeling.py`) describing it as a + "non-JAX fallback". + - Check `PyAutoHands/autohands/config/no_run.yaml` per the ship_library + reference. +2. **PR bodies need the `## API Changes` breaking entry** (release-notes + contract): `TransformerNUFFTPyNUFFT` removed; migration is `TransformerNUFFT` + (nufftax), or `TransformerDFT` where JAX is unavailable. +3. All three library PRs **must merge together** — the downstream re-exports + break at import the moment PyAutoArray's removal lands alone. +4. Closes the separate bug draft + `draft/bug/autoarray/pynufft_scipy_pinv2_dev_extra.md`: the + `pynufft==2022.2.2` dev pin calls `scipy.linalg.pinv2`, absent from SciPy + 1.17.1 (confirmed 2026-08-22 — `hasattr(scipy.linalg, "pinv2")` is `False`). + Retiring the backend was one of that prompt's three sanctioned remedies. + +## Shipped 2026-08-22 + +Library tier merged: @PyAutoArray#475, @PyAutoGalaxy#583, @PyAutoLens#709 — +all green on CI, including each repo's `unittest-nojax` job, which is the +standing evidence that the no-JAX path survives without the pynufft fallback. +Merged galaxy -> lens -> array so main was never red (dropping a re-export is +safe against an autoarray that still has the class; the reverse is not). + +Workspace tier raised: @autolens_workspace#497 (prose across `scripts/`, +`notebooks/`, `markdown/`) and @autolens_workspace_test#261 (the `nufft.py` +parity script rewritten around `TransformerDFT` as its sole reference; verified +by running it — all four tests pass, 3.0e-14 relative residual at 256x256). + +`PyAutoHands/autohands/config/no_run.yaml` checked: no nufft entries, nothing +hidden there. + +Two things found along the way that are NOT closed by this task: + +1. `use_adjoint_scaling` is now a **no-op on both remaining transformers** — it + was load-bearing only for the deleted class's pynufft-internal IFFT + normalisation. Kept, with docstrings rewritten, rather than widening this + removal. Worth a separate decision. +2. `autolens_workspace_test/scripts/interferometer/jax_likelihood/rectangular_sparse.py` + recorded the `apply_sparse_operator` / `TransformerNUFFT` incompatibility as + being caused by pynufft's kernel-deconvolved adjoint scale. That backend is + gone; whether the incompatibility still holds against the nufftax adjoint is + **unverified**. The dead attribution was removed without asserting a + replacement claim. diff --git a/draft/refactor/autoarray/remove_pynufft_legacy_transformer.md b/draft/refactor/autoarray/remove_pynufft_legacy_transformer.md deleted file mode 100644 index d9608ef3..00000000 --- a/draft/refactor/autoarray/remove_pynufft_legacy_transformer.md +++ /dev/null @@ -1,71 +0,0 @@ -# Remove pynufft + legacy TransformerNUFFTPyNUFFT (0.23s of import, one fallback class) - -Type: refactor -Target: autoarray -Repos: -- @PyAutoArray -Difficulty: low-medium -Autonomy: supervised -Priority: normal -Status: draft - -## Original request (verbatim, 2026-08-19) - -> do we even use pynufft? maybe we should file a follow up to remove it I think -> its purely for one function, so make that filing remove it but also include -> this check of import time. - -## Import-time evidence (from the lazy-heavy-imports profiling, #1505) - -After the 2026-08-19 import deferrals, `import autolens` is 1.09–1.17s and the -largest remaining third-party chunk is **scipy at 0.31s self-time** — about a -third of which enters via pynufft: `import pynufft` costs **~0.23s** (its -`pynufft.nufft` pulls `scipy.sparse` at ~0.11s), and it is imported **eagerly** -at `import autoarray` via the module-level try/except at -`autoarray/operators/transformer.py:11-14` -(`from pynufft.linalg.nufft_cpu import NUFFT_cpu`, guarded only against -absence). Removing it takes bare import toward the ~0.8-0.9s floor -(matplotlib being the other deliberate remainder). - -## Current footprint (verified 2026-08-19) - -- **Dependency**: `pynufft` is a BASE dep in `PyAutoArray/pyproject.toml:67` - (plus dev pin `pynufft==2022.2.2` at line 77). -- **Sole code use**: the legacy `TransformerNUFFTPyNUFFT` class - (`transformer.py:270`, subclasses `pynufft.NUFFT_cpu`) — the non-JAX NUFFT - fallback superseded by the nufftax-backed `TransformerNUFFT`. Re-exported via - `autoarray/__init__.py`, `autoarray/type.py`, and the autogalaxy/autolens - `__init__` chains. -- **Workspace references**: prose-only mentions in autolens_workspace - (start_here.py, using_jax.py, simulator.py, linear_light_profiles/modeling.py - all describe it as a "non-JAX fallback"); ONE executable use — - `autolens_workspace_test/scripts/interferometer/nufft.py:211` (transformer - parity script). Check `PyAutoHands/autohands/config/no_run.yaml` for hidden - risk per ship_library reference. - -## Task - -Remove the dependency and the legacy class: - -1. Delete `TransformerNUFFTPyNUFFT`, the `NUFFT_cpu`/`NUFFTPlaceholder` - try-import, and `pynufft_exception()` from `transformer.py`; drop the - re-exports (`autoarray/__init__.py`, `type.py`) and the pyproject entries - (base + dev). Sweep autogalaxy/autolens `__init__` re-exports and any - `Transformer` type unions. -2. Migrate `autolens_workspace_test/scripts/interferometer/nufft.py` (drop or - replace the PyNUFFT leg of the parity comparison) via /start_workspace. -3. Update the four autolens_workspace prose mentions (drop "legacy - pynufft-backed fallback" sentences) — prose tier per WORKFLOW.md. -4. This is a REMOVAL: the PR body needs the `## API Changes` breaking entries - (release-notes contract) — `TransformerNUFFTPyNUFFT` removed, migration = - use `TransformerNUFFT` (nufftax; also numpy-capable via `xp=np`). -5. Re-run the import-time check afterwards: `python -X importtime -c "import - autolens" | grep pynufft` must be empty, and record the new total - (expect ~0.15-0.25s off scipy's share). - -Decision the plan must confirm before deleting: `TransformerNUFFT` (nufftax) -covers the numpy/no-JAX path (`visibilities_from(..., xp=np)`), so no user -capability is lost — verify nufftax is a hard-enough dep on all supported -platforms (Intel-Mac marker excludes jax; does the numpy path of nufftax work -there, or does interferometer analysis on Intel Macs lose its transformer?). -If Intel Macs need a fallback, deprecate instead of delete. diff --git a/draft/research/libraries/intel_macos_support_policy.md b/draft/research/libraries/intel_macos_support_policy.md new file mode 100644 index 00000000..c73d5d40 --- /dev/null +++ b/draft/research/libraries/intel_macos_support_policy.md @@ -0,0 +1,72 @@ +# Is Intel macOS a supported platform, and what is the numpy-only contract? + +Type: research +Target: libraries +Repos: +- @PyAutoNerves +- @PyAutoArray +- @PyAutoGalaxy +- @PyAutoLens +Difficulty: medium +Autonomy: supervised +Priority: normal +Status: draft + +## Original request (verbatim, 2026-08-22) + +> are you sure Intel mac can't use any JAX event CPU this seems like a bigger +> issue in general? + +Asked while reviewing the pynufft removal +(`draft/maintenance/libraries/remove_pynufft_legacy_transformer.md`), whose +Intel-Mac caveat was accepted as a release-note line on the understanding that +the general platform question gets its own task. + +## The finding that prompted it (verified against PyPI, 2026-08-22) + +It is not a CPU-vs-GPU limitation — Google dropped x86_64 macOS builds outright: + +- `jaxlib`'s last macOS x86_64 wheel is **0.4.38**, uploaded **2024-12-17** + (`jaxlib-0.4.38-cp31{0,1,2,3}-macosx_10_14_x86_64.whl`). Every release since, + through 0.11.1, ships `macosx_11_0_arm64` only. +- `jaxlib` has **never shipped an sdist**, for any version — so `pip` has no + build-from-source fallback. Building it needs Bazel; conda-forge is the only + other plausible channel and was not reachable from the session that checked. +- PyAutoNerves therefore markers jax/jaxlib/jaxnnls as + `sys_platform != "darwin" or platform_machine == "arm64"` + (`PyAutoNerves/pyproject.toml:38-40`) and states the intent as "NumPy-only + path instead of failing at install". + +## Why this is bigger than one transformer + +The stated contract is that Intel macOS silently gets a NumPy-only path. What +that actually covers has never been audited. The pynufft removal surfaced one +concrete hole: `nufftax` is pure-JAX (even its `xp=np` path calls +`nufftax.nufft2d2`), so with JAX absent **both** NUFFT transformers are +unavailable and interferometer analysis falls back to `TransformerDFT` +(O(N_vis x N_pix) — correct, but impractical for real ALMA/SMA data). + +If one "NumPy-only" path is really a JAX path in disguise, others may be too. + +## Task + +1. **Audit the claim.** Enumerate what actually works on a JAX-less install: + which code paths genuinely branch on `xp`, and which import JAX + unconditionally somewhere down the stack (`jax_wrapper`, `nufftax`, + `jaxnnls`, `tfp-nightly`, the Matern-kernel regularization path). A CI job + or a container with JAX uninstalled is the cheap way to get the real answer. +2. **Decide the policy explicitly** and write it down once, somewhere users + read: is Intel macOS supported, best-effort, or unsupported? "Silently + degrades" is the current de-facto answer and it is the worst one — a user + gets an install that resolves and then fails at analysis time. +3. **Make the failure legible** whichever way the policy lands: if supported, + the numpy paths need to be real and tested; if not, say so at install or + import rather than at the first transformer construction. +4. Feed the answer back to the pynufft removal's release note, which currently + asserts Intel-Mac users keep `TransformerDFT`. + +## Acceptance + +- A written, discoverable platform-support statement covering Intel macOS. +- An evidence-backed list of what does and does not work without JAX. +- No silent install-then-fail path left for the unsupported case.