Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 33 additions & 18 deletions draft/docs/autolens/docs_three_regime_restructure.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Repos:
Difficulty: medium
Autonomy: supervised
Priority: high
Status: draft
Status: in progress — all planned edits landed 2026-07-26 (branch claude/pyautolens-doc-reorganization-w6a1l5); pending RTD build confirmation on merge
Parent: draft/docs/autolens/split_lensing_regimes.md

Restructure the PyAutoLens Sphinx/RTD documentation (`PyAutoLens/docs/`) around
Expand All @@ -19,7 +19,31 @@ example links, API pointers and modelling philosophy.

- `docs/overview/overview_2_new_user_guide.md`: the multi_galaxy rung and the
four-step ladder routing with the analysis-split prose landed (PyAutoLens
`c086863` + review fixes). The remaining items below are still open.
`c086863` + review fixes). `overview_3_features.md`: group section renamed
"Multi-Galaxy Lenses, Groups and Clusters" (PyAutoLens#653).

## Landed (2026-07-26, this task branch)

- `docs/overview/overview_1_start_here.md`: the three regime enumerations
now spell out the four-rung ladder (galaxy-scale, multi-galaxy,
group-scale, cluster-scale).
- `docs/api/mass.rst`: regime note on the Total section — untruncated
Isothermal/PowerLaw for galaxy/multi_galaxy, tidally truncated dPIE
members for group/cluster (sigma_lt vs b0 parameterizations named).
- `docs/api/point.rst`: intro states point sources are the standard source
strategy at cluster scale (per-source redshifts, multi-plane) with the
workspace `cluster` package pointer.
- `docs/api/galaxy.rst`: new "Galaxy Catalogues (CSV)" section documenting
`galaxy_table_from_csv` / `galaxies_from_csv_tables` /
`galaxy_af_models_from_csv_tables` (previously absent from the API docs)
with the scaling-tier regime framing.
- `docs/general/model_cookbook.md`: new "Multi Galaxy, Group and Cluster
Models" section — per-deflector loop recipe, shared-prior scaling-relation
tie recipe, cluster point-source framing, links to the four regime
notebooks.

This completes the planned scope: the API reference stays single-copy with
per-surface regime notes (as specified), rather than a three-way fork.

## Changes

Expand All @@ -33,22 +57,13 @@ example links, API pointers and modelling philosophy.
point-source/multi-image-position workflow by default.
Include the taxonomy sentence: all groups and clusters are multi-galaxy
systems, but not vice versa.
- `docs/overview/overview_1_start_here.md` + `overview_3_features.md`: add the
regime split where scales are enumerated; link the three start_here Colab
notebooks.
- `docs/api/`: ensure the mass/galaxy/point API pages surface the
regime-relevant surfaces where users will look for them — dPIE profiles,
`al.sr` scaling relations, `galaxy_table_from_csv` /
`galaxies_from_csv_tables` / `galaxy_models_from_csv` CSV APIs, and the
point-source/`PointSolver` machinery — grouped or cross-referenced by
regime (a short "which regime uses this" note per surface is enough; do not
fork the API reference into three copies).
- `docs/general/model_cookbook.md`: add multi_galaxy, group (both
with/without-halo compositions) and cluster model recipes.
- Scientific grounding: each regime section cites 2–3 flagship
systems/surveys from the parent plan's literature research (e.g. the
multi_galaxy flagship, SL2S/CASSOWARY groups, HFF/A2744 clusters) so the
docs point at real, recognisable science.
- ~~`docs/overview/overview_1_start_here.md` + `overview_3_features.md`~~
LANDED (see above).
- ~~`docs/api/`~~ LANDED (see above; per-surface regime notes, no fork).
- ~~`docs/general/model_cookbook.md`~~ LANDED (see above).
- ~~Scientific grounding~~ LANDED 2026-07-26: the New User Guide ladder now
cites J1011+0143/B1608+656 (multi_galaxy), CSWA 19/SL2S (group) and
HFF/A2744 (cluster).

## Ordering

Expand Down
33 changes: 27 additions & 6 deletions draft/docs/autolens/multi_galaxy_package.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Repos:
Difficulty: large
Autonomy: supervised
Priority: high
Status: in progress — core landed on branch claude/pyautolens-doc-reorganization-w6a1l5 (2026-07-25)
Status: in progress — core landed 2026-07-25; features/fit/jax legs landed 2026-07-26 (branch claude/pyautolens-doc-reorganization-w6a1l5); only the real-data swap-in remains
Parent: draft/docs/autolens/split_lensing_regimes.md

## Landed (2026-07-25, this task branch)
Expand All @@ -26,15 +26,36 @@ Parent: draft/docs/autolens/split_lensing_regimes.md
- PyAutoLens docs: New User Guide four-rung ladder + multi_galaxy links
(full RTD restructure remains with docs_three_regime_restructure.md).

## Landed (2026-07-26, this task branch)

- `features/scaling_galaxies/` (simulator + modeling): five faint galaxies
4–7" out on an untruncated-`IsothermalSph` relation
(`einstein_radius = einstein_radius_ref * (L/L_ref)**0.5`, truth ref
0.15"), shared-prior tier costing one free parameter, the "load of
galaxies far from the lens / not a standard ingredient" framing, and the
group/cluster truncation contrast. Smoke-registered and validated green.
- `fit.py`: N-deflector fit anatomy — per-galaxy deflection fields compared
(mean-|deflection| co-dominance ratio 0.82), truth-composition
`FitImaging` (full-res truth LL +27220), pointing at `imaging/fit.py` for
the step-by-step API anatomy. Smoke-registered and validated green.
Scope decision: a `likelihood_function.py` mirror was deliberately NOT
added — the likelihood machinery is regime-independent and fully
documented by `imaging/` + the group package; `fit.py` covers the one
regime-specific piece (summed deflection fields).
- autolens_workspace_test `multi_galaxy/jax_likelihood/lp.py`: batched
`fitness._vmap` literal + `jit(fit_from)` NumPy-parity round-trip over a
two-co-dominant-deflector model; smoke-registered, runner-validated
(15s).

## Remaining

- features/ scripts (extra_galaxies, scaling_galaxies with untruncated
isothermals, pixelization) — currently README cross-links only.
- Swap start_here to the REAL SDSS J1011+0143 HST data (F555W/F814W via
MAST) once frames are prepared; the simulated look-alike is the interim.
- likelihood_function.py / fit.py mirrors of the group package equivalents.
- autolens_workspace_test multi_galaxy/jax_likelihood/ variant (only
model_fit + the relocated composition test landed).
BLOCKED from cloud sessions (2026-07-26): MAST is unreachable through the
session proxy (`Tunnel connection failed: 403`) — needs a
local/unrestricted-network session to download + prepare the frames.
- Extra-galaxies / pixelization feature variants remain README cross-links
(the group/imaging feature scripts apply verbatim with the lens loop).

Create the new `scripts/multi_galaxy/` package in @autolens_workspace — the first
of the three above-galaxy-scale regimes (see the parent plan for the full design
Expand Down
12 changes: 11 additions & 1 deletion draft/docs/autolens/split_lensing_regimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,13 +307,23 @@ regenerate the dataset alongside the convention change.
1. `draft/docs/autolens/multi_galaxy_package.md` — new
`autolens_workspace/scripts/multi_galaxy/` package (+ workspace_test
mirror). Unblocks everything user-facing; do first.
CORE SHIPPED 2026-07-25; features/scaling_galaxies, fit.py and the
workspace_test jax_likelihood variant 2026-07-26. Remaining: real
J1011+0143 data swap-in (MAST unreachable from cloud sessions — needs a
local/unrestricted-network session).
2. `draft/docs/workspaces/group_halo_explicit_choice.md` — group start_here
gains all three tiers; new `features/group_halo/` halo-choice tutorial.
CORE SHIPPED: halo tutorial 2026-07-25; three-tier start_here (with an
image-derived scaling_galaxies.csv for the real Euclid dataset)
2026-07-26. Remaining: modeling/simulator halo-narrative threading.
3. `draft/docs/workspaces/cluster_regime_narrative.md` — cluster narrative
alignment + `features/extended_source/` follow-up example + conventions
cross-check.
4. `draft/docs/autolens/docs_three_regime_restructure.md` — PyAutoLens RTD
docs ladder rewrite (after 1, so links resolve).
docs ladder rewrite (after 1, so links resolve). SHIPPED 2026-07-26:
New User Guide ladder + flagships, overview_1 enumerations, api
mass/point/galaxy regime notes + CSV-catalogue section, model_cookbook
regime recipes.
5. `draft/docs/workspaces/galaxy_scale_scaling_extra_features.md` —
imaging/interferometer/point_source extra_galaxies + scaling_galaxies
feature parity with regime caveats.
Expand Down
23 changes: 19 additions & 4 deletions draft/docs/workspaces/group_halo_explicit_choice.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Repos:
Difficulty: large
Autonomy: supervised
Priority: high
Status: in progress — signature tutorial landed on branch claude/pyautolens-doc-reorganization-w6a1l5 (2026-07-25)
Status: in progress — signature tutorial landed 2026-07-25; three-tier start_here landed 2026-07-26 (branch claude/pyautolens-doc-reorganization-w6a1l5)
Parent: draft/docs/autolens/split_lensing_regimes.md

## Landed (2026-07-25, this task branch)
Expand All @@ -22,11 +22,26 @@ Parent: draft/docs/autolens/split_lensing_regimes.md
features README + regime-ladder README edits landed with the multi_galaxy
package commit.

## Landed (2026-07-26, this task branch)

- `group/start_here.py` default model is now three-tier: main (2 galaxies —
the central galaxy plus the bright companion 0.36" away, promoted from
the extras catalogue after image inspection confirmed a distinct peak;
each MGE + Isothermal), extra (1, from `extra_galaxies_centres.json` —
MGE with a ±0.1" uniform light-centre prior + mass-centre-fixed bounded
`IsothermalSph`), scaling (5, mass-only untruncated `IsothermalSph` on a
shared `einstein_radius_ref` prior via
`einstein_radius_ref * luminosity_ratio**0.5`; the truncated-dPIE variant
stays pointed at `features/group_halo`). Scaling-galaxy
centres/luminosities for the real Euclid dataset were derived from the
image itself (gaussian-smoothed peak detection outside the main pair,
1"-aperture photometry, normalized to the brightest) and committed as
`dataset/group/.../scaling_galaxies.csv` with provenance documented in
the script prose. 24-free-parameter model validated end-to-end under
test mode.

## Remaining

- `start_here.py` default model gains the extra_galaxies + scaling_galaxies
tiers (currently main_galaxies only) — needs scaling-galaxy
centres/luminosities prepared for the real Euclid dataset.
- `modeling.py`/`simulator.py` halo-narrative threading; CSWA 19 as the
possible future real-data flagship for this feature (public HST +
published PyAutoLens model, arXiv:2504.11445).
Expand Down