feat: WFC3 (UVIS + IR) reduction support (phase 2)

[Jammy2211]

Overview

Phase 2: extend PyAutoReduce to HST/WFC3 (UVIS + IR) — the first real test of the instrument-adapter boundary built in phase 1 (#2, PR #3). Autonomous run (--auto, effective level supervised): plan recorded here, mechanical stretches proceed, judgment gates arrive as batched questions, run parks at ship sign-off.

Routing notes: (a) Brain Feature Agent heuristic again suggested research re-home; overridden — the WFC3 design delta is scoped in docs/design/roadmap.md and phase 1 established the architecture. (b) Work stacks on the unmerged feature/hst-acs-phase1 (PR #3): branch feature/wfc3-reduction from it; the PR will target main after #3 merges (or be re-based accordingly).

Plan

• Two new instrument adapters, zero detector names outside instruments/: wfc3_uvis (ACS-like: _flc CTE-corrected, iref references, native 0.0396″/pix) and wfc3_ir (_flt, no CTE correction — up-the-ramp readout rejects CRs per read; native ~0.128″/pix; IR saturation/DQ semantics).
• Whatever generalization the adapters force on phase-1 stage code lands with tests, and the ACS path must not change (53-test regression suite stays green).
• New docs/design/wfc3.md: per-stage deltas vs ACS (products, references key, scale choices, IR sky/CR behaviour); audit coverage against the ajshajib/hst-lens IR+UVIS notebooks as a checklist.
• Unit tests for adapter registry entries, per-channel drizzle-kwargs assembly, UVIS-vs-IR branch differences.
• Integration: UVIS — reduce SDSS J0252+0039 F390W and check against the published Bayer et al. (arXiv:1803.05952) reduction (0.0396″/pix, pixfrac 1.0, σ_sky ≈ 0.002 e-/s): a literature anchor. IR — discover an F160W (or similar) lens dataset at J0252 or a nearby SLACS system via MAST; validate internally (noise closure vs empirical sky, WHT uniformity, PSF diagnostics). If no suitable IR dataset is discoverable, that leg parks as a batched question.

Detailed implementation plan

Affected Repositories

• PyAutoReduce (primary, only)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoReduce	feature/hst-acs-phase1 (pushed, PR #3 awaiting merge)	clean

Suggested branch: feature/wfc3-reduction stacked on feature/hst-acs-phase1.

Implementation Steps

1. instruments/adapter.py: add recommended_final_scale field (documentation + adapter-owned default recommendation; TargetSpec.final_scale stays the user dial).
2. instruments/wfc3_uvis.py + instruments/wfc3_ir.py: registry entries with MAST names ("WFC3/UVIS", "WFC3/IR"), product suffixes (FLC / FLT), iref + references/hst/wfc3 CRDS subpath, saturation levels, per-channel default drizzle kwargs.
3. docs/design/wfc3.md + roadmap tick.
4. test_autoreduce/test_wfc3.py: registry, kwargs assembly, CTE flags, scale ratios, suffix differences; ACS suite untouched.
5. scripts/reduce_j0252_uvis.py: F390W end-to-end + comparison against Bayer's published parameters.
6. IR target discovery + scripts/reduce_<target>_ir.py (internal validation), or parked question.

Key Files

• docs/design/roadmap.md — the WFC3 section being implemented
• autoreduce/instruments/acs_wfc.py — the adapter pattern to follow

Original Prompt

Click to expand starting prompt

See PyAutoMind/issued/wfc3_reduction.md (filed from feature/pyautoreduce/wfc3_reduction.md).

[Jammy2211]

Parked at ship sign-off (supervised --auto — checkpoint-and-continue).

Phase 2 is complete and validated on feature/wfc3-reduction (4 local commits, unpushed, stacked on feature/hst-acs-phase1 / PR #3). Both channels reduced real MAST data of SDSS J0252+0039 end-to-end; the UVIS leg matches the published Bayer et al. reduction parameters; two loud-failure guards demonstrated themselves on the IR leg and their dial resolutions are documented in docs/design/wfc3.md. Review found one HIGH (saturation-cap physics — fixed and IR re-validated: 71 ePSF stars now correctly including bright ones) and two LOWs (fixed). 62/62 tests.

Draft PR body (posted verbatim on approval):

---

Implements WFC3 (UVIS + IR) support per #4, stacked on feature/hst-acs-phase1 (PR #3). Autonomous run (--auto supervised), parked at ship sign-off.

What this adds

• instruments/wfc3_uvis.py — the ACS-like path: _flc CTE-corrected, iref/references/hst/wfc3, native 0.0396″/pix (also the recommended output scale), 63 ke- full well.
• instruments/wfc3_ir.py — the different path: _flt (no CTE correction; up-the-ramp CR rejection), native 0.128″/pix with recommended 0.065″ output, 78 ke- full well.
• InstrumentAdapter.recommended_final_scale — adapter-owned sampling recommendation; TargetSpec.final_scale stays the user dial (ACS default unchanged).
• docs/design/wfc3.md — per-stage deltas vs ACS, the hst-lens notebook coverage audit, and two integration findings (below); roadmap section ticked to in-progress.
• scripts/reduce_j0252_wfc3.py — both channels on SDSS J0252+0039.
• Cross-channel fix in phase-1 code: the PSF star saturation cap divided full well by mosaic-total EXPTIME (~8× too strict on multi-exposure stacks, physically wrong for IR ramps); now uses the longest single exposure.

Integration results (real MAST data)

	UVIS F390W (Bayer dials: 0.0396″, pixfrac 1.0)	IR F160W (0.065″, pixfrac 1.0, 14″ cutout)
exposures	16 (deep program)	8 (4794s, prog 11202)
units	e-/s ✓	e-/s ✓
WHT uniformity	0.107 ✓	0.129 ✓
R applied	1.500 (= shift-and-add, matches Bayer config)	2.370 (fine-grid s≈0.51, honestly reported)
sky RMS	0.00226 e-/s vs published ~0.002 (1.13×, within quoting precision)	0.00535 e-/s
ePSF	tier 1, 35 stars	tier 1, 71 stars (post star-cap fix)

Two loud-failure demonstrations (the guards working as designed, both documented in wfc3.md): IR at pixfrac 0.8 left zero-weight speckle → guard refused to package; and the oversized 18.3″ cutout clipped a detector-defect blob 8.5″ off-target → matched to the ACS 14″ sky footprint (215 px at 0.065″).

API Changes

New adapters wfc3_uvis, wfc3_ir; new InstrumentAdapter.recommended_final_scale field (defaulted — existing adapter construction unaffected). No other PyAuto* repo touched.

Scripts Changed

New scripts/reduce_j0252_wfc3.py (UVIS anchor validation + IR discovery/validation). Design docs updated.

Gate legs (supervised — parked at sign-off)

1. Tests: 62/62 (53 phase-1 regression + 9 WFC3). Downstream suites n/a (no dependents).
2. Smoke: n/a — no workspace surface consumes autoreduce (stated per contract).
3. Review: finder+verify on the phase-2 delta → 1 HIGH (saturation-cap physics, fixed, IR re-validated end-to-end), 2 LOW (stale doc path, HAP 'detection' pseudo-filter — both fixed). Clean on constants, coordinate math, adapter boundary, and test/network conventions.
4. Heart: YELLOW — same pre-existing organism reason set as PR feat: HST/ACS reduction pipeline (phase 1) #3 (workspace validation 5 fails, 58 stale parked scripts, assistant version skew, install verification, no release validation, 9d-old Mind PR); none related to this repo. Parked for acknowledgement.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3

---

Questions (batched):

1. Ship? Push feature/wfc3-reduction and open the PR? Requires your Heart-YELLOW acknowledgement of the reason set above (same set as PR feat: HST/ACS reduction pipeline (phase 1) #3). Base branch fork: if you merge PR feat: HST/ACS reduction pipeline (phase 1) #3 first, this PR targets main cleanly (recommended); otherwise it opens stacked against feature/hst-acs-phase1.
2. UVIS ePSF refresh: the star-cap fix landed after the UVIS run (its exposures were evicted — re-running means re-downloading ~9.5 GB). Re-run UVIS now for the improved bright-star ePSF, or accept the 35-star ePSF (validation numbers unaffected) and refresh on the next natural UVIS reduction? Recommendation: accept — the cap fix only widens star selection.
3. Roadmap next after merge: JWST/NIRCam adapter, per-exposure _flt frame products, or the tier-2 PSF back-end (focus-diverse ePSF grids — now motivated by two star-rich fields not needing it, but SLACS snapshots will)? No work starts without your pick.

[Jammy2211]

Shipped at PR-open: PR #5 targets main (PR #3 merged first per your answer, branch rebased cleanly — 4 commits, 62/62 tests re-verified post-rebase). UVIS ePSF accepted as-is per your answer 2 (the star-cap fix only widens selection; refresh rides the next natural UVIS reduction). Roadmap pick (JWST / per-exposure frames / tier-2 PSF) remains open — no phase 3 starts without it. Merge is yours.

[Jammy2211]

Phase 2 merged via PR #5 (squash). UVIS ePSF refresh rides the next natural UVIS reduction (maintainer decision); tier-2 PSF and further WFC3 work tracked on the roadmap.