feat: JWST/NIRCam via the COSMOS-Web ring, four bands (phase 3) - #7
Merged
Conversation
…MOS-Web ring integration Adapter fields observatory/crds_server_url/combine_backend/ mast_obs_collection; nircam_sw+nircam_lw with filter->channel routing; drizzle/jwst_combine.py (calwebb_image3, dials mapped to resample, _i2d normalized to sci/wht/err contract); noise = R x propagated ERR with sky consistency check; CRDS env adapter-owned (jwst-crds, CRDS_PATH direct); NaN-masked star finding; no full-well cut in surface-brightness units. 75 tests; docs/design/jwst.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
ShOpt/PSFEx/PIFF (COSMOS-Web WL practice), Zhuang & Shen NIRCam PSF characterization (spatial variation ~5% SW -> 0.6% LW RMS; PSFEx > photutils; empirical > STPSF for decomposition). Adopted tiering: LW single ePSF (tier 1, shipped); SW/WL-grade spatially-varying PSFEx or ShOpt (tier 2, follow-up); STPSF explicit 2b fallback; STARRED/PSFr tier 3 unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…ault) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…/DQ holes) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…rovenance, proposal filter image3 OOM'd the machine (took the session down): only exposures whose detector footprint covers the target now enter combination, and image3 runs with on-disk models. data_units read from BUNIT (jwst provenance has no drizzle_kwargs — the cross-backend KeyError that failed F444W after an otherwise-clean run). Ring script pinned to proposal 1727 so parity stays COSMOS-Web-pure (5893 exposures were creeping in). 77 tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…matters Deep resampled stacks routinely carry isolated fully-rejected pixels (F444W tripped the strict guard on exactly one). Policy: bad noise pixels -> MASKED_NOISE_VALUE (1e8, the legacy noise-scaling convention) with data zeroed, recorded in provenance; hard failure preserved for >0.5% of the cutout or any bad pixel within 1.5" of the lens. cutout.py split into make_cutout/write_fits for the post-processing seam. 81 tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…eproduction (maintainer guidance) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…ructured-defect loudness, masked-pixel parity stats, path resolve Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
…atches blocks first) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements JWST/NIRCam support per #6, rebased onto
mainafter PR #5 merged. Autonomous run (--autosupervised); shipped at PR-open on maintainer sign-off.What this adds
InstrumentAdapter.combine_backendroutes stage 3 toastrodrizzle(HST) orjwst_image3(calwebb_image3); CRDS server/env shape (observatory,crds_server_url,mast_obs_collection) is adapter-owned.instruments/nircam.py—nircam_sw/nircam_lw+ filter→channel routing; recommended output scales 0.03″/0.06″ (COSMOS-Web convention).drizzle/jwst_combine.py— calwebb_image3 with the lensing dials mapped to resample;save_results=True,in_memory=False(image3 OOM'd this machine otherwise);_i2dnormalized to standalone sci/wht/err so downstream stages stay backend-agnostic.noise/jwst_rms.py— read propagated ERR × Casertano R + blank-sky consistency check (the roadmap's "read, don't construct").acquire/footprint.py— detector-footprint filter (pixel-space containment, immune to RA-wrap/cos-dec box pitfalls): only exposures covering the target enter combination — on COSMOS-Web SW visits this discarded 52 of 64 exposures.noise/rms.py+package/cutout.pysplit): isolated dead/rejected pixels → masked-by-noise (1e8, the legacy convention), recorded in provenance; loud failure preserved for structured clusters, >0.5% of the cutout, or anything within 1.5″ of the lens.docs/design/jwst.md): ShOpt/PSFEx/PIFF (COSMOS-Web WL) + Zhuang & Shen (AGN) ⇒ LW single ePSF adequate (~0.6% RMS spatial variation), SW/WL-grade needs spatially-varying PSFEx/ShOpt (tier-2 follow-up), STPSF an explicit flagged fallback only.Four-band COSMOS-Web ring validation (vs the assistant demo products)
Acceptance bar per maintainer guidance: close + internally consistent, not reproduction — the demo descends from the bespoke COSMOS-Web team pipeline.
Internal closures pass everywhere (sky vs ERR floor within ~12%; ≤1 masked pixel/band; units MJy/sr recorded). Parity is order-unity in all bands with per-band scatter in both directions — consistent with the team pipeline's per-band bespoke steps, and inside the agreed bar. Cutout WHT uniformity 0.25–0.31 where measured (sparse COSMOS-Web dithers; reported per run).
API Changes
New adapters + adapter fields (all defaulted — HST adapters unchanged, verified by regression tests);
autoreduce.acquire.footprint;noise.jwst_rms;noise.rms.mask_isolated_bad_pixels;package.cutout.make_cutout/write_fits. No other PyAuto* repo touched.Scripts Changed
New
scripts/reduce_cosmos_web_ring.py(four bands + parity). Design docs:jwst.md(deltas, PSF tiering, parity interpretation), roadmap tick.Environment note
~/venv/PyAutogainedjwst2.0.1 (1.14 wheels crash under numpy 2) and scipy moved 1.14.0 → 1.17.1 (inside autoarray's declared cap); astropy/numpy/jax unchanged and autolens verified working.Gate legs (supervised — parked at sign-off)
🤖 Generated with Claude Code
https://claude.ai/code/session_01WNhuuvZLgZjHvWBGdhUst3