From 65a2d97e679d6daf92cec7d79107046c2474516b Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 12:56:25 +0000 Subject: [PATCH 1/7] prompt: re-scope the profiling compile-time task into a three-phase arc The 2026-07-14 prompt asked the Profiling Agent to measure JAX compile/eval time of the release-validation heavy scripts so we could speed them up. Both halves of that framing are spent, so it is re-scoped rather than started as written. The speed-up shipped: the #71/#74/#77 arc landed the persistent compilation cache (117.0s -> 2.3s CPU MGE vag; 5517.8s -> 937.1s A100 end-to-end) and --xla_gpu_autotune_level=0 (17x on the pathological cold probe, -40% on the full fit, bit-identical likelihoods), and ruled out source jit-boundaries by evidence. The prompt's candidate root causes are resolved with them. What survives is regression surveillance. Both wins are settings, so a config drift or an XLA_FLAGS clobber puts the 70-minute worst case back silently -- exactly the PyAutoNerves#127 failure that went undetected for two months. Nothing watches for that: profiling/AGENTS.md still lists compile-time profiling under Future modes, and _profiling.py has no occurrence of "compile". The release-validation-scripts leg is dropped. Script-suite cost is the hygiene conductor's perf mode by both agents' documented boundaries, having been moved out of profiling's staged future modes once already. Three blockers, measured against the 93 committed probe records: - coverage is 3-4 of the 24 runtime CELLS grid cells, hst only, with no interferometer, datacube, jwst or ao row - warm rows are not machine-identifiable -- warmness lives only in a free-text tag (~40 ad-hoc values), and cache_dir is non-empty on cold rows too - compile rows have no pin, so there is no baseline to regress against Split accordingly into campaign coverage (Brain), warm baselines + dashboard (workspace + Brain), and drift triage (Brain). The workspace-side jax_compile_time_profiling.md is absorbed into phase 2, since the dashboard and the agent reading it are two halves of one thing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- .../jax_compile_time_profiling.md | 9 ++ .../compile_axis_campaign_coverage.md | 81 ++++++++++ .../profiling/compile_axis_triage_drift.md | 62 ++++++++ .../compile_warm_baseline_dashboard.md | 102 ++++++++++++ .../profiling_agent_jax_compile_time_scope.md | 148 +++++++++++++++++- 5 files changed, 400 insertions(+), 2 deletions(-) create mode 100644 draft/feature/profiling/compile_axis_campaign_coverage.md create mode 100644 draft/feature/profiling/compile_axis_triage_drift.md create mode 100644 draft/feature/profiling/compile_warm_baseline_dashboard.md diff --git a/draft/feature/autolens_profiling/jax_compile_time_profiling.md b/draft/feature/autolens_profiling/jax_compile_time_profiling.md index 6485ddc4..e403733b 100644 --- a/draft/feature/autolens_profiling/jax_compile_time_profiling.md +++ b/draft/feature/autolens_profiling/jax_compile_time_profiling.md @@ -1,5 +1,14 @@ # Profile and speed up JAX likelihood-function compile times (all use cases) +> **ABSORBED 2026-08-10 — do not start from this file.** The surviving scope (the +> recurring cell-grid compile dashboard, per the 2026-07-28 note below) is now the +> workspace leg of +> `draft/feature/profiling/compile_warm_baseline_dashboard.md`, phase 2 of the +> compile-axis arc `draft/feature/profiling/profiling_agent_jax_compile_time_scope.md`. +> Merged there because the dashboard and the Profiling Agent that reads it are two +> halves of one thing, and building them from opposite ends risks incompatible +> schemas. This file is kept for its measurements and its overturned-claim trail. + > **Status note (2026-07-28):** largely superseded. The compile-time arc > (#71→#77) shipped cache-by-default + autotune-off and closed the speedup > question ("settings suffice"); this prompt's "autotuning ruled out" section diff --git a/draft/feature/profiling/compile_axis_campaign_coverage.md b/draft/feature/profiling/compile_axis_campaign_coverage.md new file mode 100644 index 00000000..3b13ee32 --- /dev/null +++ b/draft/feature/profiling/compile_axis_campaign_coverage.md @@ -0,0 +1,81 @@ +# Profiling Agent phase 1 — `campaign --axis compile`: what compile coverage do we actually have? + +Type: feature +Target: PyAutoBrain +Repos: +- PyAutoBrain +Difficulty: small +Autonomy: supervised +Priority: high +Status: formalised + +Phase 1 of `profiling_agent_jax_compile_time_scope.md` (read that first — it +carries the re-scope, the evidence and the comparability constraint). + +## Why this leg first + +It is the cheapest of the three, it is pure-read, and it produces the number that +justifies the other two. Right now nobody can say how much of the science grid the +compile corpus covers, because nothing cross-references the two trees. + +Measured by hand 2026-08-10 — this is what the mode should compute automatically: + +- **93 records**, all under `scripts/misc/jax_compile/results//.json`. +- Distinct `(dataset_class, model_type, instrument)` touched: **`imaging/mge/hst`, + `imaging/pixelization/hst`, `imaging/delaunay_matern/hst`, `imaging/knn/hst`**, + plus two synthetic multi-band classes (`datacube_img`, `datacube_img_hetero`). +- The runtime grid (`scripts/misc/likelihood_runtime/sweep.py::CELLS`) is **24** + `(class, model, instrument)` cells across `imaging` / `interferometer` / + `datacube` × `hst,jwst,ao` / `sma,alma,alma_high,jvla`. +- So: **no interferometer row, no datacube row, no `jwst`, no `ao`** — one + instrument out of seven, and `delaunay_matern` / `knn` are mesh variants that are + not grid cells at all. +- Hardware: `local_cpu` (82 rows), A100 (9, all `pixelization`/`mge` × `jit`/`vag`), + RTX 2060 (2). Plus **4 malformed rows** with `hardware`/`dataset_class`/ + `instrument` all `null`. + +## Scope + +Add `--axis compile` to the existing `campaign` mode in +`agents/conductors/profiling/_profiling.py`. The runtime axis stays the default and +is untouched. + +1. **Read the compile corpus.** Load `scripts/misc/jax_compile/results/*/*.json` + (append-only lists of flat records). Stdlib `json` only — the same + never-import-the-workspace rule the runtime path already follows via `ast`. +2. **Resolve each record to a grid cell** using its in-record + `(dataset_class, model_type, instrument)`, not its file path — the results tree + is filed by `/` and drops class/instrument from the path. +3. **Report coverage** over cell × transform × hardware-tier. The transform axis is + `probe.py`'s seven: `jit`, `grad`, `vag`, `vmap`, `vmap_vag`, `laxmap_vag`, + `pyloop_vag`. Cells not in `CELLS` are reported in their own **off-grid** bucket + rather than silently counted or silently dropped — `knn` and `delaunay_matern` + are real measurements from the Prodigy census and must not read as noise. +4. **Report malformed records** (missing `hardware` / `dataset_class` / + `instrument`) as their own bucket with file + index, so they can be fixed or + deleted deliberately. +5. **Emit a dispatch plan** of concrete `probe.py` invocations for what is missing, + mirroring how the runtime campaign emits `sweep.py` lines: + `python jax_compile/probe.py --model-type --transforms ` for local, + and the `hpc/batch_gpu/submit_*` form for `--tier a100`. + +## Explicitly not in scope + +- Any judgement about whether a compile time is *good* — that is phase 3. +- Any pin, baseline or dashboard — that is phase 2. +- Running `probe.py`. The agent reasons and emits a plan; it never executes the + workspace (`ProfilingDecision` contract). + +## Acceptance + +- `pyauto-brain profiling campaign --axis compile` and + `... --axis compile --tier a100` both run against a real `autolens_profiling` + checkout and report done / missing / off-grid / malformed counts. +- The reported on-grid coverage matches the hand count above (3–4 cells of 24, + `hst` only) on the corpus as it stands. +- `--json` emits the same structure, consistent with the existing modes. +- The runtime axis output is byte-identical to before the change (regression test). +- No file in `autolens_profiling` is written or executed by the agent. + + diff --git a/draft/feature/profiling/compile_axis_triage_drift.md b/draft/feature/profiling/compile_axis_triage_drift.md new file mode 100644 index 00000000..2362143b --- /dev/null +++ b/draft/feature/profiling/compile_axis_triage_drift.md @@ -0,0 +1,62 @@ +# Profiling Agent phase 3 — `triage --axis compile`: classify compile drift, route the real ones + +Type: feature +Target: PyAutoBrain +Repos: +- PyAutoBrain +Difficulty: small +Autonomy: supervised +Priority: high +Status: formalised + +> Depends on phase 2, `draft/feature/profiling/compile_warm_baseline_dashboard.md` +> (its pins). Not declared as `Blocked-by:` yet because that key grades **issue** +> refs and phase 2 has no issue — add the real ref at `/start_dev` time. + +Phase 3, and the point of the arc: phases 1–2 make compile drift *visible*; this +makes it *actionable*. Read `profiling_agent_jax_compile_time_scope.md` first. + +## Scope + +Extend `triage` with `--axis compile`, classifying each drifted pin from phase 2 +into one of four outcomes — the same shape as the runtime triage's per-finding +classification. + +| classification | signal | action | +|---|---|---| +| **cache regression** | a `warm` row's compile reverts toward its `cold` scale on an unchanged key | **the alarm this arc exists for** — the persistent cache is not being hit; route to config/stack, not to the library | +| **autotune regression** | GPU compile jumps ~an order of magnitude on the pathological shapes | `--xla_gpu_autotune_level=0` is not reaching XLA — the exact PyAutoNerves#127 `XLA_FLAGS`-clobbering failure, which went undetected for two months | +| **expected recompile** | `jax_version` moved | **not drift** — cache keys include jax version, so one recompile is by design. Re-pin, do not report | +| **stale pin** | measurement conditions changed (host, core count, `mixed_precision`) | re-pin here | +| **library regression** | compile grows on an unchanged key with no config explanation | route to `bug/` via intake — *classify and route only* | + +## The two constraints inherited from the agent's existing contract + +- **Profiling records and flags; it never adjudicates library correctness.** A + suspected library regression is routed to `bug/` via intake; the debug is never + planned inside the profiling repo. This is the existing `triage` boundary and it + applies unchanged. +- **Never compare across the comparability key.** `(hardware, jax_version, + mixed_precision, cache_state)`. Compile timings are host-load-sensitive — the + 7×-wrong measurements (851 s vs 117 s, same compile) are the standing reminder. + +## Close the arc + +- Move "JAX compilation-time profiling of likelihood functions" from **Future + modes** to the **Modes** table in `agents/conductors/profiling/AGENTS.md`. +- Record in **Boundaries** that release-validation script cost stayed with the + hygiene conductor's `perf` mode, so the question does not get re-opened a third + time. + +## Acceptance + +- `pyauto-brain profiling triage --axis compile` classifies every phase-2 drift row + into exactly one of the five outcomes, with the evidence that drove it. +- A synthetic warm-reverting-to-cold row classifies as **cache regression**. +- A drift explained solely by a `jax_version` bump classifies as **expected + recompile** and is not reported as a regression. +- Nothing is written to `autolens_profiling`; library findings are emitted as + intake-routable `bug/` candidates only. +- `AGENTS.md` Modes/Boundaries updated as above. + + diff --git a/draft/feature/profiling/compile_warm_baseline_dashboard.md b/draft/feature/profiling/compile_warm_baseline_dashboard.md new file mode 100644 index 00000000..9b7ca720 --- /dev/null +++ b/draft/feature/profiling/compile_warm_baseline_dashboard.md @@ -0,0 +1,102 @@ +# Profiling Agent phase 2 — make warm compile machine-identifiable, pinnable, and dashboarded + +Type: feature +Target: autolens_profiling +Repos: +- autolens_profiling +- PyAutoBrain +Difficulty: medium +Autonomy: supervised +Priority: high +Status: formalised + +> Depends on phase 1, `draft/feature/profiling/compile_axis_campaign_coverage.md` +> (its cell-resolution logic). Not declared as `Blocked-by:` yet because that key +> grades **issue** refs and phase 1 has no issue — add the real ref at +> `/start_dev` time, once phase 1 is issued. + +Phase 2 of `profiling_agent_jax_compile_time_scope.md`. **Absorbs** +`draft/feature/autolens_profiling/jax_compile_time_profiling.md`, whose surviving +scope (2026-07-28 re-scope: "the recurring cell-grid compile **dashboard** — track +warm compile per cell so cache regressions are caught") is this prompt's workspace +leg. + +## The blocker this prompt exists to remove + +The whole arc's purpose is to catch the persistent-compilation-cache win +(117.0 s → 2.3 s CPU; 5517.8 s → 937.1 s A100 end-to-end) silently reverting. +That requires tracking **warm** compile per cell. + +**Warm rows are not machine-identifiable today.** Warmness is encoded only in a +free-text `tag`. Across the 93 committed records there are ~40 distinct ad-hoc +values: `census-warm`, `census-warm2`, `cache-warm`, `a100-census-warm`, +`prodigy-census-warm`, `prodigy-census-warm-retry`, `prodigy-census-ral32-warm`, +`mb_homo_warm`, `mb_hetero_warm`, against colds spelled a dozen other ways, plus +`idle-check`, `smoke`, `matrix`, `pix-first`, `a100-at0`, `flag-parallel-codegen`. +No parser should be asked to guess at that, and a substring match on `"warm"` +would be a trap: `cache_dir` is non-empty on rows tagged `*-cold` too (the cache is +*configured*, and the cold row is the one that populates it). + +`cache_dir` is recorded but as a machine-specific path +(`/tmp/jax_cache` vs the RAL path), so it answers "a cache was configured", never +"this row hit it". + +## Workspace leg — `autolens_profiling` + +1. **Add an explicit `cache_state` field to `probe.py`'s record**: `cold` | `warm` | + `none` (no cache configured). Derive it from what the probe actually did, not + from the tag. `--cache-dir` with a pre-existing populated entry for the shape is + `warm`; a fresh/empty cache dir is `cold`; no `--cache-dir` is `none`. +2. **Add `host_state`** (or equivalent) capturing the idle/loaded provenance the + README currently carries in prose. This is load-bearing, not bookkeeping: the + first measurements were wrong by **7×** (851 s vs 117 s for the same compile) + purely from host load, and `prodigy-census-ral32-*` rows sit in the same tree at + a different core count. Record at least core count and load average; leave `tag` + as the free-text human note it already is. +3. **Backfill** the existing 93 records where the mapping is unambiguous from tag + + README provenance, and leave the rest `unknown` rather than guessing. Fix or + delete the **4 malformed records** (`hardware`/`dataset_class`/`instrument` all + `null`) as a deliberate call. +4. **Pin warm compile per cell** — the compile-axis equivalent of the runtime + results' `pinned_expected`, which is the hook `ingest`/`triage` are already + built around. A pin is only meaningful within one comparability key + `(hardware, jax_version, mixed_precision, cache_state)`, so the pin store must be + keyed on it. +5. **Dashboard rows** through the existing `build_baseline.py` / `build_readme.py` + pattern, so warm compile per cell appears alongside the runtime tables. + +## Brain leg — `PyAutoBrain` + +6. **`ingest --axis compile`**: report compile records that are unpinned, and + records whose warm compile has moved away from its pin — *only ever within one + comparability key*. Cross-key pairs are not a regression and must not be + reported as one. Mirror the runtime `ingest`'s freshness discipline (it skips + probes older than the table file for a documented reason). +7. Emit the same shape as the runtime axis: rows to apply, plus `steps` and + `next_action`. + +## The trap + +Do **not** let the dashboard compare a `cold` row against a `warm` pin, or an +A100 row against a CPU pin, or rows across a `jax_version` bump. Cache keys include +jax version and shapes, so a version bump recompiles once **by design** — that is +expected behaviour, not drift, and phase 3 classifies it as such. A tool that flags +it is a tool people learn to ignore. + +## Acceptance + +- `probe.py` records carry `cache_state` derived from probe behaviour; a fresh + cold/warm pair on one cell produces exactly one `cold` and one `warm` record with + no tag parsing anywhere in the pipeline. +- Warm compile per cell is pinned and rendered in the workspace dashboard. +- `pyauto-brain profiling ingest --axis compile` reports unpinned and drifted rows, + and **never** pairs rows across `(hardware, jax_version, mixed_precision, + cache_state)`. +- A synthetic warm row at cold-scale timing is reported as drift; the same row + under a bumped `jax_version` is not. +- The 4 malformed records are resolved. +- Runtime-axis behaviour unchanged. + + diff --git a/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md b/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md index 5f896b8b..e9932b4e 100644 --- a/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md +++ b/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md @@ -1,4 +1,4 @@ -# Extend the Profiling Agent scope to track JAX compile/eval times of release-validation heavy scripts +# Give the Profiling Agent a compile-time axis — the arc Type: feature Target: PyAutoBrain @@ -8,7 +8,145 @@ Repos: Difficulty: large Autonomy: supervised Priority: high -Status: formalised +Status: planned (arc — execute via the three child prompts below) + +> **RE-SCOPED 2026-08-10.** This prompt was filed 2026-07-14 asking the Profiling +> Agent to *measure JAX compile/eval time of the release-validation heavy scripts +> so we can speed them up*. Both halves of that framing are now spent: the +> speed-up shipped, and the release-script half belongs to a different conductor. +> What survives — and is still unbuilt — is **regression surveillance**. The +> original text is preserved under § What the prompt originally said. + +## Why the original framing no longer holds + +**The speed-up already shipped.** The `#71 → #74 → #77` arc +(`autolens_profiling/scripts/misc/jax_compile/README.md`) settled the question +this prompt opened, with controlled A/Bs: + +| lever | effect | status | +|---|---|---| +| persistent compilation cache | 117.0 s → **2.3 s** (CPU MGE `vag`, 51×); 5517.8 s → **937.1 s** (A100 pixelized Nautilus end-to-end) | shipped | +| `--xla_gpu_autotune_level=0` | FD probe 498 s → **29 s** (17×); full 300×16 Adam fit −40 %; likelihoods bit-identical | shipped | +| source jit-boundaries | ruled out by evidence — "**Do not restructure**" | closed | + +The prompt's own candidate root causes are likewise resolved: `lax.map` is +innocent (every batched-gradient structure compiles in the same ~105–125 s band), +the >30-min repeated-fusion observations were host-load contention plus a +re-firing alarm banner, and the tracing floor is 58 % jax-internal with no PyAuto +lever. + +**So the durable risk is no longer slowness — it is silent regression.** Both wins +are *settings*. A config drift, a `jax` version bump, or a wrapper that clobbers +`XLA_FLAGS` (exactly what PyAutoNerves#127 fixed, and what invalidated the +2026-07-15 "autotuning ruled out" A/B for two months) puts the 70-minute +worst case straight back without anything failing. Nothing watches for that today. + +This is precisely the handoff the research left open — `jax_compile/README.md` +Verdict item 4: *"should track **warm** compile times per cell so cache +regressions are caught"*, and `probe.py`'s own docstring: *"industrializes this +across the full cell grid once the research settles the method."* The method has +settled. + +## Why the release-validation-scripts leg is dropped + +The original prompt asked profiling to cover the release-validation heavy scripts +(`jax_grad/imaging_pixelization.py` et al. blowing the 300 s cap). That crosses a +boundary both conductors already document: + +- `agents/conductors/profiling/AGENTS.md` — profiling owns "the product's + modelling / compute speed (likelihood on the science grid, GPU tiers, A100)". +- `agents/conductors/hygiene/AGENTS.md` — hygiene owns "the *developer loop's* + cost (unit tests, `PYAUTO_TEST_MODE` / `PYAUTO_SMALL_DATASETS` integration + scripts, import time)", and "Hunting generally-slow functions flagged by + integration tests is hygiene's `perf` mode (**moved here from profiling's staged + future modes**)". + +Script-suite cost was deliberately moved *out* of this agent once already. Pulling +it back in would re-open a settled boundary to serve a motivation (the 300 s cap) +that PyAutoHeart#72 has since handled. Decision 2026-08-10: **out of scope here**; +if release-script cost needs an owner, it is a `/hygiene` `perf` prompt, filed +separately. + +## The gap, confirmed against current main + +- `agents/conductors/profiling/AGENTS.md` still lists *"JAX compilation-time + profiling of likelihood functions"* under **Future modes**. +- `agents/conductors/profiling/_profiling.py` (353 lines) contains **zero** + occurrences of "compile". `campaign` / `ingest` / `triage` read only + `results/runtime/`. +- The workspace instrument exists and is already producing data — 93 records under + `scripts/misc/jax_compile/results//.json`. + +So this is a **wiring** task, not an instrument-building one. The agent cannot see +a tree that is already full. + +## The three blockers the child prompts exist to clear + +Measured against the 93 committed records, 2026-08-10: + +1. **Coverage is 3 of 24 grid cells.** Every record is `hst`; there is **no + interferometer, no datacube, no `jwst`, no `ao`** row, and only `local_cpu` is + meaningfully populated (A100 has 9 rows, all `pixelization`/`mge` × `jit`/`vag`). + Two model types present (`knn`, `delaunay_matern`) are mesh variants that are + not in the runtime `CELLS` grid at all. → **child prompt 1**. +2. **Warm rows are not machine-identifiable.** "Warm" is encoded only in a + free-text `tag` — ~40 distinct ad-hoc values across the corpus + (`census-warm`, `census-warm2`, `prodigy-census-warm-retry`, `cache-warm`, + `a100-census-warm`, `mb_homo_cold`, `idle-check`, `smoke`, `matrix`, …). + A dashboard that must track *warm* compile literally cannot tell which rows are + warm. `cache_dir` is recorded but as a machine-specific path, so it answers + "a cache was configured", not "this row hit it". → **child prompt 2**. +3. **Compile rows have no pin.** Runtime results carry `pinned_expected`, which is + what `ingest` and `triage` are built around. Nothing equivalent exists for + compile, so there is no baseline to regress against. → **child prompt 2**, used + by **child prompt 3**. + +Also present and needing a decision rather than a fix: **4 malformed records** with +`hardware`, `dataset_class` and `instrument` all `null`. + +## The constraint any design must carry + +**Compile timings are host-load-sensitive and cross-host comparison is invalid.** +The README records the first measurements being wrong by up to **7×** (851 s vs +117 s for the same compile) purely from host load, and those rows are retained +"only with their original tags for provenance". The corpus already contains +`prodigy-census-ral32-*` rows taken on a different core count from the +`local_cpu` rows filed beside them. + +So comparability is a **key**, not a nicety: +`(hardware, jax_version, mixed_precision, cache_state)` — plus an explicit +idle/loaded provenance signal. An agent that compares across that key will report +regressions that are only a busy laptop, which is worse than no surveillance at +all. + +## Phasing + +| # | Prompt | Repos | Size | +|---|---|---|---| +| 1 | `compile_axis_campaign_coverage.md` | PyAutoBrain | small | +| 2 | `compile_warm_baseline_dashboard.md` | autolens_profiling + PyAutoBrain | medium | +| 3 | `compile_axis_triage_drift.md` | PyAutoBrain | small–medium | + +Strictly sequential: 2 needs 1's cell-resolution logic, 3 needs 2's pins. + +`draft/feature/autolens_profiling/jax_compile_time_profiling.md` (already +re-scoped 2026-07-28 to "the recurring cell-grid compile dashboard") is the +workspace half of child prompt 2 and is **absorbed** into it — see its header. + +## Acceptance (arc) + +- `pyauto-brain profiling --axis compile` answers all three modes. +- The compile axis never compares rows across the comparability key. +- `AGENTS.md` moves compile-time profiling from **Future modes** to **Modes**. +- A cache regression (warm compile reverting toward cold) is *detected*, which is + the surveillance function the whole arc exists for. + +--- + +## What the prompt originally said + +
+Original text, filed 2026-07-14 (superseded — kept for provenance) Extend the **PyAutoBrain Profiling Agent** (`agents/conductors/profiling/`, workspace `autolens_profiling`) scope to track **JAX compile-time and eval-time** of the @@ -36,4 +174,10 @@ release-validation JAX scripts) + the ranked report driving a hygiene/perf backl Large — expect to phase at start_dev time. Cross-ref the mode=release timeout/scope policy question on PyAutoHeart#72. +
+ + From 0393edced43c707d3c307a5501329551e0bc8683 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 13:07:25 +0000 Subject: [PATCH 2/7] =?UTF-8?q?prompt:=20route=20compile-axis-campaign-cov?= =?UTF-8?q?erage=20(#218)=20=E2=86=92=20start=5Flibrary?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 1 of the compile-axis arc is issued as PyAutoBrain#218. The prompt advances draft/ -> active/ and registers in active.md as library-dev. No worktree claim: this is a Claude-Code-on-the-web session with no ~/Code/PyAutoLabs-wt/ to create, so the entry records the branch on one line rather than as a 2-space repo bullet, which worktree_check_conflict would read as a live worktree claim. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- active.md | 18 ++++++++++++++++++ .../compile_axis_campaign_coverage.md | 0 2 files changed, 18 insertions(+) rename {draft/feature/profiling => active}/compile_axis_campaign_coverage.md (100%) diff --git a/active.md b/active.md index fc6b299b..dfdcfd53 100644 --- a/active.md +++ b/active.md @@ -34,3 +34,21 @@ - artifacts-are-laptop-only: Actions artifact downloads are blocked from cloud/mobile sessions (egress policy 403s `productionresultssa2.blob.core.windows.net` on CONNECT) — this is what stopped the cloud session finishing the ingest. Both wiki drift reports were captured to `~/.pyauto-heart/release_20260807_wiki_drift/` while on the laptop. - do-not: do NOT use the nightly driver for a manual release — AUTONOMY.md forbids converting a manual release into the scheduled-nightly exception. - repos-none-claimed: this entry claims NO repos — deliberately on one line, NOT as 2-space ` - Repo` bullets, because `worktree_check_conflict` treats any such bullet as a live claim. + +## compile-axis-campaign-coverage +- issue: https://github.com/PyAutoLabs/PyAutoBrain/issues/218 +- session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree — see below) +- status: library-dev +- prompt: active/compile_axis_campaign_coverage.md +- arc: phase 1 of 3 — parent `draft/feature/profiling/profiling_agent_jax_compile_time_scope.md`; + phase 2 `draft/feature/profiling/compile_warm_baseline_dashboard.md`, + phase 3 `draft/feature/profiling/compile_axis_triage_drift.md`. Strictly sequential. +- worktree: none — this is a Claude-Code-on-the-web session, which has no + `~/Code/PyAutoLabs-wt/` to create. Work happens on branch + `feature/compile-axis-campaign-coverage` in the session's own PyAutoBrain clone + and is pushed to origin. If the task is resumed on the laptop, create the + worktree then and check out the pushed branch rather than starting fresh. +- repos-claimed-on-one-line: PyAutoBrain (branch feature/compile-axis-campaign-coverage). + Deliberately NOT written as a 2-space ` - Repo` bullet: `worktree_check_conflict` + treats any such bullet as a live worktree claim, and there is no worktree here. + A second session wanting PyAutoBrain should still treat this as taken. diff --git a/draft/feature/profiling/compile_axis_campaign_coverage.md b/active/compile_axis_campaign_coverage.md similarity index 100% rename from draft/feature/profiling/compile_axis_campaign_coverage.md rename to active/compile_axis_campaign_coverage.md From 7ffbe0e586273a784b23a5ef97f1ef116b6a91e3 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 13:12:48 +0000 Subject: [PATCH 3/7] prompt: correct the compile-grid figures from the phase-1 build The re-scope estimated "3 of 24 grid cells" by hand. Phase 1 computed it: the sweep CELLS grid is 21 cells, and only imaging/mge/hst and imaging/pixelization/hst are on-grid -- 11 of 147 cell x transform runs on the local tier, 3 on a100. knn and delaunay_matern are off-grid mesh variants rather than partial coverage, which the estimate had folded in. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- active/compile_axis_campaign_coverage.md | 10 +++++++--- .../profiling_agent_jax_compile_time_scope.md | 6 +++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/active/compile_axis_campaign_coverage.md b/active/compile_axis_campaign_coverage.md index 3b13ee32..698b85c9 100644 --- a/active/compile_axis_campaign_coverage.md +++ b/active/compile_axis_campaign_coverage.md @@ -24,7 +24,7 @@ Measured by hand 2026-08-10 — this is what the mode should compute automatical - Distinct `(dataset_class, model_type, instrument)` touched: **`imaging/mge/hst`, `imaging/pixelization/hst`, `imaging/delaunay_matern/hst`, `imaging/knn/hst`**, plus two synthetic multi-band classes (`datacube_img`, `datacube_img_hetero`). -- The runtime grid (`scripts/misc/likelihood_runtime/sweep.py::CELLS`) is **24** +- The runtime grid (`scripts/misc/likelihood_runtime/sweep.py::CELLS`) is **21** `(class, model, instrument)` cells across `imaging` / `interferometer` / `datacube` × `hst,jwst,ao` / `sma,alma,alma_high,jvla`. - So: **no interferometer row, no datacube row, no `jwst`, no `ao`** — one @@ -71,8 +71,12 @@ is untouched. - `pyauto-brain profiling campaign --axis compile` and `... --axis compile --tier a100` both run against a real `autolens_profiling` checkout and report done / missing / off-grid / malformed counts. -- The reported on-grid coverage matches the hand count above (3–4 cells of 24, - `hst` only) on the corpus as it stands. +- The reported on-grid coverage matches the hand count above on the corpus as it + stands. MEASURED once built: **11 of 147** cell×transform runs on the `local` + tier and **3 of 147** on `a100`, i.e. only `imaging/mge/hst` and + `imaging/pixelization/hst` are on-grid. (The pre-build estimate said "3–4 cells + of 24" — the grid is 21 cells, and `knn`/`delaunay_matern` are off-grid rather + than partial coverage.) - `--json` emits the same structure, consistent with the existing modes. - The runtime axis output is byte-identical to before the change (regression test). - No file in `autolens_profiling` is written or executed by the agent. diff --git a/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md b/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md index e9932b4e..edc4ec69 100644 --- a/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md +++ b/draft/feature/profiling/profiling_agent_jax_compile_time_scope.md @@ -84,11 +84,15 @@ a tree that is already full. Measured against the 93 committed records, 2026-08-10: -1. **Coverage is 3 of 24 grid cells.** Every record is `hst`; there is **no +1. **Coverage is 2 of 21 grid cells.** Every record is `hst`; there is **no interferometer, no datacube, no `jwst`, no `ao`** row, and only `local_cpu` is meaningfully populated (A100 has 9 rows, all `pixelization`/`mge` × `jit`/`vag`). Two model types present (`knn`, `delaunay_matern`) are mesh variants that are not in the runtime `CELLS` grid at all. → **child prompt 1**. + + *(Corrected 2026-08-10 once phase 1 computed it: the grid is 21 cells, not the + 24 first written here, and only `imaging/mge/hst` + `imaging/pixelization/hst` + are on-grid — 11 of 147 cell×transform runs on the local tier, 3 on A100.)* 2. **Warm rows are not machine-identifiable.** "Warm" is encoded only in a free-text `tag` — ~40 distinct ad-hoc values across the corpus (`census-warm`, `census-warm2`, `prodigy-census-warm-retry`, `cache-warm`, From 366489b5c12c93161a3c924431ab5af91a333b99 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 14:23:17 +0000 Subject: [PATCH 4/7] =?UTF-8?q?prompt:=20route=20compile-warm-baseline-das?= =?UTF-8?q?hboard=20(#103)=20=E2=80=94=20schema=20leg=20in=20PR?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 is issued as autolens_profiling#103 with the schema leg in PR #104. Registered as PARTIAL rather than in-flight: pins and dashboard rows remain, and the Brain leg is blocked on PyAutoBrain#219 merging. Also records a CI gap found while working: autolens_profiling runs no pytest step, so scripts/misc/test/ never runs in CI and a vram_config test is failing on main unnoticed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- active.md | 22 +++++++++++++++++++ .../compile_warm_baseline_dashboard.md | 0 2 files changed, 22 insertions(+) rename {draft/feature/profiling => active}/compile_warm_baseline_dashboard.md (100%) diff --git a/active.md b/active.md index dfdcfd53..abbcfe2d 100644 --- a/active.md +++ b/active.md @@ -52,3 +52,25 @@ Deliberately NOT written as a 2-space ` - Repo` bullet: `worktree_check_conflict` treats any such bullet as a live worktree claim, and there is no worktree here. A second session wanting PyAutoBrain should still treat this as taken. + +## compile-warm-baseline-dashboard +- issue: https://github.com/PyAutoLabs/autolens_profiling/issues/103 +- pr: https://github.com/PyAutoLabs/autolens_profiling/pull/104 (schema leg only) +- session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree) +- status: workspace-dev — PARTIAL. Schema leg (cache_state / host_state / + record_kind / backfill of 89 records) is in PR #104. Pins + dashboard rows + remain; the Brain leg (`ingest --axis compile`) is blocked on PyAutoBrain#219 + merging, since it needs that PR's `load_compile_corpus` / `compile_tier_of`. +- prompt: active/compile_warm_baseline_dashboard.md +- arc: phase 2 of 3 — phase 1 PyAutoBrain#218 (PR #219, CI green, awaiting merge); + phase 3 `draft/feature/profiling/compile_axis_triage_drift.md` not started. +- worktree: none — cloud session; work is on pushed branches, not a + `~/Code/PyAutoLabs-wt/` worktree. Resume on the laptop by checking those out. +- repos-claimed-on-one-line: autolens_profiling (branch + feature/compile-warm-baseline-dashboard), PyAutoBrain (branch + feature/compile-axis-campaign-coverage). Deliberately NOT 2-space ` - Repo` + bullets: `worktree_check_conflict` reads those as live worktree claims. +- ci-gap-found: autolens_profiling runs NO pytest step (lint.yml is ruff + + build_readme --check + lychee + smoke), so scripts/misc/test/ never runs in CI + and test_vram_config.py::test_aggregate_matches_cell_prefixed_stems is failing + on main unnoticed. Flagged on issue #103; not folded into PR #104. diff --git a/draft/feature/profiling/compile_warm_baseline_dashboard.md b/active/compile_warm_baseline_dashboard.md similarity index 100% rename from draft/feature/profiling/compile_warm_baseline_dashboard.md rename to active/compile_warm_baseline_dashboard.md From 472adede7ee5f15d8eccb34c300eb7fee2148772 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 14:42:23 +0000 Subject: [PATCH 5/7] =?UTF-8?q?prompt:=20compile-warm-baseline-dashboard?= =?UTF-8?q?=20complete=20=E2=80=94=20all=20three=20legs=20in=20PRs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 is no longer partial. Records the merge order (PyAutoBrain#219 before the stacked #220) and the three corrections that only surfaced by running the tools against the real corpus rather than reasoning about the design. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- active.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/active.md b/active.md index abbcfe2d..987976b7 100644 --- a/active.md +++ b/active.md @@ -55,12 +55,22 @@ ## compile-warm-baseline-dashboard - issue: https://github.com/PyAutoLabs/autolens_profiling/issues/103 -- pr: https://github.com/PyAutoLabs/autolens_profiling/pull/104 (schema leg only) +- pr: https://github.com/PyAutoLabs/autolens_profiling/pull/104 (workspace legs) +- pr: https://github.com/PyAutoLabs/PyAutoBrain/pull/220 (Brain leg; stacked on #219) - session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree) -- status: workspace-dev — PARTIAL. Schema leg (cache_state / host_state / - record_kind / backfill of 89 records) is in PR #104. Pins + dashboard rows - remain; the Brain leg (`ingest --axis compile`) is blocked on PyAutoBrain#219 - merging, since it needs that PR's `load_compile_corpus` / `compile_tier_of`. +- status: workspace-dev — COMPLETE, awaiting review. All three legs are in PRs: + schema (cache_state / host_state / record_kind + backfill of 89 records), + pins + dashboard (25 pins, sticky, rendered via the existing sentinel-block + mechanism), and the Brain's `ingest --axis compile` (25 pins, 0 drifted, + 0 unpinned against the real corpus). +- merge-order: PyAutoBrain#219 first (PR #220 is stacked on it and retargets to + main once it lands); autolens_profiling#104 is independent of both. +- corrections-found-by-running: (1) phase 1 called 4 sibling-instrument records + "malformed" — export_probe.py/trace_profile.py share the results tree with a + different schema; fixed in #219. (2) `hardware` alone pooled a laptop and a + 32-core RAL node under `local_cpu`, so `hostname` joined the comparability + key. (3) pins auto-following the newest row would have baked a regression in; + pins are now sticky, `--repin` required. - prompt: active/compile_warm_baseline_dashboard.md - arc: phase 2 of 3 — phase 1 PyAutoBrain#218 (PR #219, CI green, awaiting merge); phase 3 `draft/feature/profiling/compile_axis_triage_drift.md` not started. From b216e2b094d89179fd53386a5f1fccbed12517b1 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 15:02:09 +0000 Subject: [PATCH 6/7] =?UTF-8?q?prompt:=20route=20compile-axis-triage-drift?= =?UTF-8?q?=20(#221)=20=E2=80=94=20arc=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3 is issued and built; PR #222 closes the compile-axis arc. Records the three-PR merge order and the one deviation from the prompt: two of its five classifications cannot reach triage by construction, since a jax_version bump or a changed host is a different comparability key and ingest reports those as unpinned rather than drifted. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- active.md | 21 +++++++++++++++++++ .../compile_axis_triage_drift.md | 0 2 files changed, 21 insertions(+) rename {draft/feature/profiling => active}/compile_axis_triage_drift.md (100%) diff --git a/active.md b/active.md index 987976b7..efa69f6b 100644 --- a/active.md +++ b/active.md @@ -84,3 +84,24 @@ build_readme --check + lychee + smoke), so scripts/misc/test/ never runs in CI and test_vram_config.py::test_aggregate_matches_cell_prefixed_stems is failing on main unnoticed. Flagged on issue #103; not folded into PR #104. + +## compile-axis-triage-drift +- issue: https://github.com/PyAutoLabs/PyAutoBrain/issues/221 +- pr: https://github.com/PyAutoLabs/PyAutoBrain/pull/222 (stacked on #220, itself on #219) +- session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree) +- status: library-dev — COMPLETE, awaiting review. Closes the compile-axis arc: + all three modes now serve `--axis compile`, and AGENTS.md moves compile-time + profiling out of Future modes. +- prompt: active/compile_axis_triage_drift.md +- arc: phase 3 of 3. Phase 1 PyAutoBrain#219, phase 2 PyAutoBrain#220 + + autolens_profiling#104. MERGE ORDER: #219 -> #220 -> #222; each retargets to + main as the one below lands. autolens_profiling#104 is independent. +- deviation-from-prompt: the prompt's five classifications assumed jax-version + and changed-conditions drift would reach triage. They cannot — both are + different comparability keys, so ingest reports them UNPINNED, never drifted. + Classified as bookkeeping so nothing vanishes; `host-load` added in their + place (what host_state was added for, and host load alone has produced 7x + errors in this corpus). +- repos-claimed-on-one-line: PyAutoBrain (branch feature/compile-triage-drift). + Deliberately not a 2-space ` - Repo` bullet, which worktree_check_conflict + reads as a live worktree claim. diff --git a/draft/feature/profiling/compile_axis_triage_drift.md b/active/compile_axis_triage_drift.md similarity index 100% rename from draft/feature/profiling/compile_axis_triage_drift.md rename to active/compile_axis_triage_drift.md From a7bda52e51e47e26b0a1fe474c0c576557825d1d Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 15:18:05 +0000 Subject: [PATCH 7/7] =?UTF-8?q?mind:=20record=20the=20compile-axis=20arc?= =?UTF-8?q?=20shipped=20=E2=80=94=20all=20three=20phases=20merged?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Profiling Agent now has a compile-time axis across campaign, ingest and triage, and AGENTS.md no longer lists it under Future modes. Merged: PyAutoBrain#219 (cd79005), autolens_profiling#104 (355d555), PyAutoBrain#220 (74d7b1b), PyAutoBrain#222 (a50efc3). Verified end to end on merged main against the merged workspace: campaign 11/147 covered, ingest 25 pins / 0 drifted / 0 unpinned, triage 0 findings; Brain suite 329 passed, workspace 47 passed in CI. Records carry the traps worth remembering, all of which surfaced by running the tools against real data rather than from the design: - cache_dir is non-empty on COLD rows too, so warmness cannot be read from it or from the tag - `hardware` alone pooled a laptop and a 32-core RAL node under one label; hostname had to join the comparability key - pins auto-following the newest row would have baked a regression in permanently, reporting all-clear forever - rows predating a pin are history, not drift - export_probe.py / trace_profile.py share the results tree with a different schema, so their records are not malformed Also records the two corrections to my own earlier claims: the grid is 21 cells rather than 24, and the workspace test that looked red outside CI was a missing matplotlib, not a defect. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv --- active.md | 71 ------------------- .../2026/08/compile-axis-campaign-coverage.md | 38 ++++++++++ .../2026/08/compile-axis-triage-drift.md | 38 ++++++++++ .../08/compile-warm-baseline-dashboard.md | 42 +++++++++++ complete/index.md | 5 +- 5 files changed, 122 insertions(+), 72 deletions(-) rename active/compile_axis_campaign_coverage.md => complete/2026/08/compile-axis-campaign-coverage.md (69%) rename active/compile_axis_triage_drift.md => complete/2026/08/compile-axis-triage-drift.md (64%) rename active/compile_warm_baseline_dashboard.md => complete/2026/08/compile-warm-baseline-dashboard.md (70%) diff --git a/active.md b/active.md index efa69f6b..fc6b299b 100644 --- a/active.md +++ b/active.md @@ -34,74 +34,3 @@ - artifacts-are-laptop-only: Actions artifact downloads are blocked from cloud/mobile sessions (egress policy 403s `productionresultssa2.blob.core.windows.net` on CONNECT) — this is what stopped the cloud session finishing the ingest. Both wiki drift reports were captured to `~/.pyauto-heart/release_20260807_wiki_drift/` while on the laptop. - do-not: do NOT use the nightly driver for a manual release — AUTONOMY.md forbids converting a manual release into the scheduled-nightly exception. - repos-none-claimed: this entry claims NO repos — deliberately on one line, NOT as 2-space ` - Repo` bullets, because `worktree_check_conflict` treats any such bullet as a live claim. - -## compile-axis-campaign-coverage -- issue: https://github.com/PyAutoLabs/PyAutoBrain/issues/218 -- session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree — see below) -- status: library-dev -- prompt: active/compile_axis_campaign_coverage.md -- arc: phase 1 of 3 — parent `draft/feature/profiling/profiling_agent_jax_compile_time_scope.md`; - phase 2 `draft/feature/profiling/compile_warm_baseline_dashboard.md`, - phase 3 `draft/feature/profiling/compile_axis_triage_drift.md`. Strictly sequential. -- worktree: none — this is a Claude-Code-on-the-web session, which has no - `~/Code/PyAutoLabs-wt/` to create. Work happens on branch - `feature/compile-axis-campaign-coverage` in the session's own PyAutoBrain clone - and is pushed to origin. If the task is resumed on the laptop, create the - worktree then and check out the pushed branch rather than starting fresh. -- repos-claimed-on-one-line: PyAutoBrain (branch feature/compile-axis-campaign-coverage). - Deliberately NOT written as a 2-space ` - Repo` bullet: `worktree_check_conflict` - treats any such bullet as a live worktree claim, and there is no worktree here. - A second session wanting PyAutoBrain should still treat this as taken. - -## compile-warm-baseline-dashboard -- issue: https://github.com/PyAutoLabs/autolens_profiling/issues/103 -- pr: https://github.com/PyAutoLabs/autolens_profiling/pull/104 (workspace legs) -- pr: https://github.com/PyAutoLabs/PyAutoBrain/pull/220 (Brain leg; stacked on #219) -- session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree) -- status: workspace-dev — COMPLETE, awaiting review. All three legs are in PRs: - schema (cache_state / host_state / record_kind + backfill of 89 records), - pins + dashboard (25 pins, sticky, rendered via the existing sentinel-block - mechanism), and the Brain's `ingest --axis compile` (25 pins, 0 drifted, - 0 unpinned against the real corpus). -- merge-order: PyAutoBrain#219 first (PR #220 is stacked on it and retargets to - main once it lands); autolens_profiling#104 is independent of both. -- corrections-found-by-running: (1) phase 1 called 4 sibling-instrument records - "malformed" — export_probe.py/trace_profile.py share the results tree with a - different schema; fixed in #219. (2) `hardware` alone pooled a laptop and a - 32-core RAL node under `local_cpu`, so `hostname` joined the comparability - key. (3) pins auto-following the newest row would have baked a regression in; - pins are now sticky, `--repin` required. -- prompt: active/compile_warm_baseline_dashboard.md -- arc: phase 2 of 3 — phase 1 PyAutoBrain#218 (PR #219, CI green, awaiting merge); - phase 3 `draft/feature/profiling/compile_axis_triage_drift.md` not started. -- worktree: none — cloud session; work is on pushed branches, not a - `~/Code/PyAutoLabs-wt/` worktree. Resume on the laptop by checking those out. -- repos-claimed-on-one-line: autolens_profiling (branch - feature/compile-warm-baseline-dashboard), PyAutoBrain (branch - feature/compile-axis-campaign-coverage). Deliberately NOT 2-space ` - Repo` - bullets: `worktree_check_conflict` reads those as live worktree claims. -- ci-gap-found: autolens_profiling runs NO pytest step (lint.yml is ruff + - build_readme --check + lychee + smoke), so scripts/misc/test/ never runs in CI - and test_vram_config.py::test_aggregate_matches_cell_prefixed_stems is failing - on main unnoticed. Flagged on issue #103; not folded into PR #104. - -## compile-axis-triage-drift -- issue: https://github.com/PyAutoLabs/PyAutoBrain/issues/221 -- pr: https://github.com/PyAutoLabs/PyAutoBrain/pull/222 (stacked on #220, itself on #219) -- session: cloud session https://claude.ai/code/session_01L4STU81pQP1GkMzZVvsMsv (no laptop worktree) -- status: library-dev — COMPLETE, awaiting review. Closes the compile-axis arc: - all three modes now serve `--axis compile`, and AGENTS.md moves compile-time - profiling out of Future modes. -- prompt: active/compile_axis_triage_drift.md -- arc: phase 3 of 3. Phase 1 PyAutoBrain#219, phase 2 PyAutoBrain#220 + - autolens_profiling#104. MERGE ORDER: #219 -> #220 -> #222; each retargets to - main as the one below lands. autolens_profiling#104 is independent. -- deviation-from-prompt: the prompt's five classifications assumed jax-version - and changed-conditions drift would reach triage. They cannot — both are - different comparability keys, so ingest reports them UNPINNED, never drifted. - Classified as bookkeeping so nothing vanishes; `host-load` added in their - place (what host_state was added for, and host load alone has produced 7x - errors in this corpus). -- repos-claimed-on-one-line: PyAutoBrain (branch feature/compile-triage-drift). - Deliberately not a 2-space ` - Repo` bullet, which worktree_check_conflict - reads as a live worktree claim. diff --git a/active/compile_axis_campaign_coverage.md b/complete/2026/08/compile-axis-campaign-coverage.md similarity index 69% rename from active/compile_axis_campaign_coverage.md rename to complete/2026/08/compile-axis-campaign-coverage.md index 698b85c9..f671355c 100644 --- a/active/compile_axis_campaign_coverage.md +++ b/complete/2026/08/compile-axis-campaign-coverage.md @@ -1,3 +1,41 @@ +# Compile-axis phase 1 — `campaign --axis compile` + +- shipped: 2026-08-10 +- issue: https://github.com/PyAutoLabs/PyAutoBrain/issues/218 +- pr: https://github.com/PyAutoLabs/PyAutoBrain/pull/219 (squash cd79005) +- repos: PyAutoBrain +- arc: phase 1 of 3 (see compile-warm-baseline-dashboard, compile-axis-triage-drift) + +## Summary + +The Profiling Agent could not see the compile corpus `autolens_profiling` was +already producing: `AGENTS.md` listed compile-time profiling under *Future modes*, +`_profiling.py` had zero occurrences of "compile", and all three modes read only +`results/runtime/` — while 93 committed probe records sat uncross-referenced. + +`campaign --axis compile` answers coverage: **2 of 21 grid cells**, `hst` only, no +interferometer / datacube / `jwst` / `ao` row anywhere. That number is what +justified phases 2 and 3. + +## Traps and findings + +- **The tier vocabularies do not interchange.** Runtime buckets by sweep *config* + name (`local_cpu_fp64`), folding precision into the name; a compile record has a + raw `hardware` string plus a **separate** `mixed_precision` bool. Reusing + `TIER_CONFIGS` would have mis-bucketed every row. +- **Records must be placed by their own fields, not their path.** Results are filed + under `/`, which drops class and instrument entirely. +- **`export_probe.py` / `trace_profile.py` share the results tree** with a different + schema. The first implementation reported their 4 records as "malformed", which + would have sent someone to fix two files that work correctly. Missing the whole + `(hardware, dataset_class, instrument)` identity triple means *sibling + instrument*; missing only some key fields is corruption. +- Profiling was the **only conductor without a test file**, so "runtime axis + unchanged" had nothing to assert against. `tests/test_profiling_conductor.py` was + part of the work, not a bonus. + +## Original prompt + # Profiling Agent phase 1 — `campaign --axis compile`: what compile coverage do we actually have? Type: feature diff --git a/active/compile_axis_triage_drift.md b/complete/2026/08/compile-axis-triage-drift.md similarity index 64% rename from active/compile_axis_triage_drift.md rename to complete/2026/08/compile-axis-triage-drift.md index 2362143b..acddfeb7 100644 --- a/active/compile_axis_triage_drift.md +++ b/complete/2026/08/compile-axis-triage-drift.md @@ -1,3 +1,41 @@ +# Compile-axis phase 3 — `triage --axis compile`, and the arc closed + +- shipped: 2026-08-10 +- issue: https://github.com/PyAutoLabs/PyAutoBrain/issues/221 +- pr: https://github.com/PyAutoLabs/PyAutoBrain/pull/222 (squash a50efc3) +- repos: PyAutoBrain +- arc: phase 3 of 3 — closes it + +## Summary + +`ingest` says a warm compile moved; that alone is useless, because a dead cache, a +flag that stopped reaching XLA, a busy laptop and a real library regression are the +same number. `triage --axis compile` separates them into seven classifications, +three actionable, and names the owner. + +`AGENTS.md` moves compile-time profiling out of **Future modes** into the Modes +table; all three modes now serve `--axis compile`. + +## Traps and findings + +- **The cold-scale comparison makes `cache-regression` a measurement, not a + guess.** 25 of 32 cell/transform keys carry both a warm and a cold row, so the + yardstick is real data from the same machine. Verified by injecting a synthetic + regression into a copy of the real workspace (warm `vag` 1.622s → its own + 34.592s cold cost). +- **Two of the prompt's five categories cannot reach triage by construction.** A + `jax_version` bump or a changed host is a *different comparability key*, so + `ingest` reports it as unpinned, never drifted. Classified as bookkeeping so + nothing vanishes; never regressions. +- **`host-load` was added in their place** — not in the prompt, but host load alone + has produced 7x errors here, and a classifier that cannot say "your laptop was + busy" sends people chasing phantoms. +- Boundaries now records that **release-validation script cost stayed with the + hygiene conductor**. It had been moved out of this agent once already; the note + exists to stop a third round-trip. + +## Original prompt + # Profiling Agent phase 3 — `triage --axis compile`: classify compile drift, route the real ones Type: feature diff --git a/active/compile_warm_baseline_dashboard.md b/complete/2026/08/compile-warm-baseline-dashboard.md similarity index 70% rename from active/compile_warm_baseline_dashboard.md rename to complete/2026/08/compile-warm-baseline-dashboard.md index 9b7ca720..0f2f3f48 100644 --- a/active/compile_warm_baseline_dashboard.md +++ b/complete/2026/08/compile-warm-baseline-dashboard.md @@ -1,3 +1,45 @@ +# Compile-axis phase 2 — warm compile made identifiable, pinned, dashboarded + +- shipped: 2026-08-10 +- issues: https://github.com/PyAutoLabs/autolens_profiling/issues/103 +- prs: autolens_profiling#104 (squash 355d555) · PyAutoBrain#220 (squash 74d7b1b) +- repos: autolens_profiling, PyAutoBrain +- arc: phase 2 of 3 + +## Summary + +Tracking **warm** compile is the point of the arc — the persistent cache turned +117.0s into 2.3s (CPU MGE `vag`) and 5517.8s into 937.1s (A100 end-to-end), and +nothing watched for that reverting. Both wins are *settings*. + +`cache_state` is now derived from what the compile DID (cache entries counted +either side of `lowered.compile()`), 25 warm pins are committed, the dashboard +renders through the existing sentinel-block mechanism, and +`ingest --axis compile` reports drift. + +## Traps and findings + +- **`cache_dir` is non-empty on COLD rows too** — the cold run is the one that + populates the cache. Any truthiness check on it, or substring match on the tag, + gets warmness exactly backwards. The corpus carries ~40 ad-hoc tag spellings. +- **`hardware` alone pools different machines.** It is only ever `local_cpu` / + `local_gpu_`, so one `local_cpu` label spanned a laptop (66 records) and a + 32-core RAL node (12). The first rendered dashboard put pins from both side by + side under one heading — the 7x host-load hazard, live. `hostname` joined the + comparability key. +- **Pins must be STICKY.** "Most recent warm row wins" meant re-deriving pins after + a cache regression would move the pin *onto* the regressed value and report + all-clear forever — exactly inverted from the purpose. `--repin` is now required. +- **Rows predating a pin are not drift.** The first `ingest` run flagged four, all + measurements the pin had been *chosen over*. +- Backfill used **end-anchored** tag matching: `mb_homo_cold_laxmap_gpu` contains + "cold" mid-tag and is left `unknown` rather than mislabelled. +- `scripts/misc/test/` **ran in no workflow at all**; a pytest step was added to + `lint.yml` (47 tests, ~2s). Its apparent failure outside CI was a missing + `matplotlib` (imported at module scope by `aggregate.py`), not a defect. + +## Original prompt + # Profiling Agent phase 2 — make warm compile machine-identifiable, pinnable, and dashboarded Type: feature diff --git a/complete/index.md b/complete/index.md index 85da2500..d918722a 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -963 records across 7 buckets. +966 records across 7 buckets. ## Highlights @@ -28,6 +28,9 @@ _(curate hard-won records here — survives regeneration.)_ - [autohands-firewall-allowlist](2026/08/autohands-firewall-allowlist.md) — auto-closed by the merge - [autolens-multiplane-redshift-warning](2026/08/autolens-multiplane-redshift-warning.md) - [autolens-tracer-validation-guards](2026/08/autolens-tracer-validation-guards.md) +- [compile-axis-campaign-coverage](2026/08/compile-axis-campaign-coverage.md) +- [compile-axis-triage-drift](2026/08/compile-axis-triage-drift.md) +- [compile-warm-baseline-dashboard](2026/08/compile-warm-baseline-dashboard.md) - [conductor-discovery-lifecycle-split](2026/08/conductor-discovery-lifecycle-split.md) — closed on merge - [covariance-interpolator-rng-seed](2026/08/covariance-interpolator-rng-seed.md) — auto-closed by the merge - [database-guide-sample-weight-threshold](2026/08/database-guide-sample-weight-threshold.md) — `guides/results/database/start_here.py` ran its own Nautilus fits capped at n_like_max=300 then indexed sample…