SoundMindsAI · SoundMindsAI · Jun 1, 2026 · Jun 1, 2026
@@ -2,7 +2,7 @@
 
 > Read this first. A one-page snapshot: current focus, the last few merges, what's in flight, what's queued, and where the project sits in the MVP1 → MVP2 → MVP3 → GA roadmap. **Historical feature-merge narrative + chained execution context lives in [`state_history.md`](state_history.md)** — new merge entries land there, not here (per `chore_state_md_size_compression`, 2026-05-29). Keep this file loadable in a single `Read` call.
 
-**Last updated:** 2026-06-01 (MVP2 backlog batch PR #364 merged — fixes the P1 caplog-isolation bug + 3 other pre-existing bugs and lands 6 cross-model-reviewed spec/plan pairs. Surfaced a P1 Solr-CI-readiness gap, captured as [`infra_solr_ci_readiness`](docs/00_overview/planned_features/02_mvp2/infra_solr_ci_readiness/idea.md) — backend `pr.yml` has no Solr service container + smoke's Solr container crashes; `test_demo_seeding_ubi_full` + `smoke` are red until fixed.)
+**Last updated:** 2026-06-01 (`infra_solr_ci_readiness` Phase 1 merged, PR #367 `214cdfcd` — the demo reseed is now engine-tolerant, so the `pr.yml` **backend** job is GREEN again without a Solr service container. The `smoke` job still crashes on the Solr container (`relyloop-solr-1 exited (1)`) — deferred to Phase 2 (`phase2_idea.md`). The feature folder stays in `planned_features/` until Phase 2 ships.)
 
 ## Where the roadmap sits
 
@@ -14,8 +14,8 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer
 
 ## Current branch / execution context
 
-- **Branch:** `feature/mvp2-backlog-spec-plan-batch` (MVP2 backlog batch, PR #364). Merging over 2 pre-existing Solr-CI reds (captured as `infra_solr_ci_readiness`); everything the PR touches is green (3168 backend tests pass).
-- **Active feature:** _None in flight._ `feat_study_convergence_indicator` shipped — completed studies now carry a plain-language convergence verdict (converged / still_improving / too_few_trials) backed by a best-so-far curve, surfaced on `/studies/[id]`, threaded into the digest narrative's lead recommendation, and exported as a soft contract for the autopilot chain panel. Next: pull from the MVP2 Idea backlog (run `/pipeline status`).
+- **Branch:** `main` (PR #367 `infra_solr_ci_readiness` Phase 1 just merged, `214cdfcd`). The `pr.yml` backend job is green again; the `smoke` job stays red on the deferred Phase-2 Solr-container crash.
+- **Active feature:** _None in flight._ `infra_solr_ci_readiness` Phase 1 shipped — the demo reseed skips unreachable engines, so the backend CI job no longer needs a Solr service container. Phase 2 (smoke Solr stability) is tracked in `infra_solr_ci_readiness/phase2_idea.md`; the folder stays in `planned_features/` until it ships. Next: pull from the MVP2 Idea backlog (run `/pipeline status`).
 - **Alembic head:** `0022_solr_engine_auth_check` (added by `infra_adapter_solr` Story A6 — extends `clusters.engine_type` + `clusters.auth_kind` CHECK constraints for Solr).
 - **Python:** 3.13. **Frontend stack:** Next 16 (App Router + Turbopack), React 19, Tailwind 4 (CSS-first), Vitest 4, ESLint 9 (flat), TypeScript 6, Playwright (chromium, single worker) for E2E.
 - **Coverage gates:** backend 80% (`fail_under` in pyproject), UI vitest + tsc + ESLint + Next build, plus a full-stack smoke E2E job. Live pass counts: see the latest `pr.yml` run (the historical per-feature counts moved to `state_history.md`).
@@ -24,15 +24,16 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer
 
 Detail + reasoning for each is in [`state_history.md`](state_history.md).
 
+- **2026-06-01** — `infra_solr_ci_readiness` Phase 1 (PR #367, squash-merged `214cdfcd`). Makes the demo reseed **engine-tolerant** so the `pr.yml` backend job is green again without a Solr service container: the orchestrator + CLI probe each engine before dispatch (`is_engine_reachable` / `snapshot_engine_reachability`, total never-raises probe, slug-keyed, covers the 5 `SCENARIOS` + the separately-seeded rich ESCI scenario) and **skip** unreachable engines instead of `ConnectError`-ing the whole reseed. Partial completion → `status="complete"` + additive `ReseedStatusResponse.scenarios_skipped` (the `ReseedStatusLiteral` enum is unchanged) + one summary WARN; all-engines-unreachable → typed `AllEnginesUnreachableError` → `status="failed"` + stable `failed_reason="all_engines_unreachable"` token + skip list (avoids the Arq `keep_result` retry wedge). The heavy-lane `test_demo_seeding_ubi_full` now computes counts dynamically (8/8 with Solr absent, 10/10 with Solr). Dashboard reseed dialog shows a "partial completion — N scenario(s) skipped" hint → new runbook. No migration (head stays `0022`). 6 stories / 1 epic; 2095 unit + 327 contract + 998 UI vitest. Cross-model: GPT-5.5 phase-gate (5 findings — incl. the all-unreachable path not being total until the pre-loop wipes were gated) + Gemini (3, probe-caching) + GPT-5.5 final (2 fixed, 1 rejected as a stdlib-vs-structlog mis-read) — all adjudicated, CI-verified. **Phase 2 (smoke Solr stability) deferred** to `phase2_idea.md`; the `smoke` job stays red. Captured `chore_demo_reseed_partial_completion_fast_test`.
 - **2026-06-01** — MVP2 backlog batch (PR #364). Fixed the **P1 caplog-isolation bug** (`bug_backend_suite_nondeterministic_caplog_isolation`): root cause was `configure_logging()` replacing the structlog processors *list instance* on every call (not `pytest-randomly`, which isn't installed) — loggers cached against the stale instance went blind to `capture_logs()`; fixed by mutating the list in place + fixing a second polluter fixture in `test_position_bias_prior.py`; regression test added. Also fixed 3 other pre-existing bugs to green the suite: `bug_contract_allowlists_outdated_after_mvp2_features` (3 stale `'solr'`/endpoint allowlists), the `test_judgment_generate` `click:0` source-breakdown drift, and the `test_migration_0021` downgrade target (was `-1` assuming 0021 was head; head is now 0022). Landed **6 cross-model-reviewed spec/plan pairs** (idea-preflight → spec-gen → impl-plan-gen, each GPT-5.5-reviewed) for `feat_apply_path_normalizer_declaration` (design-ahead), `feat_overnight_studies_summary_card`, `feat_query_normalizer_typed_pipeline` (design-ahead), `infra_generated_artifact_freshness_gate`, `chore_arq_pool_aclose_deprecation`, `chore_cluster_detail_rung_badge`. 3168 backend tests pass; merged over 2 pre-existing Solr-CI reds (`infra_solr_ci_readiness`). Gemini: 1 finding (defensive fixture guard) accepted.
 - **2026-06-01** — `feat_study_convergence_indicator` (PR #352, squash-merged `0eee17a9`). MVP2 ergonomics feature: every completed study carries a plain-language **convergence verdict** — `converged` / `still_improving` / `too_few_trials` — backed by a best-so-far metric curve, answering "did the optimizer finish learning, or did I stop it too early?" **No migration** (reads existing `trials` columns; Alembic head stays `0022`). 11 stories / 7 epics: (1) hoist the lift epsilon to `AUTO_FOLLOWUP_LIFT_EPSILON` + an **AST/grep guard** that fails CI on any bare `0.005` in a lift/epsilon-shaped context under `backend/app/` — surfaced + collapsed a pre-existing duplicate in `chain_summary.py`; (2) pure-domain `classify_convergence(...)` → `StudyConvergenceShape | None` (trailing-window-flat, direction-aware running max/min, window clamp `min(20, max(5, total//5))`, warmup floor 50); (3) repo helper `list_complete_optuna_trials_for_study` (`status='complete' AND is_baseline IS NOT TRUE AND primary_metric IS NOT NULL`); (4) service `fetch_study_convergence` with in-flight short-circuit + invalid-direction WARN + classifier-exception shielding (GET never 500s); (5) additive `StudyDetail.convergence` (renamed class `StudyConvergenceShape` to coexist with `confidence.py`'s `ConvergenceShape` — winner-trial timing vs metric plateau — without OpenAPI name collision); (6) `ConvergencePanel` (Recharts curve + `ReferenceArea` trailing-window shade + 3 null-state badges + 3 glossary entries deep-linking the runbook) mounted on `/studies/[id]` with the `CONVERGENCE_VERDICT_VALUES` enum-discipline pair value-locked both sides; (7) digest worker + Jinja `<convergence>` block + system-prompt rule that leads with "re-run with a larger trial budget" on `still_improving` / `too_few_trials`; (8) `ConvergenceVerdict` exported as the FR-7 soft contract for the autopilot chain panel (AC-16 lives in that PR's lane — no autopilot files touched here); (9) operator runbook + CLAUDE.md Key Runbooks row + `data-model.md` / `ui-architecture.md` patches. Tests: 40 domain unit + 11 prompt unit + 5 docs unit + 18 integration + 4 contract + 13 frontend vitest + 1 Playwright smoke; mypy clean. Cross-model: Gemini (1 Medium accepted — `is False`→`not getattr` None-safety, `644feeed`) + GPT-5.5 final (4 findings: SQL/domain filter-mirror `is_not(True)` accepted `ad72e297`; 3 rejected as review-window-truncation artifacts). Captured `bug_contract_allowlists_outdated_after_mvp2_features` (3 pre-existing contract failures from prior MVP2 PRs not updating hand-maintained allowlists).
 - **2026-05-31** — `feat_demo_reseed_solr_and_steplog` (PR #348, squash-merged `66323aba`). Completes the deferred `infra_adapter_solr` Story A13 — the async home-button demo reseed (`make seed-demo`) now seeds the **Solr** scenario (`acme-kb-docs-solr`) end-to-end, where before it crashed. Fixed layer by layer (each verified by the live reseed getting further): Solr host-URL→Compose-DNS mapping (`8983`); the dispatcher now creates the Solr collection from its configset + bulk-indexes (reusing `seed_solr_products`) instead of the ES `index_mapping` PUT; engine-aware **synthetic-UBI write** (`ubi_queries`/`ubi_events` Solr collections + Solr `/update`, per-engine ensure gate); study **search-space cardinality** (Solr scenario tunes 2 boosts, `estimate_cardinality` floats=100 → ≤3-float cap). **Surfaced + fixed a real product bug:** the UBI **read** path (`ubi_readiness`, `UbiReader`) built Elasticsearch query DSL → `UBI judgment generation was broken on Solr for every operator` (contradicting the "works everywhere from day one" claim); now builds Solr `q`/`fq`/`{!terms}`/`rows`/`fl` when `adapter.engine_type=="solr"`. Plus a **reseed step-history log** (worker accumulates `steps[]` → status endpoint → scrolling UI panel). **Verified by a live `make seed-demo` completing 6/6** (10 studies, 10 proposals). 8 commits; backend 2012 unit + frontend 983 vitest. Gemini: 3 findings accepted+fixed (`{!terms}` query_id vs maxBooleanClauses — verified live; collision-safe synthetic ids; scroll-on-reopen dep). Final GPT-5.5 review clean. Captured `bug_reseed_failure_blocks_retry_arq_singleton_dedup` (a failed reseed deduped retries for ~1h via the Arq singleton result).
 - **2026-05-31** — `feat_overnight_autopilot` (PR #343, squash-merged `fe146950`). MVP2 ergonomics feature surfacing the shipped auto-followup chaining engine as a first-class "set it and wake up to results" path — **read-side + UI only, the engine stayed untouched**. New read-only `GET /api/v1/studies/{id}/chain` (pure-domain `chain_summary.py`: stop-reason matrix, universal cumulative-lift, best-link selection — reuses `compute_first_decile_max`/`_direction_normalized_lift` from `auto_followup.py`; + a bounded `parent_study_id` traversal repo helper with a 10-hop upward cap + cycle guard, `LIMIT 1` downward walk, `DISTINCT ON` newest-non-rejected proposal lookup). Frontend: create-study wizard relabel to "🌙 Run overnight (compound automatically)" + Deep-preset discoverability hint + `overnight_autopilot` glossary key; `AutoFollowupChainPanel` rolled-up summary (ordered links + deltas, cumulative lift, 3-branch best-config, stop-reason phrases) via a new `useStudyChain` TanStack hook (focus/cancel/transition refetch + 120s grace poll). Tutorial Step 12 + arch-doc notes. No migration (Alembic head stays `0022`). 7 stories / 4 epics. Tests: 30 unit + 31 chain integration/contract + frontend vitest (panel 13/wizard/glossary) + 1 real-backend E2E. Cross-model: GPT-5.5 Epic 1 (1 Low accepted) + Epics 2+3 clean + final (1 Medium rejected — SQLite-portability moot, Postgres-only); Gemini (1 High accepted — zero-row hydration guard `9b1d894f`). Phase 2 ("ran while away" card) deferred to `feat_overnight_studies_summary_card`. 4 tangential idea files captured (`infra_generated_artifact_freshness_gate`, `bug_judgment_header_omits_click_bucket`, `chore_arq_pool_aclose_deprecation`, `bug_e2e_teardown_chain_node_delete_500`).
-- **2026-05-31** — `infra_adapter_solr` (PR #336 squash-merged `60aec9af` + demo-seed fix #337). MVP2 three-engine headliner: a single `SolrAdapter` implements the full `SearchAdapter` Protocol against Apache Solr 9.x/10.x (SolrCloud + standalone), pivoting on a construction-time capability probe. All 13 stories (A1–A13): adapter skeleton + probe, `edismax`/`dismax`/`lucene` render with unified-param pivots, parallel `/select` search_batch, get_schema/list_targets, explain (Lucene-escaped), get_document/list_documents (RealTime Get + cursorMark), LTR rescore (`rq={!ltr}`) + `LTR_MODEL_NOT_FOUND`, migration 0022 (engine_type + auth_kind CHECK extension) + `registry.py` allowlist relocation, `POST /clusters/{id}/reprobe` + `POST /clusters/test-connection`, Compose `solr:10.0` service + configsets + `/healthz` subsystems.solr, frontend wire literals + per-engine auth filtering + 3-engine `<EngineBadge>` + test-connection button, Guide 01 + runbook + tutorial Path C, and a 5th `acme-kb-docs-solr` demo scenario. **Live-Solr rework correction** (folded into the squash): local Solr runs security-DISABLED (parity with ES/OpenSearch), `bootstrap-security.sh` deleted, LTR via `SOLR_MODULES=ltr`; stock Solr ships **no** `solr.UBIComponent`, so UBI on Solr is read-path-only (demo synthesizes events, probe reports `ubi_component_present=false`). Cross-model review: GPT-5.5 F1/F2/F3 + 6 Gemini findings adjudicated. Post-pipeline followups tracked in `00_unsure/chore_solr_post_pipeline_followups` + `chore_solr_cred_backfill_needs_api_restart`.
+_(older entries — full narrative in [`state_history.md`](state_history.md): `infra_adapter_solr` PR #336, …)_
 
 ## In flight
 
-- _None._ MVP2 backlog batch (PR #364) merged 2026-06-01. The 6 spec/plan pairs it landed are now at **Plan** stage in `02_mvp2/` (two are design-ahead: `feat_apply_path_normalizer_declaration` + `feat_query_normalizer_typed_pipeline`, both gated on `feat_query_normalization_tuning` Phase 1 merging — do not `/impl-execute` until then).
+- _None._ `infra_solr_ci_readiness` Phase 1 (PR #367) merged 2026-06-01; its Phase 2 (smoke Solr stability) is Idea-stage in `02_mvp2/infra_solr_ci_readiness/phase2_idea.md`. The 6 spec/plan pairs from PR #364 remain at **Plan** stage in `02_mvp2/` (two are design-ahead: `feat_apply_path_normalizer_declaration` + `feat_query_normalizer_typed_pipeline`, both gated on `feat_query_normalization_tuning` Phase 1 merging — do not `/impl-execute` until then).
 
 ## Queued (priority-ordered by dashboard / dep graph)
 
@@ -48,7 +49,7 @@ Detail + reasoning for each is in [`state_history.md`](state_history.md).
 
 ## Known debt / fragility
 
-- **Solr is not CI-ready (P1) — `pr.yml` backend + smoke are red on every branch.** The `backend` job has no Solr service container, so `test_demo_seeding_ubi_full` `ConnectError`s (the reseed seeds the Solr scenario unconditionally since PR #348); and the `smoke` job's `solr` container crashes (`exited (1)`) during `make up`. Pre-existing since Solr shipped; tracked in [`infra_solr_ci_readiness`](docs/00_overview/planned_features/02_mvp2/infra_solr_ci_readiness/idea.md). PR #364 merged over these two reds (everything else green, 3168 backend tests pass).
+- ~~**Solr is not CI-ready (P1) — `pr.yml` backend + smoke are red on every branch.**~~ — **Backend half RESOLVED** (`infra_solr_ci_readiness` Phase 1, PR #367, 2026-06-01). The demo reseed is now engine-tolerant — it probes each engine and skips unreachable ones — so the `backend` job's `test_demo_seeding_ubi_full` passes (8/8 with Solr absent) and the backend job is green again. **Smoke half still open (Phase 2):** the `smoke` job's `solr` container still crashes (`relyloop-solr-1 exited (1)`) during `make up`; deferred to [`infra_solr_ci_readiness/phase2_idea.md`](docs/00_overview/planned_features/02_mvp2/infra_solr_ci_readiness/phase2_idea.md) (needs `docker compose logs solr` runner evidence to pick the stabilization lever — heap-cap / start_period / smoke-tolerance). Until Phase 2, `smoke` stays red on every PR; merge on the fast lane as before.
 
 - ~~**`backend/app/eval/qrels_loader.py` is an MVP1 stub.**~~ — **Resolved.** PR #35 replaced the stub with a real `SELECT query_id, doc_id, rating FROM judgments WHERE judgment_list_id = :id`. The legacy `JudgmentsTableMissing` symbol is retained as a no-op compat shim for any imported reference in older tests. Integration tests now seed real `judgments` rows; `run_trial` consumes the loader directly.
 - **`infra_optuna_orphan_reaper`** — Phase 2 orchestrator can die between `study.ask()` and the enqueue commit, leaving orphan Optuna RUNNING trials. Operationally tolerated for MVP1 per spec §11 "Operational tolerance"; periodic reaper deferred to MVP3 ([`03_mvp3/infra_optuna_orphan_reaper`](docs/00_overview/planned_features/03_mvp3/infra_optuna_orphan_reaper/idea.md)).