diff --git a/docs/00_overview/DASHBOARD.md b/docs/00_overview/DASHBOARD.md index 4bf9f6e4..3474d357 100644 --- a/docs/00_overview/DASHBOARD.md +++ b/docs/00_overview/DASHBOARD.md @@ -6,7 +6,7 @@ _Top-level index across MVP1 → GA v1+ as of **2026-05-29**. Click a release na | Release | Theme | Progress | Status | |---|---|---|---| -| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 94 / 95 scoped done · 4 remaining | **In progress** | +| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 94 / 95 scoped done · 3 remaining | **In progress** | | [MVP2 / v0.2](MVP2_DASHBOARD.md) | Three-Engine + Real Signals | 1 / 1 scoped done · 6 remaining | **In progress** | | MVP3 / v0.3 | Observable | — | **Not yet scoped** | | [GA v1 / v1.0](GA_DASHBOARD.md) | Production-ready | 1 item(s) queued | **Held / queued** | diff --git a/docs/00_overview/MVP1_DASHBOARD.md b/docs/00_overview/MVP1_DASHBOARD.md index 5561bac1..7dedf817 100644 --- a/docs/00_overview/MVP1_DASHBOARD.md +++ b/docs/00_overview/MVP1_DASHBOARD.md @@ -21,19 +21,19 @@ Implementation in progress — resume to finish | Metric | Value | |---|---| | Scoped items done | **94 / 95** (99%) — feat_/infra_/chore_/epic_ past idea stage | -| Pending work | **4** items (every not-done feat/infra/chore/bug across all priorities) | +| Pending work | **3** items (every not-done feat/infra/chore/bug across all priorities) | | → P0 — do next | **0** unblocking / paying daily cost | | → P1 | **0** high-value, ready when P0 clears | -| → P2 (default) | 3 important to file, not blocking | +| → P2 (default) | 2 important to file, not blocking | | → Backlog | 1 captured for record, not planned | -| Open bugs | 2 | -| Legacy "Path to MVP1" | 4 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) | +| Open bugs | 1 | +| Legacy "Path to MVP1" | 3 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) | | Backlog ideas | 0 idea-only feat/infra (not yet scoped into MVP1) | | In flight | 1 feature(s) actively shipping | ## Pipeline -### Done (127) +### Done (128) | Feature | Type | One-liner | Depends on | Status | |---|---|---|---|---| @@ -133,6 +133,7 @@ Implementation in progress — resume to finish | [chore_tutorial_polish](implemented_features/2026_05_12_chore_tutorial_polish/feature_spec.md) | Chore | The release tag `v0.1.0` is pushed with: a worked tutorial at `docs/08_guides/tutorial-first-study.md`, sample data (50-query set + sample ES index of ~1,000 docs from the Amazon ESCI subset), README | `feat_chat_agent` `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_foundation` `infra_optuna_eval` | [PR #64](https://github.com/SoundMindsAI/relyloop/pull/64) merged 2026-05-12 | | [bug_auto_followup_completed_parent_stop_chain_race](implemented_features/2026_05_26_bug_auto_followup_completed_parent_stop_chain_race/idea.md) | Bug | The cycle-3 C3-1 cascade-cancel design tolerates terminal parents (cascade traverses through `completed` intermediates to reach in-flight descendants). But the FR-1 digest trigger fires `enqueue_follo | — | Complete | | [bug_capability_check_test_isolation](implemented_features/2026_05_12_bug_capability_check_test_isolation/idea.md) | Bug | Complete | — | Complete | +| [bug_ceiling_badge_assumes_maximize_direction](implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/idea.md) | Bug | The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](../ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric >= 0.99`. The threshold | — | Complete | | [bug_clone_e2e_seed_template_params_mismatch](implemented_features/2026_05_26_bug_clone_e2e_seed_template_params_mismatch/idea.md) | Bug | Two helpers produce inconsistent fixture state: | — | Complete | | [bug_contract_test_stub_missing_target_filter_kwarg](implemented_features/2026_05_23_bug_contract_test_stub_missing_target_filter_kwarg/idea.md) | Bug | `backend/tests/contract/test_error_codes.py::TestErrorCodes::test_targets_forbidden` and `::test_targets_unreachable_via_adapter` both define an inline `_Stub` class whose `list_targets` method has th | — | Complete | | [bug_cursor_decode_value_validation](implemented_features/2026_05_17_bug_cursor_decode_value_validation/idea.md) | Bug | `backend/app/db/repo/_sort.py:decode_cursor()` performs a `json.loads(base64.urlsafe_b64decode(raw))` round-trip and then takes `decoded[0]` + `str(decoded[1])` without validating the payload shape or | — | Complete | @@ -179,13 +180,12 @@ _None._ _None._ -### Idea (3) +### Idea (2) | # | Priority | Feature | Type | One-liner | Depends on | Status | |---|---|---|---|---|---|---| -| 1 | P2 | [bug_ceiling_badge_assumes_maximize_direction](planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction/idea.md) | Bug | The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric >= 0.99`. The threshold che | — | — | -| 2 | P2 | [bug_smoke_studies_data_table_search_flake](planned_features/01_mvp1/bug_smoke_studies_data_table_search_flake/idea.md) | Bug | [`ui/tests/e2e/studies-data-table.spec.ts:20-40`](../../ui/tests/e2e/studies-data-table.spec.ts#L20-L40): | — | Idea — surfaced during PR #273 CI watch. | -| 3 | Backlog | [chore_demo_reseed_stale_recovery_atomic_cas](planned_features/01_mvp1/chore_demo_reseed_stale_recovery_atomic_cas/idea.md) | Chore | PR #299 added stale-status auto-recovery to the demo-reseed POST handler ([`_test.py`](../backend/app/api/v1/_test.py)): when the Redis status is `running` but `started_at` is older than `DEMO_RESEED_ | — | Idea — captured during PR #299 GPT-5.5 final review (finding #2, adjudicated non-regression) | +| 1 | P2 | [bug_smoke_studies_data_table_search_flake](planned_features/01_mvp1/bug_smoke_studies_data_table_search_flake/idea.md) | Bug | [`ui/tests/e2e/studies-data-table.spec.ts:20-40`](../../ui/tests/e2e/studies-data-table.spec.ts#L20-L40): | — | Idea — surfaced during PR #273 CI watch. | +| 2 | Backlog | [chore_demo_reseed_stale_recovery_atomic_cas](planned_features/01_mvp1/chore_demo_reseed_stale_recovery_atomic_cas/idea.md) | Chore | PR #299 added stale-status auto-recovery to the demo-reseed POST handler ([`_test.py`](../backend/app/api/v1/_test.py)): when the Redis status is `running` but `started_at` is older than `DEMO_RESEED_ | — | Idea — captured during PR #299 GPT-5.5 final review (finding #2, adjudicated non-regression) | ## Dependency graph diff --git a/docs/00_overview/dashboard.html b/docs/00_overview/dashboard.html index 569b9082..666c0ef4 100644 --- a/docs/00_overview/dashboard.html +++ b/docs/00_overview/dashboard.html @@ -384,7 +384,7 @@

Releases

MVP1 / v0.1
The Loop
-
94 / 95 scoped done · 4 remaining
+
94 / 95 scoped done · 3 remaining
In progress
diff --git a/docs/00_overview/planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction/bug_fix.md b/docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/bug_fix.md similarity index 100% rename from docs/00_overview/planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction/bug_fix.md rename to docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/bug_fix.md diff --git a/docs/00_overview/planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction/idea.md b/docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/idea.md similarity index 100% rename from docs/00_overview/planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction/idea.md rename to docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/idea.md diff --git a/docs/00_overview/mvp1_dashboard.html b/docs/00_overview/mvp1_dashboard.html index f564ddb1..8cd854c6 100644 --- a/docs/00_overview/mvp1_dashboard.html +++ b/docs/00_overview/mvp1_dashboard.html @@ -403,12 +403,12 @@

MVP1 Progress

Pending work
-
4
+
3
every not-done feat/infra/chore/bug across all priorities
Open bugs
-
2
+
1
tracked bug_* idea files
@@ -425,7 +425,7 @@

MVP1 Progress

P2 (default)
-
3
+
2
important to file, not blocking
@@ -435,7 +435,7 @@

MVP1 Progress

Legacy "Path to MVP1"
-
4
+
3
scoped not-done + bugs + chore-ideas only (excludes feat/infra ideas)
@@ -463,20 +463,7 @@

Pipeline

-

Idea 3

- -
- -
- Bug - P2 - -
-
The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric >= 0.99`. The threshold che
- - -
- +

Idea 2

@@ -533,7 +520,7 @@

Implementing 1

-

Done 127

+

Done 128

@@ -1783,6 +1770,19 @@

Done 127

+
+ +
+ Bug + + merged 2026-05-29 +
+
The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](../ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric >= 0.99`. The threshold
+ + +
+ +
diff --git a/state.md b/state.md index 8e4397ed..b6e897a8 100644 --- a/state.md +++ b/state.md @@ -20,11 +20,11 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer Detail + reasoning for each is in [`state_history.md`](state_history.md). -- **2026-05-29** — `chore_state_md_size_compression` (PR #303 + finalization #304). Split `state.md` (360 KB → 9.3 KB snapshot) from new `state_history.md` (append-only narrative, root); added `state-md-size-guard` pre-commit hook (60 KB cap) + CLAUDE.md snapshot-vs-history convention. **This is the first merge recorded under the new convention.** +- **2026-05-29** — `bug_ceiling_badge_assumes_maximize_direction` (PR #305 + finalization #306). Studies-list CEILING badge (best_metric ≥ 0.99) mislabeled minimize studies (0.99 is a bad score there). Preflight found it had gone latent→live (feat_study_baseline_trial made `direction=minimize` creatable). Added `direction` to StudySummary (defaults maximize) + gated the badge on `direction !== 'minimize'` (rolling-deploy-safe per Gemini). 7 tests. +- **2026-05-29** — `chore_state_md_size_compression` (PR #303 + finalization #304). Split `state.md` (360 KB → 9.3 KB snapshot) from new `state_history.md` (append-only narrative, root); added `state-md-size-guard` pre-commit hook (60 KB cap) + CLAUDE.md snapshot-vs-history convention. **First merge under the new convention.** - **2026-05-29** — `chore_e2e_api_base_url_construction` (PR #301 + finalization #302). Swept 28 `${API_BASE}` concats across 10 e2e specs to `new URL(...)`; aligned dashboard-reseed's API_BASE env var; URLSearchParams for a query. Mechanical, zero behavior change. - **2026-05-29** — `bug_demo_reseed_button_silent_enqueue_failure` (PR #299 + finalization #300). Top-level `except Exception` barrier in `run_demo_reseed` + `reseed_status_is_stale()` POST auto-recovery so a worker init crash flips Redis to `failed` instead of stuck-`running`. 14 unit tests. - **2026-05-29** — `bug_smoke_seed_es_unavailable_shards_race` (PR #297 + finalization #298). Retry bulk-index on `unavailable_shards_exception` in the ES seed path; fixed the chronic smoke-gate red badge. -- **2026-05-28** — `infra_pr_yml_drop_push_main_trigger` (PR #295 + finalization #296). Dropped the `push: branches: [main]` trigger from `pr.yml`; rewrote release-checklist `gh` queries. ## In flight diff --git a/state_history.md b/state_history.md index 5cfe57d5..6fcce757 100644 --- a/state_history.md +++ b/state_history.md @@ -32,6 +32,8 @@ > Note: this section is significantly behind the actual commit history — PRs #258–#295 are not represented here (the state.md size-compression chore at [`chore_state_md_size_compression`](docs/00_overview/implemented_features/2026_05_29_chore_state_md_size_compression/idea.md) tracks the broader cleanup). New entries land at the top; backfill is out of scope for any single feature PR. +- **2026-05-29 — `bug_ceiling_badge_assumes_maximize_direction` merged into `main` as PR #305.** The studies-list "Ceiling" badge flagged any `best_metric >= 0.99` — correct for maximize objectives (NDCG/MAP/MRR/Precision/Recall pinned at their upper bound) but wrong for minimize objectives, where 0.99 is a *bad* score and the badge claimed the opposite. **Preflight finding — latent→live:** the idea (2026-05-27) deferred on the premise that no minimize study could be created, but `feat_study_baseline_trial` had since added `direction: Literal["maximize","minimize"] = "maximize"` to `ObjectiveSpec` (schemas.py:579), so a `direction=minimize` study is creatable via the API today — the bug actively mislabels it. **Fix (smallest option):** add `direction: ObjectiveDirection = "maximize"` to `StudySummary` (`_summary()` reads `row.objective.get("direction","maximize")` so pre-feature rows default correctly), gate the badge on `direction !== 'minimize'` so a minimize study shows no false badge. `types.ts` hand-edited (CI treats committed types as source of truth). **Cross-model review:** GPT-5.5 final CLEAN; Gemini 1 Medium accepted (use `!== 'minimize'` not `=== 'maximize'` so an absent/undefined direction during a rolling deploy where the FE ships ahead of the BE still defaults to maximize — matches the backend default; added a regression test for the undefined case). 7 regression tests (4 FE column-config + 3 BE `_summary`), FE minimize-case verified failing on `main`. **No migration** (`direction` already lives in the existing `objective` JSONB); additive, backward-compatible, code-only. + - **2026-05-29 — `chore_state_md_size_compression` merged into `main` as PR #303.** Split `state.md` into a one-page snapshot (360 KB / 490 lines → 9.3 KB / 98 lines) + this `state_history.md` (append-only narrative + archived chained execution context, repo root). Per operator decisions (AskUserQuestion): history file at root, last 5 merges retained as one-liners in `state.md`, blocking pre-commit hook added now. Ships `scripts/check-state-md-size.sh` + `.pre-commit-config.yaml` `state-md-size-guard` hook (60 KB cap; portable POSIX `wc -c`; failure message says "move to history, don't raise the cap"; verified PASS@9.3 KB / FAIL@61 KB) + `CLAUDE.md` §"Active Work" + §"Compressed Context First" snapshot-vs-history convention (new merge entries land here, `state.md` keeps only the last 5 one-liners). **Cross-model review:** GPT-5.5 1 Low accepted (error-message section reference points to both CLAUDE.md sections); Gemini 1 Medium accepted (dropped BSD/GNU `stat` branching for single POSIX `wc -c`). The note above (#258–#295 backfill gap) is intentionally left — backfill remains out of scope. **No migration; no production code change** beyond a doc-hygiene script + pre-commit hook. **This is the first feature recorded under the new snapshot-vs-history convention.** - **2026-05-29 — `bug_smoke_seed_es_unavailable_shards_race` merged into `main` as PR #297.** Fixes the chronic smoke-gate red badge that's been blocking every PR since 2026-05-23. **Actual root cause** (found via the smoke-logs artifact from PR #297 run `26612512222`, third CI iteration): ES's disk-watermark allocation gate. GHA runners boot with ~6.8 GB free out of ~71 GB (~9.5%), tripping ES's default 90% high-watermark threshold; ES then refuses to allocate the products-index primary shard — `cluster_status: red`, `active_primary_shards: 0`, `initializing_shards: 0`. The watermark behavior is by-design on multi-node clusters (relocate shards off a full node) but harmful on the single-node dev/CI setup. Fix: `cluster.routing.allocation.disk.threshold_enabled=false` added to the `elasticsearch` service env in `docker-compose.yml`. **Secondary hardening shipped in the same PR** as defense-in-depth for cold-start timing variance: (a) `_cluster/health/?wait_for_status=yellow&timeout=10m` probe between index create and bulk (synchronizes with ES allocation state machine instead of guessing); (b) `_bulk_with_retry` + `_first_bulk_error` helpers extracted as pure functions; (c) retry on `unavailable_shards_exception` only (3 attempts × 2s) — mapping bugs / type mismatches still fail loudly on attempt 1; (d) `RETRYABLE_BULK_ERROR_TYPES` frozenset for source-of-truth allowlist; (e) 408 + `timed_out: true` handled the same as 200 + `timed_out: true` (the documented ES wait_for_status timeout response). 11 unit tests in `backend/tests/unit/scripts/test_seed_es_retry.py` mock httpx to exercise the retry contract: succeed-after-N-transients, exhaust-retries-returns-False, non-retryable-fails-immediately, happy-path-one-call, plus 4 helper tests and 2 constant-pinning tests. **CI**: 9/9 green — first all-green PR run on this branch since the chronic flake began on 2026-05-23 (6 days). **Process reflection**: the iterative trip (3 CI cycles, ~30 minutes of agent + GHA time) on retry budget tuning + health probe design was wasted; the right move would have been to pull the `smoke-logs` artifact on the first failure and read the ES container output before tuning the script. The disk-watermark WARN was right there in the container logs from the start. Lesson: when a fix's expected effect doesn't match the observed CI behavior, the symptom is misleading you about the layer — look at the container/system logs before iterating on the application layer. **Cross-model review**: Gemini Code Assist (1 Medium finding rejected with cited counter-evidence — defensive isinstance ladders on `_first_bulk_error` for hypothetical-only ES response shapes); final GPT-5.5 returned `[]` empty findings (convergence cycle 1). **Tangential discovery captured + filed inline**: [`bug_seed_meaningful_demos_silent_bulk_errors`](docs/00_overview/planned_features/00_unsure/bug_seed_meaningful_demos_silent_bulk_errors/idea.md) — sibling script `scripts/seed_meaningful_demos.py:917-935` has the same race + worse, silently `resp.read()`s the bulk response body without parsing `payload["errors"]`. Affects local `make seed-demo` operators (CI doesn't exercise that path anymore per `chore_drop_demo_seed_from_ci`). Pick up after this fix establishes the retry pattern. **No migration; no Alembic head movement; no production code change** beyond a script + Docker config.