Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/00_overview/DASHBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ _Top-level index across MVP1 → GA v1+ as of **2026-05-29**. Click a release na

| Release | Theme | Progress | Status |
|---|---|---|---|
| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 94 / 95 scoped done · 4 remaining | **In progress** |
| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 94 / 95 scoped done · 3 remaining | **In progress** |
| [MVP2 / v0.2](MVP2_DASHBOARD.md) | Three-Engine + Real Signals | 1 / 1 scoped done · 6 remaining | **In progress** |
| MVP3 / v0.3 | Observable | — | **Not yet scoped** |
| [GA v1 / v1.0](GA_DASHBOARD.md) | Production-ready | 1 item(s) queued | **Held / queued** |
Expand Down
18 changes: 9 additions & 9 deletions docs/00_overview/MVP1_DASHBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,19 @@ Implementation in progress — resume to finish
| Metric | Value |
|---|---|
| Scoped items done | **94 / 95** (99%) — feat_/infra_/chore_/epic_ past idea stage |
| Pending work | **4** items (every not-done feat/infra/chore/bug across all priorities) |
| Pending work | **3** items (every not-done feat/infra/chore/bug across all priorities) |
| → P0 — do next | **0** unblocking / paying daily cost |
| → P1 | **0** high-value, ready when P0 clears |
| → P2 (default) | 3 important to file, not blocking |
| → P2 (default) | 2 important to file, not blocking |
| → Backlog | 1 captured for record, not planned |
| Open bugs | 2 |
| Legacy "Path to MVP1" | 4 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) |
| Open bugs | 1 |
| Legacy "Path to MVP1" | 3 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) |
| Backlog ideas | 0 idea-only feat/infra (not yet scoped into MVP1) |
| In flight | 1 feature(s) actively shipping |

## Pipeline

### Done (127)
### Done (128)

| Feature | Type | One-liner | Depends on | Status |
|---|---|---|---|---|
Expand Down Expand Up @@ -133,6 +133,7 @@ Implementation in progress — resume to finish
| [chore_tutorial_polish](implemented_features/2026_05_12_chore_tutorial_polish/feature_spec.md) | Chore | The release tag `v0.1.0` is pushed with: a worked tutorial at `docs/08_guides/tutorial-first-study.md`, sample data (50-query set + sample ES index of ~1,000 docs from the Amazon ESCI subset), README | `feat_chat_agent` `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_foundation` `infra_optuna_eval` | [PR #64](https://github.com/SoundMindsAI/relyloop/pull/64) merged 2026-05-12 |
| [bug_auto_followup_completed_parent_stop_chain_race](implemented_features/2026_05_26_bug_auto_followup_completed_parent_stop_chain_race/idea.md) | Bug | The cycle-3 C3-1 cascade-cancel design tolerates terminal parents (cascade traverses through `completed` intermediates to reach in-flight descendants). But the FR-1 digest trigger fires `enqueue_follo | — | Complete |
| [bug_capability_check_test_isolation](implemented_features/2026_05_12_bug_capability_check_test_isolation/idea.md) | Bug | Complete | — | Complete |
| [bug_ceiling_badge_assumes_maximize_direction](implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/idea.md) | Bug | The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](../ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric >= 0.99`. The threshold | — | Complete |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The relative link ../ui/src/components/studies/studies-table.column-config.tsx is broken. Since docs/00_overview/MVP1_DASHBOARD.md is located at depth 2 (docs/00_overview/), the correct relative path to the root-level ui/ directory is ../../ui/....

Root Cause

In the source file docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/idea.md (which is at depth 4), several relative links to ui/ and backend/ are incorrect because they only go up 3 levels (../../../) instead of 4 levels (../../../../):

  • Line 17: ../../../ui/src/components/studies/studies-table.column-config.tsx (should be ../../../../ui/...)
  • Line 22: ../../../ui/src/lib/glossary.ts (should be ../../../../ui/...)
  • Line 91: ../../../ui/src/lib/glossary.ts (should be ../../../../ui/...)
  • Line 92: ../../../backend/app/eval/scoring.py (should be ../../../../backend/...)

Because the path rewriter _rewrite_markdown_links preserves the relative target directory structure, it translated the incorrect ../../../ui/... to ../ui/... relative to docs/00_overview/.

How to Fix

  1. Correct the relative links in docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction/idea.md to use ../../../../ instead of ../../../.
  2. Run make dashboard to regenerate the dashboard files.

| [bug_clone_e2e_seed_template_params_mismatch](implemented_features/2026_05_26_bug_clone_e2e_seed_template_params_mismatch/idea.md) | Bug | Two helpers produce inconsistent fixture state: | — | Complete |
| [bug_contract_test_stub_missing_target_filter_kwarg](implemented_features/2026_05_23_bug_contract_test_stub_missing_target_filter_kwarg/idea.md) | Bug | `backend/tests/contract/test_error_codes.py::TestErrorCodes::test_targets_forbidden` and `::test_targets_unreachable_via_adapter` both define an inline `_Stub` class whose `list_targets` method has th | — | Complete |
| [bug_cursor_decode_value_validation](implemented_features/2026_05_17_bug_cursor_decode_value_validation/idea.md) | Bug | `backend/app/db/repo/_sort.py:decode_cursor()` performs a `json.loads(base64.urlsafe_b64decode(raw))` round-trip and then takes `decoded[0]` + `str(decoded[1])` without validating the payload shape or | — | Complete |
Expand Down Expand Up @@ -179,13 +180,12 @@ _None._

_None._

### Idea (3)
### Idea (2)

| # | Priority | Feature | Type | One-liner | Depends on | Status |
|---|---|---|---|---|---|---|
| 1 | P2 | [bug_ceiling_badge_assumes_maximize_direction](planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction/idea.md) | Bug | The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric >= 0.99`. The threshold che | — | — |
| 2 | P2 | [bug_smoke_studies_data_table_search_flake](planned_features/01_mvp1/bug_smoke_studies_data_table_search_flake/idea.md) | Bug | [`ui/tests/e2e/studies-data-table.spec.ts:20-40`](../../ui/tests/e2e/studies-data-table.spec.ts#L20-L40): | — | Idea — surfaced during PR #273 CI watch. |
| 3 | Backlog | [chore_demo_reseed_stale_recovery_atomic_cas](planned_features/01_mvp1/chore_demo_reseed_stale_recovery_atomic_cas/idea.md) | Chore | PR #299 added stale-status auto-recovery to the demo-reseed POST handler ([`_test.py`](../backend/app/api/v1/_test.py)): when the Redis status is `running` but `started_at` is older than `DEMO_RESEED_ | — | Idea — captured during PR #299 GPT-5.5 final review (finding #2, adjudicated non-regression) |
| 1 | P2 | [bug_smoke_studies_data_table_search_flake](planned_features/01_mvp1/bug_smoke_studies_data_table_search_flake/idea.md) | Bug | [`ui/tests/e2e/studies-data-table.spec.ts:20-40`](../../ui/tests/e2e/studies-data-table.spec.ts#L20-L40): | — | Idea — surfaced during PR #273 CI watch. |
| 2 | Backlog | [chore_demo_reseed_stale_recovery_atomic_cas](planned_features/01_mvp1/chore_demo_reseed_stale_recovery_atomic_cas/idea.md) | Chore | PR #299 added stale-status auto-recovery to the demo-reseed POST handler ([`_test.py`](../backend/app/api/v1/_test.py)): when the Redis status is `running` but `started_at` is older than `DEMO_RESEED_ | — | Idea — captured during PR #299 GPT-5.5 final review (finding #2, adjudicated non-regression) |

## Dependency graph

Expand Down
2 changes: 1 addition & 1 deletion docs/00_overview/dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ <h2>Releases</h2>
<div class="roadmap-row">
<div class="release-name"><a href="mvp1_dashboard.html">MVP1 / v0.1</a></div>
<div class="theme">The Loop</div>
<div class="progress">94 / 95 scoped done · 4 remaining</div>
<div class="progress">94 / 95 scoped done · 3 remaining</div>
<span class="state-pill in_progress">In progress</span>
</div>

Expand Down
38 changes: 19 additions & 19 deletions docs/00_overview/mvp1_dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -403,12 +403,12 @@ <h2>MVP1 Progress</h2>
</div>
<div class="kpi warn">
<div class="label">Pending work</div>
<div class="value">4</div>
<div class="value">3</div>
<div class="sub">every not-done feat/infra/chore/bug across all priorities</div>
</div>
<div class="kpi bug">
<div class="label">Open bugs</div>
<div class="value">2</div>
<div class="value">1</div>
<div class="sub">tracked bug_* idea files</div>
</div>
<div class="kpi ">
Expand All @@ -425,7 +425,7 @@ <h2>MVP1 Progress</h2>
</div>
<div class="kpi">
<div class="label">P2 (default)</div>
<div class="value">3</div>
<div class="value">2</div>
<div class="sub">important to file, not blocking</div>
</div>
<div class="kpi">
Expand All @@ -435,7 +435,7 @@ <h2>MVP1 Progress</h2>
</div>
<div class="kpi">
<div class="label">Legacy "Path to MVP1"</div>
<div class="value">4</div>
<div class="value">3</div>
<div class="sub">scoped not-done + bugs + chore-ideas only (excludes feat/infra ideas)</div>
</div>
</div>
Expand Down Expand Up @@ -463,20 +463,7 @@ <h2>Pipeline</h2>
</div>
<div class="kanban">
<div class="col idea">
<h3>Idea <span class="count">3</span></h3>

<div class="card bug" data-prefix="bug" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/planned_features/01_mvp1/bug_ceiling_badge_assumes_maximize_direction">Ceiling Badge Assumes Maximize Direction</a></div>
<div class="meta">
<span class="badge bug">Bug</span>
<span class="badge priority" data-priority="P2">P2</span>

</div>
<div class="one-liner">The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric &gt;= 0.99`. The threshold che</div>


</div>

<h3>Idea <span class="count">2</span></h3>

<div class="card bug" data-prefix="bug" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/planned_features/01_mvp1/bug_smoke_studies_data_table_search_flake">Smoke Studies Data Table Search Flake</a></div>
Expand Down Expand Up @@ -533,7 +520,7 @@ <h3>Implementing <span class="count">1</span></h3>
</div>

<div class="col done">
<h3>Done <span class="count">127</span></h3>
<h3>Done <span class="count">128</span></h3>

<div class="card feat" data-prefix="feat" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_21_feat_agent_propose_search_space/feature_spec.md">Agent Propose Search Space</a></div>
Expand Down Expand Up @@ -1783,6 +1770,19 @@ <h3>Done <span class="count">127</span></h3>
</div>


<div class="card bug" data-prefix="bug" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_29_bug_ceiling_badge_assumes_maximize_direction">Ceiling Badge Assumes Maximize Direction</a></div>
<div class="meta">
<span class="badge bug">Bug</span>

<span>merged 2026-05-29</span>
</div>
<div class="one-liner">The `CEILING` badge in [`studies-table.column-config.tsx:METRIC_CEILING_THRESHOLD`](../ui/src/components/studies/studies-table.column-config.tsx) flags rows where `best_metric &gt;= 0.99`. The threshold </div>


</div>


<div class="card bug" data-prefix="bug" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_26_bug_clone_e2e_seed_template_params_mismatch">Clone E2E Seed Template Params Mismatch</a></div>
<div class="meta">
Expand Down
4 changes: 2 additions & 2 deletions state.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer

Detail + reasoning for each is in [`state_history.md`](state_history.md).

- **2026-05-29** — `chore_state_md_size_compression` (PR #303 + finalization #304). Split `state.md` (360 KB → 9.3 KB snapshot) from new `state_history.md` (append-only narrative, root); added `state-md-size-guard` pre-commit hook (60 KB cap) + CLAUDE.md snapshot-vs-history convention. **This is the first merge recorded under the new convention.**
- **2026-05-29** — `bug_ceiling_badge_assumes_maximize_direction` (PR #305 + finalization #306). Studies-list CEILING badge (best_metric ≥ 0.99) mislabeled minimize studies (0.99 is a bad score there). Preflight found it had gone latent→live (feat_study_baseline_trial made `direction=minimize` creatable). Added `direction` to StudySummary (defaults maximize) + gated the badge on `direction !== 'minimize'` (rolling-deploy-safe per Gemini). 7 tests.
- **2026-05-29** — `chore_state_md_size_compression` (PR #303 + finalization #304). Split `state.md` (360 KB → 9.3 KB snapshot) from new `state_history.md` (append-only narrative, root); added `state-md-size-guard` pre-commit hook (60 KB cap) + CLAUDE.md snapshot-vs-history convention. **First merge under the new convention.**
- **2026-05-29** — `chore_e2e_api_base_url_construction` (PR #301 + finalization #302). Swept 28 `${API_BASE}<path>` concats across 10 e2e specs to `new URL(...)`; aligned dashboard-reseed's API_BASE env var; URLSearchParams for a query. Mechanical, zero behavior change.
- **2026-05-29** — `bug_demo_reseed_button_silent_enqueue_failure` (PR #299 + finalization #300). Top-level `except Exception` barrier in `run_demo_reseed` + `reseed_status_is_stale()` POST auto-recovery so a worker init crash flips Redis to `failed` instead of stuck-`running`. 14 unit tests.
- **2026-05-29** — `bug_smoke_seed_es_unavailable_shards_race` (PR #297 + finalization #298). Retry bulk-index on `unavailable_shards_exception` in the ES seed path; fixed the chronic smoke-gate red badge.
- **2026-05-28** — `infra_pr_yml_drop_push_main_trigger` (PR #295 + finalization #296). Dropped the `push: branches: [main]` trigger from `pr.yml`; rewrote release-checklist `gh` queries.

## In flight

Expand Down
2 changes: 2 additions & 0 deletions state_history.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@

> Note: this section is significantly behind the actual commit history — PRs #258–#295 are not represented here (the state.md size-compression chore at [`chore_state_md_size_compression`](docs/00_overview/implemented_features/2026_05_29_chore_state_md_size_compression/idea.md) tracks the broader cleanup). New entries land at the top; backfill is out of scope for any single feature PR.

- **2026-05-29 — `bug_ceiling_badge_assumes_maximize_direction` merged into `main` as PR #305.** The studies-list "Ceiling" badge flagged any `best_metric >= 0.99` — correct for maximize objectives (NDCG/MAP/MRR/Precision/Recall pinned at their upper bound) but wrong for minimize objectives, where 0.99 is a *bad* score and the badge claimed the opposite. **Preflight finding — latent→live:** the idea (2026-05-27) deferred on the premise that no minimize study could be created, but `feat_study_baseline_trial` had since added `direction: Literal["maximize","minimize"] = "maximize"` to `ObjectiveSpec` (schemas.py:579), so a `direction=minimize` study is creatable via the API today — the bug actively mislabels it. **Fix (smallest option):** add `direction: ObjectiveDirection = "maximize"` to `StudySummary` (`_summary()` reads `row.objective.get("direction","maximize")` so pre-feature rows default correctly), gate the badge on `direction !== 'minimize'` so a minimize study shows no false badge. `types.ts` hand-edited (CI treats committed types as source of truth). **Cross-model review:** GPT-5.5 final CLEAN; Gemini 1 Medium accepted (use `!== 'minimize'` not `=== 'maximize'` so an absent/undefined direction during a rolling deploy where the FE ships ahead of the BE still defaults to maximize — matches the backend default; added a regression test for the undefined case). 7 regression tests (4 FE column-config + 3 BE `_summary`), FE minimize-case verified failing on `main`. **No migration** (`direction` already lives in the existing `objective` JSONB); additive, backward-compatible, code-only.

- **2026-05-29 — `chore_state_md_size_compression` merged into `main` as PR #303.** Split `state.md` into a one-page snapshot (360 KB / 490 lines → 9.3 KB / 98 lines) + this `state_history.md` (append-only narrative + archived chained execution context, repo root). Per operator decisions (AskUserQuestion): history file at root, last 5 merges retained as one-liners in `state.md`, blocking pre-commit hook added now. Ships `scripts/check-state-md-size.sh` + `.pre-commit-config.yaml` `state-md-size-guard` hook (60 KB cap; portable POSIX `wc -c`; failure message says "move to history, don't raise the cap"; verified PASS@9.3 KB / FAIL@61 KB) + `CLAUDE.md` §"Active Work" + §"Compressed Context First" snapshot-vs-history convention (new merge entries land here, `state.md` keeps only the last 5 one-liners). **Cross-model review:** GPT-5.5 1 Low accepted (error-message section reference points to both CLAUDE.md sections); Gemini 1 Medium accepted (dropped BSD/GNU `stat` branching for single POSIX `wc -c`). The note above (#258–#295 backfill gap) is intentionally left — backfill remains out of scope. **No migration; no production code change** beyond a doc-hygiene script + pre-commit hook. **This is the first feature recorded under the new snapshot-vs-history convention.**

- **2026-05-29 — `bug_smoke_seed_es_unavailable_shards_race` merged into `main` as PR #297.** Fixes the chronic smoke-gate red badge that's been blocking every PR since 2026-05-23. **Actual root cause** (found via the smoke-logs artifact from PR #297 run `26612512222`, third CI iteration): ES's disk-watermark allocation gate. GHA runners boot with ~6.8 GB free out of ~71 GB (~9.5%), tripping ES's default 90% high-watermark threshold; ES then refuses to allocate the products-index primary shard — `cluster_status: red`, `active_primary_shards: 0`, `initializing_shards: 0`. The watermark behavior is by-design on multi-node clusters (relocate shards off a full node) but harmful on the single-node dev/CI setup. Fix: `cluster.routing.allocation.disk.threshold_enabled=false` added to the `elasticsearch` service env in `docker-compose.yml`. **Secondary hardening shipped in the same PR** as defense-in-depth for cold-start timing variance: (a) `_cluster/health/<index>?wait_for_status=yellow&timeout=10m` probe between index create and bulk (synchronizes with ES allocation state machine instead of guessing); (b) `_bulk_with_retry` + `_first_bulk_error` helpers extracted as pure functions; (c) retry on `unavailable_shards_exception` only (3 attempts × 2s) — mapping bugs / type mismatches still fail loudly on attempt 1; (d) `RETRYABLE_BULK_ERROR_TYPES` frozenset for source-of-truth allowlist; (e) 408 + `timed_out: true` handled the same as 200 + `timed_out: true` (the documented ES wait_for_status timeout response). 11 unit tests in `backend/tests/unit/scripts/test_seed_es_retry.py` mock httpx to exercise the retry contract: succeed-after-N-transients, exhaust-retries-returns-False, non-retryable-fails-immediately, happy-path-one-call, plus 4 helper tests and 2 constant-pinning tests. **CI**: 9/9 green — first all-green PR run on this branch since the chronic flake began on 2026-05-23 (6 days). **Process reflection**: the iterative trip (3 CI cycles, ~30 minutes of agent + GHA time) on retry budget tuning + health probe design was wasted; the right move would have been to pull the `smoke-logs` artifact on the first failure and read the ES container output before tuning the script. The disk-watermark WARN was right there in the container logs from the start. Lesson: when a fix's expected effect doesn't match the observed CI behavior, the symptom is misleading you about the layer — look at the container/system logs before iterating on the application layer. **Cross-model review**: Gemini Code Assist (1 Medium finding rejected with cited counter-evidence — defensive isinstance ladders on `_first_bulk_error` for hypothetical-only ES response shapes); final GPT-5.5 returned `[]` empty findings (convergence cycle 1). **Tangential discovery captured + filed inline**: [`bug_seed_meaningful_demos_silent_bulk_errors`](docs/00_overview/planned_features/00_unsure/bug_seed_meaningful_demos_silent_bulk_errors/idea.md) — sibling script `scripts/seed_meaningful_demos.py:917-935` has the same race + worse, silently `resp.read()`s the bulk response body without parsing `payload["errors"]`. Affects local `make seed-demo` operators (CI doesn't exercise that path anymore per `chore_drop_demo_seed_from_ci`). Pick up after this fix establishes the retry pattern. **No migration; no Alembic head movement; no production code change** beyond a script + Docker config.
Expand Down
Loading