From a5f6c51de7155ac4933db8fdd23fff89a13be49e Mon Sep 17 00:00:00 2001 From: SoundMindsAI Date: Tue, 12 May 2026 12:23:36 -0400 Subject: [PATCH 01/18] docs(feat-chat-agent): /idea-preflight ground-truth pass against post-MVP1-backend codebase MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Correct backend paths from `backend/agent/` to `backend/app/agent/` (4 spots: §3, FR-4, §7.4, agent-tools.md) - Correct frontend paths from `ui/lib/api/` to `ui/src/lib/api/` and `ui/tests/unit/` to `ui/src/__tests__/` - Tool count corrected from 18 to 19 (3 + 2 + 5 + 1 + 3 + 5; see FR-4 + agent-tools.md inventory) - Settings citations canonicalized: `settings.openai_model_chat` (lowercase, line 117), `openai_base_url` (line 108) - Capability cache citation tightened to `read_capability_result(redis_client, base_url)` at `capability_check.py:372` - §9 expanded with column inventory + Alembic revision `0007_conversations_messages` + `deleted_at` soft-delete (per CLAUDE.md convention) - §11 multi-tab "Open question" resolved against the 2026-05-09 laissez-faire decision - §14 added `test_conversations_migration.py` for Alembic round-trip (mirroring feat_llm_judgments) - §7.4 cited the verify_enum_source_of_truth.sh CI gate that feat_studies_ui Story 4.2 shipped - §1 tutorial-flow language aligned to `local-es` (MVP1 has no staging) - data-model.md `conversations` schema: `started_at` → `created_at` + add `deleted_at TIMESTAMPTZ` (CLAUDE.md soft-delete convention) - mvp1_dashboard regenerated by pre-commit hook (planned-features count unchanged) Captured in §19 Decision log under 2026-05-12 entry. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/00_overview/MVP1_DASHBOARD.md | 2 +- docs/00_overview/mvp1_dashboard.html | 2 +- docs/01_architecture/agent-tools.md | 8 +- docs/01_architecture/data-model.md | 5 +- .../feat_chat_agent/feature_spec.md | 115 +++++++++++------- 5 files changed, 83 insertions(+), 49 deletions(-) diff --git a/docs/00_overview/MVP1_DASHBOARD.md b/docs/00_overview/MVP1_DASHBOARD.md index 7703b68e..45f728eb 100644 --- a/docs/00_overview/MVP1_DASHBOARD.md +++ b/docs/00_overview/MVP1_DASHBOARD.md @@ -55,7 +55,7 @@ _None._ | Feature | Type | One-liner | Depends on | Status | |---|---|---|---|---| -| [feat_chat_agent](../02_product/planned_features/feat_chat_agent/feature_spec.md) | Feature | A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. | `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_arq_subprocess_test` `infra_ci_smoke_makeup` `infra_foundation` `infra_frontend_stack_refresh` `infra_nvmrc` `infra_optuna_eval` `infra_per_trial_timeout` | Draft | +| [feat_chat_agent](../02_product/planned_features/feat_chat_agent/feature_spec.md) | Feature | A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. | `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_arq_subprocess_test` `infra_ci_smoke_makeup` `infra_foundation` `infra_frontend_stack_refresh` `infra_nvmrc` `infra_optuna_eval` `infra_per_trial_timeout` | [PR #4](https://github.com/SoundMindsAI/relyloop/pull/4) merged 2026-05-09 | | [chore_tutorial_polish](../02_product/planned_features/chore_tutorial_polish/feature_spec.md) | Chore | The release tag `v0.1.0` is pushed with: a worked tutorial at `docs/08_guides/tutorial-first-study.md`, sample data (50-query set + pre-baked judgment list + sample ES index of ~1,000 docs), README po | `feat_chat_agent` `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_arq_subprocess_test` `infra_ci_smoke_makeup` `infra_foundation` `infra_frontend_stack_refresh` `infra_nvmrc` `infra_optuna_eval` `infra_per_trial_timeout` | Draft | ### Idea (22) diff --git a/docs/00_overview/mvp1_dashboard.html b/docs/00_overview/mvp1_dashboard.html index c6fe5c59..0d5abfa8 100644 --- a/docs/00_overview/mvp1_dashboard.html +++ b/docs/00_overview/mvp1_dashboard.html @@ -646,7 +646,7 @@

Spec 2

Chat Agent
Feature - + PR #4merged 2026-05-09
A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE.
diff --git a/docs/01_architecture/agent-tools.md b/docs/01_architecture/agent-tools.md index 6d301565..6cd8ba4b 100644 --- a/docs/01_architecture/agent-tools.md +++ b/docs/01_architecture/agent-tools.md @@ -11,7 +11,7 @@ ## MVP1 tool inventory -The MVP1 agent ships these tools in `backend/agent/tools/`: +The MVP1 agent ships these **19** tools in `backend/app/agent/tools/` (counted across the 6 categories below: 3 + 2 + 5 + 1 + 3 + 5 = 19): | Category | Tool | Description | Backing endpoint / function | |---|---|---|---| @@ -38,7 +38,7 @@ The MVP1 agent ships these tools in `backend/agent/tools/`: ## Tool definition pattern ```python -# backend/agent/tools/studies.py +# backend/app/agent/tools/studies.py from pydantic import BaseModel from openai.types.chat import ChatCompletionToolParam @@ -60,12 +60,12 @@ CANCEL_STUDY_TOOL: ChatCompletionToolParam = { } ``` -The tool registry at `backend/agent/tools/__init__.py` collects every tool's `*_TOOL` constant into a `TOOLS` list and every `*_impl` function into a `TOOL_REGISTRY: dict[str, Callable]` dispatcher. +The tool registry at `backend/app/agent/tools/__init__.py` collects every tool's `*_TOOL` constant into a `TOOLS` list and every `*_impl` function into a `TOOL_REGISTRY: dict[str, Callable]` dispatcher. ## Tool dispatch loop (MVP1) ```python -# backend/agent/orchestrator.py (sketch) +# backend/app/agent/orchestrator.py (sketch) from openai import AsyncOpenAI from .tools import TOOLS, TOOL_REGISTRY diff --git a/docs/01_architecture/data-model.md b/docs/01_architecture/data-model.md index fc34b259..424518e2 100644 --- a/docs/01_architecture/data-model.md +++ b/docs/01_architecture/data-model.md @@ -261,7 +261,8 @@ CREATE TABLE proposals ( CREATE TABLE conversations ( id UUID PRIMARY KEY, title TEXT, -- auto-generated from first message; nullable - started_at TIMESTAMPTZ NOT NULL DEFAULT now() + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + deleted_at TIMESTAMPTZ -- soft-delete per CLAUDE.md convention ); CREATE TABLE messages ( @@ -274,6 +275,8 @@ CREATE TABLE messages ( ); ``` +Soft-delete on `conversations` filters the row out of `GET /api/v1/conversations` and `GET /api/v1/conversations/{id}` (`deleted_at IS NULL` predicate); messages remain joined via the FK so a future hard-purge runbook can drop both atomically. Hard delete cascades to messages via `ON DELETE CASCADE`. + ## Forthcoming: `audit_log` (MVP2 + MVP4 evolution) Documented here so MVP2 authoring has a target. Not in MVP1. diff --git a/docs/02_product/planned_features/feat_chat_agent/feature_spec.md b/docs/02_product/planned_features/feat_chat_agent/feature_spec.md index 0267209f..123f692a 100644 --- a/docs/02_product/planned_features/feat_chat_agent/feature_spec.md +++ b/docs/02_product/planned_features/feat_chat_agent/feature_spec.md @@ -15,42 +15,52 @@ ## 1) Purpose -- **Problem:** Without a chat surface, every operation requires the UI's structured forms. Chat lets the engineer describe the goal in plain language ("tune product_search overnight on staging-products-es") and let the agent translate that to API calls. -- **Outcome:** A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. The agent has a tool registry covering the 18 MVP1 tools (per [`agent-tools.md`](../../../01_architecture/agent-tools.md)). Conversation state persists in `conversations` + `messages` tables; tool calls are visible in expandable panels. +- **Problem:** Without a chat surface, every operation requires the UI's structured forms. Chat lets the engineer describe the goal in plain language ("tune product_search overnight on local-es") and let the agent translate that to API calls. +- **Outcome:** A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. The agent has a tool registry covering the 19 MVP1 tools (per [`agent-tools.md`](../../../01_architecture/agent-tools.md) §"MVP1 tool inventory"). Conversation state persists in `conversations` + `messages` tables; tool calls are visible in expandable panels. - **Non-goal:** No `propose_search_space` LLM tool (deferred to MVP2 per Decision log). No LangGraph state graph or subagents (GA v1). No human-in-the-loop interrupts (GA v1). No Fusion-specific tools (MVP3). No `fork_study` (MVP2). No multi-conversation parallelism enforcement (MVP4 with multi-tenant). ## 2) Current state audit -After all dependencies ship: -- Every consumed API endpoint exists. -- `conversations` and `messages` tables do NOT exist yet — this feature creates them in their full MVP1 shape per [`data-model.md`](../../../01_architecture/data-model.md). They are terminal (no other features depend on them). -- The Next.js skeleton + layout + nav (per `infra_foundation` + `feat_studies_ui`) include a `/chat` link that goes nowhere yet. -- `openai` Python SDK is installed; no LLM calls are being made yet beyond `feat_llm_judgments` and `feat_digest_proposal`. +As of 2026-05-12, every dependency has shipped: + +- **Every consumed API endpoint exists** and is verified by contract tests: + - `GET /api/v1/clusters`, `GET /api/v1/clusters/{id}`, `GET /api/v1/clusters/{id}/schema`, `POST /api/v1/clusters/{id}/run_query` (all in `backend/app/api/v1/clusters.py` from `infra_adapter_elastic`). + - `GET /api/v1/query-templates`, `GET /api/v1/query-templates/{id}` (from `feat_study_lifecycle` Phase 2). + - `GET /api/v1/query-sets`, `POST /api/v1/query-sets`, `POST /api/v1/query-sets/{id}/queries` (from `feat_study_lifecycle` Phase 2). + - `POST /api/v1/judgments/generate`, `GET /api/v1/judgment-lists/{id}` (from `feat_llm_judgments`). + - `POST /api/v1/studies`, `GET /api/v1/studies/{id}`, `POST /api/v1/studies/{id}/cancel` (from `feat_study_lifecycle` Phase 2). + - `GET /api/v1/proposals`, `GET /api/v1/proposals/{id}`, `POST /api/v1/proposals`, `POST /api/v1/proposals/{id}/open_pr` (from `feat_digest_proposal` + `feat_github_pr_worker`). +- `conversations` and `messages` tables do NOT exist yet — this feature creates them via Alembic revision `0007_conversations_messages` (the next sequential after `0006_proposals_pr_url_idx`), in the full MVP1 shape per [`data-model.md`](../../../01_architecture/data-model.md). They are terminal (no other features depend on them). +- The Next.js shell + nav include a `/chat` link in `ui/src/components/layout/top-nav.tsx:14` that currently routes to a 404 — this feature fills it (mirroring the `/proposals` shipping pattern from `feat_proposals_ui`). +- The OpenAI client + capability check (`backend/app/llm/openai_judge.py`, `backend/app/llm/capability_check.py`) and budget gate (`backend/app/llm/budget_gate.py`) are in place from `feat_llm_judgments`; this feature reuses them. +- `prompts/` already contains the system + user templates for `feat_llm_judgments` and `feat_digest_proposal`; this feature adds `prompts/orchestrator.system.md`. +- `backend/app/agent/` does NOT exist yet — this feature creates it. ## 3) Scope ### In scope - **Backend**: - - Tool registry at `backend/agent/tools/__init__.py` collecting the 18 MVP1 tools per [`agent-tools.md`](../../../01_architecture/agent-tools.md). Each tool is one Python module under `backend/agent/tools//.py`. - - Orchestrator at `backend/agent/orchestrator.py` running the OpenAI function-calling loop (per [`llm-orchestration.md`](../../../01_architecture/llm-orchestration.md)). + - Tool registry at `backend/app/agent/tools/__init__.py` collecting the 19 MVP1 tools per [`agent-tools.md` §"MVP1 tool inventory"](../../../01_architecture/agent-tools.md). Each tool is one Python module under `backend/app/agent/tools//.py`. + - Orchestrator at `backend/app/agent/orchestrator.py` running the OpenAI function-calling loop (per [`llm-orchestration.md`](../../../01_architecture/llm-orchestration.md)). - SSE endpoint `POST /api/v1/conversations/{id}/messages` accepting a user message and streaming the assistant turn (token + tool_call + tool_result + done events per [`agent-tools.md` §"Streaming + SSE"](../../../01_architecture/agent-tools.md)). - REST endpoints: - `POST /api/v1/conversations` — create new conversation (returns `conversation_id`) - - `GET /api/v1/conversations` — list (paginated) + - `GET /api/v1/conversations` — list (cursor-paginated, soft-deleted rows filtered out) - `GET /api/v1/conversations/{id}` — detail with full message history - - `DELETE /api/v1/conversations/{id}` — soft-delete + - `DELETE /api/v1/conversations/{id}` — soft-delete (sets `deleted_at`; messages preserved via FK CASCADE on hard purge only) + - Router file at `backend/app/api/v1/conversations.py` (matches the project's `backend/app/api/v1/.py` convention); registered in `backend/app/main.py` with `prefix="/api/v1"`. - System prompt at `prompts/orchestrator.system.md` framing the agent's role + the available tools. - Per-call validation (Pydantic args schema) before dispatch; validation failures appended as tool_result with error payload (LLM gets a retry). - **Frontend** (`/chat/{conversation_id}` route in `feat_studies_ui`'s Next.js app): - Conversation list at `/chat` (sidebar). "New conversation" button. - - Single-conversation view at `/chat/{conversation_id}`: + - Single-conversation view at `/chat/[id]/page.tsx`: - Message stream rendered in chronological order (user / assistant / tool messages). - Composer at the bottom (auto-grows; cmd+enter to send). - Tool calls rendered as collapsible ``s (default collapsed) showing `name` + `arguments` (JSON). Tool results render as a sibling `` (default collapsed) showing the JSON response. - Token streaming: assistant text appears character-by-character as the SSE delivers it. - - Errors (network, 4xx, 5xx) surface as toast + an inline `` below the conversation. - - Streaming consumer in `ui/lib/api/conversations.ts` using `fetch()` with `ReadableStream` (per [`ui-architecture.md` §"Streaming chat"](../../../01_architecture/ui-architecture.md)). **Native `EventSource` is NOT used** — it's GET-only and the user message belongs in the POST body. + - Errors (network, 4xx, 5xx) surface as toast (via the central `MutationCache.onError` wiring shipped by `feat_studies_ui`) + an inline `` below the conversation. + - Streaming consumer in `ui/src/lib/api/conversations.ts` using `fetch()` with `ReadableStream` (per [`ui-architecture.md` §"Streaming chat"](../../../01_architecture/ui-architecture.md), lines 186-226). **Native `EventSource` is NOT used** — it's GET-only and the user message belongs in the POST body. ### Out of scope @@ -83,15 +93,23 @@ Single-phase. The MVP1 deliverable: "the operator opens chat, types 'tune produc - **Do not** allow the agent to mutate without confirmation (per system prompt). If the LLM tries to call `create_study` without the user's explicit "yes," dispatch returns an error appended as a tool_result; the LLM gets the message and asks for confirmation. - **Do not** use the same OpenAI model as judgment generation (`gpt-4o-2024-08-06`). Chat orchestration is cost-sensitive; use `gpt-4o-mini-2024-07-18`. -- **Do not** invent tools. The 18 MVP1 tools per [`agent-tools.md`](../../../01_architecture/agent-tools.md) are the complete set. New tools require their own feature spec. +- **Do not** invent tools. The 19 MVP1 tools per [`agent-tools.md` §"MVP1 tool inventory"](../../../01_architecture/agent-tools.md) are the complete set. New tools require their own feature spec. - **Do not** dispatch tool calls in parallel. Sequential dispatch keeps the conversation auditable; the agent rarely needs parallelism. (LangGraph at GA v1 introduces parallel subagents.) - **Do not** stream tool RESULTS to the client mid-execution. Wait for the tool to complete, then stream the result event. (This avoids partial-result races.) ## 5) Assumptions and dependencies -- **Dependency: ALL backend features** — the tool registry dispatches into every feature's API. Any feature that ships late breaks tools. -- **Dependency: `feat_studies_ui`** — provides the layout shell + nav + TanStack Query setup that the chat UI plugs into. -- **OpenAI API key** — required at chat time (returns `OPENAI_NOT_CONFIGURED` otherwise per `feat_llm_judgments` precedent). +All dependencies shipped between 2026-05-09 and 2026-05-12; this section captures what each one delivered that this feature consumes. + +- **`infra_foundation` (PR #4, merged 2026-05-09)** — `OPENAI_BASE_URL` + `openai_api_key_file` + `openai_model_chat` settings; capability cache infrastructure (`backend/app/llm/capability_check.py:read_capability_result(redis, base_url)`) populated at startup. +- **`infra_adapter_elastic` (PR #16, merged 2026-05-10)** — backs `list_clusters`, `get_cluster`, `get_schema`, `run_query` tools via `backend/app/api/v1/clusters.py`. +- **`feat_study_lifecycle` Phase 2 (PR #25, merged 2026-05-11)** — backs `list_templates`, `get_template`, `list_query_sets`, `create_query_set`, `import_queries_from_csv`, `create_study`, `get_study`, `cancel_study` via `backend/app/api/v1/query_templates.py`, `query_sets.py`, `studies.py`. +- **`feat_llm_judgments` (PR #35, merged 2026-05-11)** — backs `generate_judgments_llm` + `get_calibration` via `backend/app/api/v1/judgments.py`. Also: the OpenAI client + budget gate + capability check are reused here; `OPENAI_NOT_CONFIGURED` (503) + `OPENAI_BUDGET_EXCEEDED` (503) error precedents apply. +- **`feat_digest_proposal` (PR #41, merged 2026-05-11)** — backs `list_proposals`, `get_proposal`, `create_proposal_from_study`, `create_proposal_manual` via `backend/app/api/v1/proposals.py`. +- **`feat_github_pr_worker` (PR #45, merged 2026-05-12)** — backs `open_pr` via `POST /api/v1/proposals/{id}/open_pr`. The tool surfaces all 5 error codes the endpoint can return (`PROPOSAL_NOT_FOUND`, `INVALID_STATE_TRANSITION`, `CLUSTER_HAS_NO_CONFIG_REPO`, `GITHUB_NOT_CONFIGURED`, `QUEUE_UNAVAILABLE`) to the LLM as tool_result payloads. +- **`feat_studies_ui` (PR #50, merged 2026-05-12)** — provides the Next.js shell + nav (with the placeholder `/chat` link), TanStack Query setup, central `MutationCache.onError` toast wiring, and the `apiClient` singleton with `X-Request-ID` injection. +- **`feat_proposals_ui` (PR #58, merged 2026-05-12)** — frontend siblings the chat UI links to via tool-result deep links (e.g., after the agent calls `create_proposal_from_study`, it can offer a `/proposals/{id}` link in plain text). +- **OpenAI API key** — required at chat time (returns 503 `OPENAI_NOT_CONFIGURED` otherwise per the `feat_llm_judgments` precedent in `backend/app/api/v1/judgments.py:201`). ## 6) Actors and roles @@ -121,8 +139,8 @@ N/A — `audit_log` is MVP2. When MVP2 ships, this feature's mutating tool dispa - Returns 503 `OPENAI_NOT_CONFIGURED` if `OPENAI_API_KEY_FILE` is missing. ### FR-3: Orchestrator loop -- The orchestrator **MUST** call OpenAI's `chat.completions.create` with `model={settings.OPENAI_MODEL_CHAT}` (default `gpt-4o-mini-2024-07-18` for OpenAI; for local providers reuses `OPENAI_MODEL`), against the configured `OPENAI_BASE_URL`, with `tools=TOOLS`, `tool_choice='auto'`, `stream=True`. -- The orchestrator **MUST** read the capability cache (per `infra_foundation` FR-7). If `function_calling != "ok"` for the configured endpoint, the orchestrator runs WITHOUT tools (passes `tools=[]`); the agent can still chat but cannot dispatch. The first assistant turn in such a session emits a system-level message (visible in the chat UI) explaining: "Tool dispatch is unavailable on this LLM provider (`{base_url}` lacks reliable function-calling). Use the UI to create studies / open PRs." +- The orchestrator **MUST** call OpenAI's `chat.completions.create` with `model=settings.openai_model_chat` (default `gpt-4o-mini-2024-07-18` per `backend/app/core/settings.py:117`; for local providers operators can override to reuse `openai_model`), against the configured `openai_base_url`, with `tools=TOOLS`, `tool_choice='auto'`, `stream=True`. +- The orchestrator **MUST** read the capability cache via `read_capability_result(redis_client, base_url)` from `backend/app/llm/capability_check.py:372`. If `function_calling != "ok"` for the configured endpoint, the orchestrator runs WITHOUT tools (passes `tools=[]`); the agent can still chat but cannot dispatch. The first assistant turn in such a session emits a system-level message (visible in the chat UI) explaining: "Tool dispatch is unavailable on this LLM provider (`{base_url}` lacks reliable function-calling). Use the UI to create studies / open PRs." - The orchestrator **MUST** stream tokens to the SSE connection as they arrive. - The orchestrator **MUST** detect tool_calls in the stream and, after the stream ends: - Persist the assistant message (with `tool_calls` JSONB) @@ -133,15 +151,15 @@ N/A — `audit_log` is MVP2. When MVP2 ships, this feature's mutating tool dispa - Notes: covers US-25, US-26, US-27. ### FR-4: Tool registry -- The system **MUST** ship 18 MVP1 tools per [`agent-tools.md` §"MVP1 tool inventory"](../../../01_architecture/agent-tools.md): - - `list_clusters`, `get_cluster`, `get_schema` - - `list_templates`, `get_template` - - `list_query_sets`, `create_query_set`, `import_queries_from_csv`, `generate_judgments_llm`, `get_calibration` - - `run_query` - - `create_study`, `get_study`, `cancel_study` - - `list_proposals`, `get_proposal`, `create_proposal_from_study`, `create_proposal_manual`, `open_pr` -- Each tool is a separate module at `backend/agent/tools//.py` with a Pydantic args schema, an async impl function, and a `*_TOOL` constant. -- The registry collector at `backend/agent/tools/__init__.py` exports `TOOLS: list[ChatCompletionToolParam]` and `TOOL_REGISTRY: dict[str, Callable]`. +- The system **MUST** ship 19 MVP1 tools per [`agent-tools.md` §"MVP1 tool inventory"](../../../01_architecture/agent-tools.md) (counted across the 6 categories below: 3 + 2 + 5 + 1 + 3 + 5 = 19): + - **Cluster & schema (3):** `list_clusters`, `get_cluster`, `get_schema` + - **Templates (2):** `list_templates`, `get_template` + - **Query sets & judgments (5):** `list_query_sets`, `create_query_set`, `import_queries_from_csv`, `generate_judgments_llm`, `get_calibration` + - **Quick experiments (1):** `run_query` + - **Studies (3):** `create_study`, `get_study`, `cancel_study` + - **Proposals & PRs (5):** `list_proposals`, `get_proposal`, `create_proposal_from_study`, `create_proposal_manual`, `open_pr` +- Each tool is a separate module at `backend/app/agent/tools//.py` with a Pydantic args schema, an async impl function, and a `*_TOOL` constant. +- The registry collector at `backend/app/agent/tools/__init__.py` exports `TOOLS: list[ChatCompletionToolParam]` and `TOOL_REGISTRY: dict[str, Callable]`. ### FR-5: System prompt + confirmation rule - The system **MUST** load `prompts/orchestrator.system.md` at startup. @@ -179,8 +197,10 @@ The SSE response body uses standard SSE framing per FR-2. | Field | Accepted values | Backend source of truth | |---|---|---| -| `messages.role` | `user`, `assistant`, `tool` | `backend/db/models/message.py` | -| SSE event types | `token`, `tool_call`, `tool_result`, `done` | `backend/api/conversations.py` (`SSEEventType` `Literal[...]`) | +| `messages.role` | `user`, `assistant`, `tool` | `backend/app/db/models/message.py` (CHECK constraint on the `role` column) | +| SSE event types | `token`, `tool_call`, `tool_result`, `done` | `backend/app/api/v1/conversations.py` (`SSEEventType = Literal["token", "tool_call", "tool_result", "done"]`) | + +Frontend tests + components consuming these values MUST add a `// Values must match backend/app/db/models/message.py …` source-of-truth comment in `ui/src/lib/enums.ts` (mirroring the 19 allowlists shipped by `feat_studies_ui`'s Story 4.2). The CI gate at `scripts/ci/verify_enum_source_of_truth.sh` enforces drift detection. ### 7.5 Error code catalog @@ -192,7 +212,13 @@ The SSE response body uses standard SSE framing per FR-2. ## 9) Data model and state transitions -This feature creates `conversations` and `messages` per [`data-model.md`](../../../01_architecture/data-model.md). Both are terminal tables (no other MVP1 feature depends on them). Migration adds full MVP1 shape; no other features ALTER these tables. +This feature creates `conversations` and `messages` per [`data-model.md`](../../../01_architecture/data-model.md) via Alembic revision `0007_conversations_messages` (next sequential after `0006_proposals_pr_url_idx`). Both are terminal tables (no other MVP1 feature depends on them). Migration adds full MVP1 shape; no other features ALTER these tables. + +**Required columns** (per CLAUDE.md conventions): +- `conversations`: `id UUID PK`, `title TEXT NULL`, `created_at TIMESTAMPTZ NOT NULL DEFAULT now()`, `deleted_at TIMESTAMPTZ NULL` (soft-delete per CLAUDE.md "soft delete via `deleted_at` on user-facing tables"). +- `messages`: `id UUID PK`, `conversation_id UUID NOT NULL REFERENCES conversations(id) ON DELETE CASCADE`, `role TEXT NOT NULL CHECK (role IN ('user', 'assistant', 'tool'))`, `content JSONB NOT NULL`, `tool_calls JSONB NULL`, `created_at TIMESTAMPTZ NOT NULL DEFAULT now()`. + +Both `created_at` columns use `TIMESTAMPTZ` storing UTC (per `data-model.md` §"Conventions"). Migration MUST include `downgrade()` per CLAUDE.md Absolute Rule #5; round-trip verified via `alembic upgrade head && alembic downgrade -1 && alembic upgrade head`. ## 10) Security, privacy, and compliance @@ -216,7 +242,7 @@ This feature creates `conversations` and `messages` per [`data-model.md`](../../ - **Tool dispatch raises** (e.g., backend 500). Tool result event payload includes `{error: '', message: ''}`; LLM gets the message and either retries with corrections or surfaces to user. - **Tool loop limit hit (10 iterations).** Loop terminates with `done.error = 'tool_loop_limit_exceeded'`; conversation is in a recoverable state — user can retry their request. - **User reloads page mid-stream.** SSE connection drops; the assistant turn that was in flight is incomplete in the database (only the user message persisted). UI on reload shows the conversation up to the last completed turn; user can re-send the message. -- **Multiple browser tabs open on the same conversation.** Both tabs receive `useConversation` polls; both attempt SSE on send; first one wins, second one gets a 409 `CONVERSATION_BUSY` (added — let me note this in §19 if needed). Recommend: simpler approach — both connections succeed; LLM messages interleave (rare in practice). Open question. +- **Multiple browser tabs open on the same conversation.** Locked in §19 decision log (2026-05-09): **laissez-faire** — both connections succeed; LLM messages interleave (rare in practice). `CONVERSATION_BUSY` (409) deferred to MVP2 if real complaints emerge. ## 12) Given/When/Then acceptance criteria @@ -277,17 +303,21 @@ This feature creates `conversations` and `messages` per [`data-model.md`](../../ ## 14) Test strategy requirements -- **Unit tests** (`backend/tests/unit/`): - - `agent/test_tool_registry.py` — every MVP1 tool is registered; each has a Pydantic args schema; each has a tool description. - - `agent/test_dispatch_validation.py` — invalid args produce a tool_result with `validation_failed`. - - `agent/test_tool_loop_limit.py` — 10-iteration cap is enforced. +- **Unit tests** (`backend/tests/unit/agent/`): + - `test_tool_registry.py` — every MVP1 tool is registered; each has a Pydantic args schema; each has a tool description. + - `test_dispatch_validation.py` — invalid args produce a tool_result with `validation_failed`. + - `test_tool_loop_limit.py` — 10-iteration cap is enforced. - **Integration tests** (`backend/tests/integration/`): - `test_chat_simple.py` — single-turn conversation (read-only tool); asserts SSE framing. - `test_chat_create_study.py` — full flow: confirm + create_study; cassette-replayed OpenAI. - `test_chat_persistence.py` — conversation reconstructable across restart. -- **Contract tests:** - - `test_conversations_api_contract.py` — REST endpoint shapes. - - `test_sse_event_shapes.py` — every SSE event type matches the documented body shape. + - `test_conversations_migration.py` — Alembic round-trip on `0007_conversations_messages` (mirror the `feat_llm_judgments` pattern at `backend/tests/integration/test_judgments_migration.py`). +- **Contract tests** (`backend/tests/contract/`): + - `test_conversations_api_contract.py` — REST endpoint shapes + the 3 error codes (`CONVERSATION_NOT_FOUND`, `OPENAI_NOT_CONFIGURED`, `OPENAI_BUDGET_EXCEEDED`). + - `test_sse_event_shapes.py` — every SSE event type (`token`, `tool_call`, `tool_result`, `done`) matches the documented body shape. +- **Frontend tests** (`ui/src/__tests__/`): + - `app/chat/page.test.tsx` — conversation list rendering. + - `app/chat/[id]/page.test.tsx` — message stream + composer + tool-call card expand/collapse + SSE consumer wiring. - **E2E tests:** N/A in MVP1. ## 15) Documentation update requirements @@ -311,12 +341,12 @@ This feature creates `conversations` and `messages` per [`data-model.md`](../../ | FR-3 (loop) | AC-1, AC-6, AC-8 | TBD | `tests/integration/test_chat_create_study.py`, `tests/unit/agent/test_tool_loop_limit.py` | — | | FR-4 (tool registry) | AC-3 | TBD | `tests/unit/agent/test_tool_registry.py` | agent-tools.md | | FR-5 (system prompt + confirmation) | AC-4 | TBD | `tests/integration/test_chat_create_study.py` | runbook | -| FR-6 (frontend) | AC-1, AC-2, AC-3 | TBD | `ui/tests/unit/app/chat/[id]/page.spec.tsx` | — | +| FR-6 (frontend) | AC-1, AC-2, AC-3 | TBD | `ui/src/__tests__/app/chat/[id]/page.test.tsx` | — | ## 18) Definition of feature done - [ ] AC-1 through AC-8 pass. -- [ ] All test layers green; ≥80% coverage on `backend/agent/`. +- [ ] All test layers green; ≥80% coverage on `backend/app/agent/`. - [ ] Tutorial chat flow demoable in <2 min. - [ ] No open questions remain in §19. @@ -335,3 +365,4 @@ None — all resolved (see Decision log). - 2026-05-09 — Multi-tab handling: **laissez-faire** (allow concurrent tabs to both POST messages; LLM responses interleave; rare in practice). Add `CONVERSATION_BUSY` 409 at MVP2 if real complaints emerge. - 2026-05-09 — Confirmation list expanded to include `import_queries_from_csv` (large bulk add risk). - 2026-05-09 — Tool loop limit: **10** (per FR-3). +- 2026-05-12 — `/idea-preflight` ground-truth pass against the codebase after all 8 backend + 2 frontend dependencies shipped: §2 + §5 rewritten past-tense with merge dates; backend paths corrected from `backend//` → `backend/app//` (4 occurrences in §3 + FR-4 + §7.4); frontend path corrected from `ui/lib/api/` → `ui/src/lib/api/` (2 occurrences); test paths corrected from `ui/tests/unit/...` → `ui/src/__tests__/...` (§14 + §17); tool count corrected from "18" to "19" (§1 + §3 + FR-4 — 3 + 2 + 5 + 1 + 3 + 5 = 19); `settings.OPENAI_MODEL_CHAT` → `settings.openai_model_chat` (FR-3); capability cache citation tightened to `read_capability_result(redis_client, base_url)` at `capability_check.py:372`; §9 expanded with the full column inventory + `0007_conversations_messages` migration revision number + soft-delete `deleted_at` column requirement (per CLAUDE.md "soft delete via deleted_at"); §11 multi-tab "Open question" cleaned up to point at the 2026-05-09 lock; §14 added `test_conversations_migration.py` for Alembic round-trip per the `feat_llm_judgments` precedent; §7.4 cited the `verify_enum_source_of_truth.sh` CI gate that `feat_studies_ui` Story 4.2 shipped. Tutorial-flow language in §1 aligned to `local-es` (was `staging-products-es`; MVP1 has no staging). From 885876937bc3c29c781fe71773ab79be45d90bd2 Mon Sep 17 00:00:00 2001 From: SoundMindsAI Date: Tue, 12 May 2026 14:42:13 -0400 Subject: [PATCH 02/18] docs(feat-chat-agent): implementation plan (3 GPT-5.5 review cycles) + spec patches MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the implementation plan and finalizes the spec for feat_chat_agent after three cross-model review cycles (Opus 4.7 ↔ GPT-5.5): - Cycle 1 (14 findings, all accepted): persistence-ownership split, TOOL_ARG_MODELS drift assertion, UUID-typed ID args, server-side confirmation guard, degraded- mode wire shape, stream_options.include_usage, contract-test coverage for OPENAI_NOT_CONFIGURED / OPENAI_BUDGET_EXCEEDED. Three spec patches landed alongside: return-shape on POST /conversations, AC-5 to 503 JSON envelope, FR-5 "18" → "19" tools. - Cycle 2 (16 findings, 15 accepted / 1 rejected): two-condition confirmation guard, history-sequencing assertion, model_dump(mode="json") for UUID safety, prompt-injection delimiters, RateLimitError handling, defensive 100-message cap, message_count repo path. Spec §7.4 patched to point at schemas.py as the enum source of truth. - Cycle 3 convergence pass (4 findings, all accepted): _emit_tool_error generator-bug rename (now _build_tool_error_events returning a list), residual pydantic.UUID → uuid.UUID sweep + model_dump invariant clarification, message_count coverage in test_conversations_crud.py, AsyncOpenAI dependency- injection clarification. Status: Ready for Execution. 5 epics / 16 stories spanning Alembic 0007 migration, agent package skeleton, 19-tool registry, system prompt + orchestrator, REST + SSE endpoints, and the /chat list + detail UI. Includes MVP1 dashboard regeneration from the pre-commit hook. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/00_overview/MVP1_DASHBOARD.md | 17 +- docs/00_overview/mvp1_dashboard.html | 37 +- .../feat_chat_agent/feature_spec.md | 17 +- .../feat_chat_agent/implementation_plan.md | 2203 +++++++++++++++++ 4 files changed, 2239 insertions(+), 35 deletions(-) create mode 100644 docs/02_product/planned_features/feat_chat_agent/implementation_plan.md diff --git a/docs/00_overview/MVP1_DASHBOARD.md b/docs/00_overview/MVP1_DASHBOARD.md index 45f728eb..01be8eca 100644 --- a/docs/00_overview/MVP1_DASHBOARD.md +++ b/docs/00_overview/MVP1_DASHBOARD.md @@ -4,14 +4,14 @@ _Reflects feature-folder state as of **2026-05-12** (latest mtime of any planned ## Next up -**[feat_chat_agent](../02_product/planned_features/feat_chat_agent/feature_spec.md)** — Feature, currently in **Spec** +**[feat_chat_agent](../02_product/planned_features/feat_chat_agent/feature_spec.md)** — Feature, currently in **Plan** > A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. -Spec exists; run /pipeline to generate the implementation plan + ship +Plan approved; run /impl-execute to ship ```bash -/pipeline docs/02_product/planned_features/feat_chat_agent --auto +/impl-execute docs/02_product/planned_features/feat_chat_agent/implementation_plan.md --all ``` ## MVP1 Progress @@ -47,15 +47,16 @@ Spec exists; run /pipeline to generate the implementation plan + ship _None._ -### Plan (0) +### Plan (1) -_None._ +| Feature | Type | One-liner | Depends on | Status | +|---|---|---|---|---| +| [feat_chat_agent](../02_product/planned_features/feat_chat_agent/feature_spec.md) | Feature | A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. | `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_arq_subprocess_test` `infra_ci_smoke_makeup` `infra_foundation` `infra_frontend_stack_refresh` `infra_nvmrc` `infra_optuna_eval` `infra_per_trial_timeout` | [PR #4](https://github.com/SoundMindsAI/relyloop/pull/4) merged 2026-05-09 | -### Spec (2) +### Spec (1) | Feature | Type | One-liner | Depends on | Status | |---|---|---|---|---| -| [feat_chat_agent](../02_product/planned_features/feat_chat_agent/feature_spec.md) | Feature | A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE. | `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_arq_subprocess_test` `infra_ci_smoke_makeup` `infra_foundation` `infra_frontend_stack_refresh` `infra_nvmrc` `infra_optuna_eval` `infra_per_trial_timeout` | [PR #4](https://github.com/SoundMindsAI/relyloop/pull/4) merged 2026-05-09 | | [chore_tutorial_polish](../02_product/planned_features/chore_tutorial_polish/feature_spec.md) | Chore | The release tag `v0.1.0` is pushed with: a worked tutorial at `docs/08_guides/tutorial-first-study.md`, sample data (50-query set + pre-baked judgment list + sample ES index of ~1,000 docs), README po | `feat_chat_agent` `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` `feat_llm_judgments` `feat_proposals_ui` `feat_studies_ui` `feat_study_lifecycle` `infra_adapter_elastic` `infra_arq_subprocess_test` `infra_ci_smoke_makeup` `infra_foundation` `infra_frontend_stack_refresh` `infra_nvmrc` `infra_optuna_eval` `infra_per_trial_timeout` | Draft | ### Idea (22) @@ -99,7 +100,7 @@ graph LR chore_tutorial_polish["tutorial polish"] class chore_tutorial_polish spec; feat_chat_agent["chat agent"] - class feat_chat_agent spec; + class feat_chat_agent plan; infra_foundation["foundation"] class infra_foundation done; feat_study_lifecycle["study lifecycle"] diff --git a/docs/00_overview/mvp1_dashboard.html b/docs/00_overview/mvp1_dashboard.html index 0d5abfa8..d49cf664 100644 --- a/docs/00_overview/mvp1_dashboard.html +++ b/docs/00_overview/mvp1_dashboard.html @@ -314,11 +314,11 @@

RelyLoop MVP1 Dashboard

-
Next up — Feature, currently in Spec
+
Next up — Feature, currently in Plan
Chat Agent
A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE.
-
Spec exists; run /pipeline to generate the implementation plan + ship
- /pipeline docs/02_product/planned_features/feat_chat_agent --auto +
Plan approved; run /impl-execute to ship
+ /impl-execute docs/02_product/planned_features/feat_chat_agent/implementation_plan.md --all
@@ -640,19 +640,7 @@

Idea 22

-

Spec 2

- -
-
Chat Agent
-
- Feature - PR #4merged 2026-05-09 -
-
A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE.
- -
depends on: feat_digest_proposalfeat_github_pr_workerfeat_github_webhookfeat_llm_judgmentsfeat_proposals_uifeat_studies_uifeat_study_lifecycleinfra_adapter_elasticinfra_arq_subprocess_testinfra_ci_smoke_makeupinfra_foundationinfra_frontend_stack_refreshinfra_nvmrcinfra_optuna_evalinfra_per_trial_timeout
-
- +

Spec 1

Tutorial Polish
@@ -668,7 +656,18 @@

Spec 2

-

Plan 0

+

Plan 1

+ +
+
Chat Agent
+
+ Feature + PR #4merged 2026-05-09 +
+
A chat surface at `/chat/{conversation_id}` streams OpenAI completions via SSE.
+ +
depends on: feat_digest_proposalfeat_github_pr_workerfeat_github_webhookfeat_llm_judgmentsfeat_proposals_uifeat_studies_uifeat_study_lifecycleinfra_adapter_elasticinfra_arq_subprocess_testinfra_ci_smoke_makeupinfra_foundationinfra_frontend_stack_refreshinfra_nvmrcinfra_optuna_evalinfra_per_trial_timeout
+
@@ -827,7 +826,7 @@

Dependency graph (feat_ + infra_)

chore_tutorial_polish["tutorial polish"] class chore_tutorial_polish spec; feat_chat_agent["chat agent"] - class feat_chat_agent spec; + class feat_chat_agent plan; infra_foundation["foundation"] class infra_foundation done; feat_study_lifecycle["study lifecycle"] @@ -903,7 +902,7 @@

Dependency graph (feat_ + infra_)

chore_tutorial_polish["tutorial polish"] class chore_tutorial_polish spec; feat_chat_agent["chat agent"] - class feat_chat_agent spec; + class feat_chat_agent plan; infra_foundation["foundation"] class infra_foundation done; feat_study_lifecycle["study lifecycle"] diff --git a/docs/02_product/planned_features/feat_chat_agent/feature_spec.md b/docs/02_product/planned_features/feat_chat_agent/feature_spec.md index 123f692a..eca6917d 100644 --- a/docs/02_product/planned_features/feat_chat_agent/feature_spec.md +++ b/docs/02_product/planned_features/feat_chat_agent/feature_spec.md @@ -45,7 +45,7 @@ As of 2026-05-12, every dependency has shipped: - Orchestrator at `backend/app/agent/orchestrator.py` running the OpenAI function-calling loop (per [`llm-orchestration.md`](../../../01_architecture/llm-orchestration.md)). - SSE endpoint `POST /api/v1/conversations/{id}/messages` accepting a user message and streaming the assistant turn (token + tool_call + tool_result + done events per [`agent-tools.md` §"Streaming + SSE"](../../../01_architecture/agent-tools.md)). - REST endpoints: - - `POST /api/v1/conversations` — create new conversation (returns `conversation_id`) + - `POST /api/v1/conversations` — create new conversation; returns the full `ConversationSummary` (`id`, `title`, `created_at`, `message_count = 0`), matching the shape of rows returned by `GET /api/v1/conversations`. Clients read `id` from the response to navigate to `/chat/{id}`. - `GET /api/v1/conversations` — list (cursor-paginated, soft-deleted rows filtered out) - `GET /api/v1/conversations/{id}` — detail with full message history - `DELETE /api/v1/conversations/{id}` — soft-delete (sets `deleted_at`; messages preserved via FK CASCADE on hard purge only) @@ -167,7 +167,7 @@ N/A — `audit_log` is MVP2. When MVP2 ships, this feature's mutating tool dispa - Confirm before calling `create_study`, `cancel_study`, `generate_judgments_llm`, `open_pr`, `create_proposal_*` (the mutating tools). - Use `gpt-4o-mini` for cost reasons. - Surface tool errors to the user (don't silently retry). - - Not invent tools beyond the 18 in the registry. + - Not invent tools beyond the 19 in the registry. ### FR-6: Frontend chat surface - `/chat` route shows the conversation list (sidebar) + "New conversation" button. Selecting a conversation routes to `/chat/{id}`. @@ -195,12 +195,12 @@ The SSE response body uses standard SSE framing per FR-2. ### 7.4 Enumerated value contracts -| Field | Accepted values | Backend source of truth | -|---|---|---| -| `messages.role` | `user`, `assistant`, `tool` | `backend/app/db/models/message.py` (CHECK constraint on the `role` column) | -| SSE event types | `token`, `tool_call`, `tool_result`, `done` | `backend/app/api/v1/conversations.py` (`SSEEventType = Literal["token", "tool_call", "tool_result", "done"]`) | +| Field | Accepted values | Backend wire-shape source of truth (gate-scanned) | DB-level source of truth (constraint) | +|---|---|---|---| +| `messages.role` | `user`, `assistant`, `tool` | `backend/app/api/v1/schemas.py` (`MessageRoleWire = Literal["user", "assistant", "tool"]`) | `backend/app/db/models/message.py` (CHECK constraint on the `role` column) | +| SSE event types | `token`, `tool_call`, `tool_result`, `done` | `backend/app/api/v1/schemas.py` (`SSEEventTypeWire = Literal["token", "tool_call", "tool_result", "done"]`) | — (no DB persistence) | -Frontend tests + components consuming these values MUST add a `// Values must match backend/app/db/models/message.py …` source-of-truth comment in `ui/src/lib/enums.ts` (mirroring the 19 allowlists shipped by `feat_studies_ui`'s Story 4.2). The CI gate at `scripts/ci/verify_enum_source_of_truth.sh` enforces drift detection. +Frontend tests + components consuming these values MUST add `// Values must match backend/app/api/v1/schemas.py MessageRoleWire` and `// Values must match backend/app/api/v1/schemas.py SSEEventTypeWire` source-of-truth comments in `ui/src/lib/enums.ts` (mirroring the 19 allowlists shipped by `feat_studies_ui`'s Story 4.2). The CI gate at `scripts/ci/verify_enum_source_of_truth.sh` imports the cited module and validates that the enum array in `enums.ts` matches the Literal character-for-character. The wire-shape Literals live in `schemas.py` (where every other allowlist in the project lives, per the `feat_studies_ui` precedent — `STUDY_STATUS_VALUES`, `TRIAL_STATUS_VALUES`, etc.); the DB CHECK constraint on `messages.role` in `message.py` is a defense-in-depth duplicate that MUST agree with the schema Literal (drift between them would be a migration bug). Verified by visual inspection that both lists are identical (`user`, `assistant`, `tool`) at migration time. ### 7.5 Error code catalog @@ -274,7 +274,7 @@ Both `created_at` columns use `TIMESTAMPTZ` storing UTC (per `data-model.md` §" - Given `./secrets/openai_key` is empty. - When the operator sends any chat message. -- Then the SSE response immediately delivers `event: done` with `error: 'OPENAI_NOT_CONFIGURED'`; the UI surfaces toast. +- Then `POST /api/v1/conversations/{id}/messages` returns **HTTP 503** with a standard JSON envelope `{"detail": {"error_code": "OPENAI_NOT_CONFIGURED", "message": "...", "retryable": false}}` — the SSE stream never opens. The frontend's `streamChatMessage` consumer catches the non-OK response, throws `ApiError`, and `toast.error(toToastMessage(err))` surfaces the toast. This matches the FR-2 wording, the `feat_llm_judgments` precedent (`OPENAI_NOT_CONFIGURED` always returns 503 JSON at `backend/app/api/v1/judgments.py:201`), and the global `MutationCache`/manual-toast pattern shipped by `feat_studies_ui`. Resolved 2026-05-12 during cross-model review of the implementation plan — original wording in this AC suggested an in-stream SSE `done.error` payload, which contradicted FR-2 and the codebase precedent. ### AC-6: Tool validation failure surfaces to LLM @@ -366,3 +366,4 @@ None — all resolved (see Decision log). - 2026-05-09 — Confirmation list expanded to include `import_queries_from_csv` (large bulk add risk). - 2026-05-09 — Tool loop limit: **10** (per FR-3). - 2026-05-12 — `/idea-preflight` ground-truth pass against the codebase after all 8 backend + 2 frontend dependencies shipped: §2 + §5 rewritten past-tense with merge dates; backend paths corrected from `backend//` → `backend/app//` (4 occurrences in §3 + FR-4 + §7.4); frontend path corrected from `ui/lib/api/` → `ui/src/lib/api/` (2 occurrences); test paths corrected from `ui/tests/unit/...` → `ui/src/__tests__/...` (§14 + §17); tool count corrected from "18" to "19" (§1 + §3 + FR-4 — 3 + 2 + 5 + 1 + 3 + 5 = 19); `settings.OPENAI_MODEL_CHAT` → `settings.openai_model_chat` (FR-3); capability cache citation tightened to `read_capability_result(redis_client, base_url)` at `capability_check.py:372`; §9 expanded with the full column inventory + `0007_conversations_messages` migration revision number + soft-delete `deleted_at` column requirement (per CLAUDE.md "soft delete via deleted_at"); §11 multi-tab "Open question" cleaned up to point at the 2026-05-09 lock; §14 added `test_conversations_migration.py` for Alembic round-trip per the `feat_llm_judgments` precedent; §7.4 cited the `verify_enum_source_of_truth.sh` CI gate that `feat_studies_ui` Story 4.2 shipped. Tutorial-flow language in §1 aligned to `local-es` (was `staging-products-es`; MVP1 has no staging). +- 2026-05-12 — Cross-model review of the `implementation_plan.md` (GPT-5.5, cycle 1) surfaced two spec patches: (a) FR-5 final straggler "18" → "19" (caught by Opus internal review before the GPT-5.5 call); (b) §3 `POST /conversations` return-shape prose tightened from "returns `conversation_id`" to "returns the full `ConversationSummary` (`id`, `title`, `created_at`, `message_count = 0`)" so the wire contract matches the GET-list row shape and the Pydantic schema in the plan; (c) AC-5 corrected from "SSE response immediately delivers `event: done` with `error: 'OPENAI_NOT_CONFIGURED'`" → "HTTP 503 JSON envelope" to match FR-2 and the `feat_llm_judgments` precedent at `backend/app/api/v1/judgments.py:201`. Both spec changes resolve in-spec contradictions that would have caused implementation drift. diff --git a/docs/02_product/planned_features/feat_chat_agent/implementation_plan.md b/docs/02_product/planned_features/feat_chat_agent/implementation_plan.md new file mode 100644 index 00000000..b0ff1d7d --- /dev/null +++ b/docs/02_product/planned_features/feat_chat_agent/implementation_plan.md @@ -0,0 +1,2203 @@ +# Implementation Plan — feat_chat_agent + +**Date:** 2026-05-12 +**Status:** Ready for Execution +**Primary spec:** [`feature_spec.md`](feature_spec.md) +**Policy sources:** +- [`docs/01_architecture/agent-tools.md`](../../../01_architecture/agent-tools.md) — tool definition + dispatch pattern + per-call validation contract +- [`docs/01_architecture/llm-orchestration.md`](../../../01_architecture/llm-orchestration.md) — OpenAI SDK + function-calling + capability check + per-task degradation +- [`docs/01_architecture/ui-architecture.md`](../../../01_architecture/ui-architecture.md) §"Streaming chat" (lines 186–226) — canonical `fetch() + ReadableStream` SSE consumer +- [`docs/01_architecture/data-model.md`](../../../01_architecture/data-model.md) — `conversations` + `messages` schema (this feature creates them) +- [`docs/01_architecture/api-conventions.md`](../../../01_architecture/api-conventions.md) — error envelope + cursor pagination + `X-Total-Count` +- Sibling-feature precedents: [`feat_studies_ui`](../../00_overview/implemented_features/2026_05_12_feat_studies_ui/implementation_plan.md) (shell + nav + TanStack + enum gate), [`feat_proposals_ui`](../../00_overview/implemented_features/2026_05_12_feat_proposals_ui/implementation_plan.md) (list/detail page idiom), [`feat_llm_judgments`](../../00_overview/implemented_features/2026_05_11_feat_llm_judgments/implementation_plan.md) (OpenAI preflight, capability cache, budget gate, structlog), [`feat_github_pr_worker`](../../00_overview/implemented_features/2026_05_12_feat_github_pr_worker/implementation_plan.md) (`open_pr` endpoint preflight chain) + +--- + +## 0) Planning principles + +- **Single phase.** Spec §3 declares this MVP1 deliverable as one phase: "operator opens chat, types tune-on-local-es, agent walks through clarifications, calls create_study with reasonable defaults, offers to monitor via get_study." No deferred phases. +- **Spec is the contract.** Every endpoint, error code, FR, and AC traces to a story below. Cross-checked in §11. +- **Tools dispatch into the service/repo layer directly, not via in-process HTTP self-calls.** This matches [`agent-tools.md`](../../../01_architecture/agent-tools.md) §"Tool definition pattern" (`return await study_state.cancel_study(...)`). The Pydantic args schema gives the LLM the same contract the HTTP endpoint exposes, but at runtime the tool calls Python functions and translates any raised exception into a `tool_result` event with the `{error_code, message, retryable}` payload. +- **For tools whose API endpoint has preflight beyond the underlying service function** (notably `open_pr` — config_repo lookup + github_token check + arq queue probe currently live inside `backend/app/api/v1/proposals.py`), Story 2.4 lifts the preflight into a thin service helper (`backend/app/services/agent_proposals_dispatch.py`) that both the router and the tool call. No router behavior changes. +- **One Alembic migration** (`0007_conversations_messages`, parent of every conversation/message column). Round-trip verified per CLAUDE.md Absolute Rule #5. +- **SSE is a first** — no prior backend endpoint streams `text/event-stream` (verified via grep). The plan establishes the pattern in Story 3.2. +- **No `EventSource` on the frontend.** The chat surface POSTs the user message in the request body; `EventSource` is GET-only. Frontend uses native `fetch() + ReadableStream` per [`ui-architecture.md`](../../../01_architecture/ui-architecture.md) §"Streaming chat". +- **Reuse the OpenAI infrastructure shipped by `feat_llm_judgments`.** `settings.openai_api_key`, `settings.openai_base_url`, `settings.openai_model_chat`, `read_capability_result(redis, base_url)`, `peek_daily_total(redis)`, `record_cost(redis, usd)` — all already exist. The orchestrator wires them together; it does NOT reimplement them. +- **Enumerated value drift is caught by the existing CI gate.** `scripts/ci/verify_enum_source_of_truth.sh` (shipped by `feat_studies_ui` Story 4.2) scans `ui/src/lib/enums.ts` for `// Values must match backend/...` comments. Story 4.4 below adds `MESSAGE_ROLE_VALUES` and `SSE_EVENT_TYPE_VALUES` with matching backend Literal types (in `backend/app/api/v1/schemas.py`) so the gate passes. + +--- + +## 1) Scope traceability (FR → epics/stories → tests) + +| FR | Epic / Story | Test files | Spec ACs | +|---|---|---|---| +| FR-1 (Conversation CRUD: POST/GET/GET/DELETE) | Epic 1 (schema) + Story 3.1 (REST endpoints) | `tests/integration/test_conversations_crud.py`, `tests/contract/test_conversations_api_contract.py` | AC-7 | +| FR-2 (SSE `POST /messages` with `OPENAI_NOT_CONFIGURED` preflight) | Story 3.2 | `tests/integration/test_chat_simple.py`, `tests/contract/test_sse_event_shapes.py` | AC-2, AC-5 | +| FR-3 (Orchestrator loop: tools-or-no-tools by capability, stream, dispatch, 10-iter cap, persist) | Story 2.5 + Story 2.6 | `tests/integration/test_chat_create_study.py`, `tests/unit/agent/test_tool_loop_limit.py`, `tests/unit/agent/test_dispatch_validation.py` | AC-1, AC-6, AC-8 | +| FR-4 (19-tool registry; per-category module layout; `TOOLS` + `TOOL_REGISTRY` collectors) | Stories 2.1–2.4 | `tests/unit/agent/test_tool_registry.py` | AC-3 | +| FR-5 (System prompt + confirm-before-mutate for 7 mutating tools) | Story 2.5 | `tests/unit/agent/test_system_prompt.py`, `tests/integration/test_chat_create_study.py` (confirmation half-turn) | AC-4 | +| FR-6 (Frontend: `/chat`, `/chat/[id]`, composer, tool-call cards, refetch on `done`) | Epic 4 (Stories 4.1–4.4) | `ui/src/__tests__/app/chat/page.test.tsx`, `ui/src/__tests__/app/chat/[id]/page.test.tsx`, `ui/src/__tests__/lib/api/conversations.test.tsx` | AC-1, AC-2, AC-3 | + +No FRs deferred. Spec §19 open questions: **none** (all resolved 2026-05-09 / 2026-05-12; see Decision log). + +--- + +## 2) Delivery structure + +**Conventions (project-specific):** + +- **Backend layout:** new code under `backend/app/agent/` (new package), `backend/app/api/v1/conversations.py`, `backend/app/db/models/{conversation,message}.py`, `backend/app/db/repo/conversation.py`, `backend/app/services/{agent_chat,agent_proposals_dispatch}.py`. Migration at `migrations/versions/0007_conversations_messages.py`. +- **Frontend layout:** `ui/src/app/chat/page.tsx` + `ui/src/app/chat/[id]/page.tsx`. Page components at `ui/src/components/chat/.tsx`. Hook + SSE consumer at `ui/src/lib/api/conversations.ts`. Test files mirror source under `ui/src/__tests__/`. +- **Tool modules:** one file per tool at `backend/app/agent/tools//.py`. Each exports `_TOOL: ChatCompletionToolParam` and `async def _impl(args: , ctx: ToolContext) -> `. The registry collector at `backend/app/agent/tools/__init__.py` builds `TOOLS: list[ChatCompletionToolParam]` and `TOOL_REGISTRY: dict[str, ToolImpl]`. +- **All repo functions** take `db: AsyncSession` first; use `db.flush()`; caller commits. Per CLAUDE.md "Repository Layer". +- **All ORM models** use `String(36)` UUIDv7 primary keys (client-generated), `DateTime(timezone=True)` timestamps, JSONB for flexible payloads, `CheckConstraint` for enums. Per CLAUDE.md "Data Model — Key Tables" + `study.py`/`proposal.py` precedent. +- **Error envelope:** `HTTPException(status_code=..., detail={"error_code": "X", "message": "...", "retryable": bool})` per `backend/app/api/errors.py:44–76`. Helper `_err()` mirrored from `studies.py:67`. +- **Settings access:** `from backend.app.core.settings import get_settings`; never instantiate `Settings()` directly. +- **LLM calls:** use the shipped `read_capability_result(redis, base_url)` + `peek_daily_total(redis)` + `record_cost(redis, usd)` helpers from `backend/app/llm/`. Read model name from `settings.openai_model_chat`. **Never hardcode** `gpt-4o-mini-2024-07-18` in service code (CLAUDE.md Absolute Rule #8). +- **Cursor pagination:** copy the encoder/decoder from `backend/app/api/v1/studies.py:74–87`. Default limit 50; max 200 per `api-conventions.md`. +- **Streaming response:** `StreamingResponse(generator(), media_type="text/event-stream", headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})`. +- **Frontend wire-value sources:** every `