diff --git a/.changeset/manifest-driven-workbench-navigation.md b/.changeset/manifest-driven-workbench-navigation.md new file mode 100644 index 000000000..08cbc028a --- /dev/null +++ b/.changeset/manifest-driven-workbench-navigation.md @@ -0,0 +1,23 @@ +--- +"agent-bundle": minor +--- + +Derive Workbench navigation and its route catalog from the compiled route graph +instead of artifact counts alone (#105 stage 1). + +The dev server exposes one new read-only route, `GET /api/routes/manifest`, +which projects the prepared project's existing `CompiledRouteGraph` into a +browser-safe DTO: route id, kind, project-relative source, provenance, a +flattened static `config` summary, MCP server surfaces with their packaging +mode, the generated CLI command surface with its argv projection, conventional +scripts, context providers, the graph digest, and the graph's own diagnostics. +There is no second discovery pass — the manifest is a projection of the compiler +pass the build, inspect, and test harness already share. + +The Workbench gains a Routes page under **Build** that renders that catalog +grouped by server and by project surface, and reports whether the manifest +matches the published build or is ahead of it. Hooks, MCP playground, and +Playground now open when either the artifact catalog or the compiled graph +declares the surface, so a routed project no longer needs configuration to reach +its own pages. Every existing page is preserved: an absent or refused manifest +degrades only the Routes page. diff --git a/docs/superpowers/plans/2026-08-25-capability-aware-workbench.md b/docs/superpowers/plans/2026-08-25-capability-aware-workbench.md index 31a5315e5..432186171 100644 --- a/docs/superpowers/plans/2026-08-25-capability-aware-workbench.md +++ b/docs/superpowers/plans/2026-08-25-capability-aware-workbench.md @@ -10,6 +10,25 @@ **Spec:** `docs/superpowers/specs/2026-08-25-capability-aware-workbench-design.md` +> Supersession note (#105 stage 1, compiler-manifest-driven navigation): this +> plan is landed and its structure invariants still hold with one amendment. +> The catalog is no longer composed from artifact, Skill, and Eval clients +> alone — it also reads the compiled route graph from one dedicated dev-server +> route (`GET /api/routes/manifest`) behind its own strict decoder, and +> `WorkbenchCapabilities` carries a `routes: RouteCatalog`. Task 1's page rules +> below are now the *union* of those counts and the compiled graph, and the +> page set gained `routes` (a compiled-catalog page under **Build**, beside +> Overview), so the Task 1 assertions and the `WorkbenchCapabilities` / +> `WorkbenchCapabilityClients` shapes quoted here are superseded by +> `packages/workbench/src/workbench-capabilities.ts` and +> `packages/workbench/tests/workbench-capabilities.test.ts`. Everything else is +> unchanged and intentionally so: there is still exactly one Workbench shell, +> one navigation rail, and one hash router in `main.tsx` / +> `workbench-screen.tsx`. The Routes catalog is a page inside them; it does not +> add a shell, a navigation component, or a router. Schema-driven input editors +> and the Agent Document stage remain stage 2 and are deliberately not +> scaffolded. + ## Global Constraints - The Workbench is desktop-only; acceptance viewport is exactly 1440×900. @@ -66,6 +85,9 @@ export const pageForHash = ( ``` - Page rules: Overview/Artifacts/Logs always; Skills for `skills > 0`; Hooks for `hooks > 0`; MCP for `mcpServers > 0`; Playground for `hooks + scripts > 0`; Evals and Comparisons for `evalSuites > 0`. + Amended by #105 stage 1: Routes is also always available once the catalog is + ready, and Hooks, MCP, and Playground additionally open when the compiled + route graph declares an event route, an MCP server surface, or a script route. - [ ] **Step 1: Write failing capability derivation tests** diff --git a/docs/superpowers/specs/2026-08-25-capability-aware-workbench-design.md b/docs/superpowers/specs/2026-08-25-capability-aware-workbench-design.md index a3c92bf99..6701d5ea6 100644 --- a/docs/superpowers/specs/2026-08-25-capability-aware-workbench-design.md +++ b/docs/superpowers/specs/2026-08-25-capability-aware-workbench-design.md @@ -8,6 +8,19 @@ Status: proposed > work is tracked in > [#105](https://github.com/ScriptedAlchemy/agent-bundle/issues/105). This > spec is not the live execution plan. +> +> Supersession note (#105 stage 1): navigation is no longer derived from +> artifact counts alone. The compiled route graph is now a first-class +> navigation input, read once from the dev server's own compiler pass through +> `GET /api/routes/manifest`, and the Workbench is a ten-page shell: a `routes` +> catalog page joins Overview under **Build**. The count-derived capabilities +> below still hold for everything configuration can declare without a route +> module; per-kind availability is now the union of the artifact catalog and the +> compiled graph, so neither source can hide the other. Schema-driven input +> editors and the Agent Document stage remain stage 2. The single Workbench +> shell, navigation, and hash router in `main.tsx`/`workbench-screen.tsx` stay +> the only ones — the manifest catalog is a page inside them, not a second +> shell. ## Context @@ -82,11 +95,20 @@ reuses the strict decoders and services already required by the pages, avoids duplicating catalog schemas, and can be replaced later by a server summary without changing product semantics. +Since #105 stage 1 the catalog also composes the compiled route graph from one +dedicated route (`GET /api/routes/manifest`) behind its own strict decoder. +That route is a projection of the prepared project's existing compiler pass, not +a second discovery: `hooks`, `mcp`, and `playground` are satisfied by either an +emitted artifact entry or a compiled route of the matching kind, and the graph's +own diagnostics render beside the catalog. A refused or absent manifest degrades +only the Routes page; every artifact-derived page keeps its evidence. + ### Navigation and direct routes Navigation receives the catalog and renders three concise groups: -- **Build:** Overview +- **Build:** Overview (and, since #105 stage 1, Routes — the compiled route + catalog, always available once the catalog is ready) - **Capabilities:** only Skills, Hooks, Playground, and MCP capabilities that exist in the catalog - **Quality:** Evals and Comparisons only when Eval suites exist diff --git a/packages/agent-bundle/src/contracts/routes.ts b/packages/agent-bundle/src/contracts/routes.ts new file mode 100644 index 000000000..08767e894 --- /dev/null +++ b/packages/agent-bundle/src/contracts/routes.ts @@ -0,0 +1,20 @@ +/** + * Browser-consumable contract surface for the compiled route manifest the + * Workbench derives its navigation and route catalog from. Type-only: the + * compiler pass that produces the graph runs on the server. + */ +export type { + RouteManifest, + RouteManifestCliCommand, + RouteManifestCliMode, + RouteManifestCliOption, + RouteManifestCliSurface, + RouteManifestConfigEntry, + RouteManifestKind, + RouteManifestProvenance, + RouteManifestProvider, + RouteManifestResponse, + RouteManifestRoute, + RouteManifestServer, + RouteManifestServerMode, +} from '../dev/routes/route-manifest.ts'; diff --git a/packages/agent-bundle/src/dev/foreground-server.ts b/packages/agent-bundle/src/dev/foreground-server.ts index bf076beb2..c7e4d559a 100644 --- a/packages/agent-bundle/src/dev/foreground-server.ts +++ b/packages/agent-bundle/src/dev/foreground-server.ts @@ -20,6 +20,7 @@ import { RuntimeMcpRoutes } from './runtime-mcp-routes.ts'; import { RuntimeRoutes } from './runtime-routes.ts'; import type { DevRuntimeSession } from './runtime-provider.ts'; import { PlaygroundRoutes, type PlaygroundRouteService } from './playground/playground-routes.ts'; +import { RouteManifestRoutes, type RouteManifestRouteService } from './routes/route-manifest-routes.ts'; import { SkillDocumentError, type SkillDocumentService } from './skill-document-service.ts'; import type { Invalidation, ProjectEventMessage, ProjectStatus } from './types.ts'; @@ -136,6 +137,11 @@ export interface ForegroundServerOptions { /** Durable playground trace store; the browser never selects its storage root or project identity. */ readonly playground?: PlaygroundRouteService; readonly port?: number; + /** + * Read-only projection of the compiled route graph. The Workbench derives + * navigation from this one compiler pass; it never re-discovers routes. + */ + readonly routeManifest?: RouteManifestRouteService; /** Optional runtime session; its lifecycle remains Workbench-owned. */ readonly runtime?: DevRuntimeSession; /** Read-only Skill document/resource service for the workbench. */ @@ -434,6 +440,7 @@ export class ForegroundServer { readonly #now: () => Date; readonly #playgroundRoutes: PlaygroundRoutes; readonly #port: number; + readonly #routeManifestRoutes: RouteManifestRoutes; readonly #server: Server; readonly #skillDocuments: SkillDocumentService | undefined; readonly #sockets = new Set(); @@ -513,6 +520,10 @@ export class ForegroundServer { authorize: (request) => this.#assertMutationSession(request), ...(options.artifacts === undefined ? {} : { service: options.artifacts }), }); + this.#routeManifestRoutes = new RouteManifestRoutes({ + authorize: (request) => this.#assertMutationSession(request), + ...(options.routeManifest === undefined ? {} : { service: options.routeManifest }), + }); this.#evalRoutes = new EvalRoutes({ authorize: (request) => this.#assertMutationSession(request), ...(options.evals === undefined ? {} : { service: options.evals }), @@ -657,6 +668,7 @@ export class ForegroundServer { this.#playgroundRoutes.close(); this.#inspectorRoutes.close(); this.#artifactRoutes.close(); + this.#routeManifestRoutes.close(); const releaseEvals = this.#evalRoutes.close(); void releaseEvals.catch(() => undefined); // Fence both public Eval authorities in this turn. Agent API handlers can @@ -742,6 +754,7 @@ export class ForegroundServer { if (await this.#playgroundRoutes.handle(request, response)) return; if (await this.#inspectorRoutes.handle(request, response)) return; if (await this.#artifactRoutes.handle(request, response)) return; + if (this.#routeManifestRoutes.handle(request, response)) return; if (await this.#evalRoutes.handle(request, response)) return; if (await this.#devLogRoutes.handle(request, response)) return; const route = skillRoute(request.url); diff --git a/packages/agent-bundle/src/dev/index.ts b/packages/agent-bundle/src/dev/index.ts index c5a0007d6..276ded971 100644 --- a/packages/agent-bundle/src/dev/index.ts +++ b/packages/agent-bundle/src/dev/index.ts @@ -89,6 +89,23 @@ export { runtimeClientSurfaceReloadChannelPath, type RuntimeClientSurfaceConnectionEvent, } from './runtime-client-surface-proxy.ts'; +export { routeManifestFor } from './routes/route-manifest.ts'; +export type { + RouteManifest, + RouteManifestCliCommand, + RouteManifestCliOption, + RouteManifestCliSurface, + RouteManifestConfigEntry, + RouteManifestProvider, + RouteManifestResponse, + RouteManifestRoute, + RouteManifestServer, +} from './routes/route-manifest.ts'; +export { + RouteManifestRoutes, + type RouteManifestRouteService, + type RouteManifestRoutesOptions, +} from './routes/route-manifest-routes.ts'; export { RuntimeRoutes, type RuntimeRoutesOptions } from './runtime-routes.ts'; export { RuntimeMcpRoutes, type RuntimeMcpRoutesOptions } from './runtime-mcp-routes.ts'; export { diff --git a/packages/agent-bundle/src/dev/routes/route-manifest-routes.ts b/packages/agent-bundle/src/dev/routes/route-manifest-routes.ts new file mode 100644 index 000000000..ce17ee504 --- /dev/null +++ b/packages/agent-bundle/src/dev/routes/route-manifest-routes.ts @@ -0,0 +1,99 @@ +import type { IncomingMessage, ServerResponse } from 'node:http'; + +import { + diagnostic, + isRequestDiagnostic, + rawPathname, + requestError, + responseDiagnostic, + responseJson as writeJsonResponse, +} from '../http.ts'; +import type { RouteManifest } from './route-manifest.ts'; + +export interface RouteManifestRouteService { + /** + * The manifest of the latest valid compiler pass. Throws when no valid + * project has been prepared yet, which the boundary reports as unavailable + * instead of inventing an empty catalog. + */ + manifest(): RouteManifest; +} + +export interface RouteManifestRoutesOptions { + /** The foreground server injects its existing same-origin, same-session guard. */ + readonly authorize: (request: IncomingMessage) => void; + /** Omitted until the workbench composes a prepared-project manifest source. */ + readonly service?: RouteManifestRouteService; +} + +const responseJson = (response: ServerResponse, body: unknown): void => + writeJsonResponse(response, body, { destroyIfEnded: true }); + +const pathError = (): never => { + throw requestError(diagnostic('AB8120', 'Route manifest path is not valid.', 400)); +}; + +const invalidShape = (): never => { + throw requestError(diagnostic('AB8122', 'Route manifest request has an invalid shape.', 400)); +}; + +const isManifestRoute = (requestTarget: string | undefined): boolean => { + const pathname = rawPathname(requestTarget); + if (pathname !== '/api/routes' && !pathname.startsWith('/api/routes/')) return false; + const parts = pathname.split('/'); + if (parts.length !== 4 || parts[0] !== '' || parts[1] !== 'api' || parts[2] !== 'routes' || parts[3] !== 'manifest') { + return pathError(); + } + return true; +}; + +const noQuery = (requestTarget: string | undefined): void => { + if (new URL(requestTarget ?? '/', 'http://localhost').searchParams.size > 0) invalidShape(); +}; + +/** + * Read-only HTTP boundary over the compiled route graph. The browser names no + * path, mode, or revision: the manifest of the latest valid compiler pass is + * the whole request, so the Workbench cannot ask for a second discovery. + */ +export class RouteManifestRoutes { + readonly #authorize: (request: IncomingMessage) => void; + readonly #service: RouteManifestRouteService | undefined; + #closed = false; + + constructor(options: RouteManifestRoutesOptions) { + this.#authorize = options.authorize; + this.#service = options.service; + } + + close(): void { + this.#closed = true; + } + + /** Synchronous by construction: the manifest is already in memory, so this boundary performs no I/O. */ + handle(request: IncomingMessage, response: ServerResponse): boolean { + if (!isManifestRoute(request.url)) return false; + this.#authorize(request); + if (this.#closed) throw this.#unavailable(503); + const service = this.#service; + if (service === undefined) throw this.#unavailable(404); + if ((request.method ?? 'GET') !== 'GET') { + responseDiagnostic(response, diagnostic('AB8007', 'Route does not accept this method.', 405)); + return true; + } + noQuery(request.url); + let manifest: RouteManifest; + try { + manifest = service.manifest(); + } catch (error) { + if (isRequestDiagnostic(error)) throw error; + throw this.#unavailable(409); + } + responseJson(response, { manifest }); + return true; + } + + #unavailable(status: number): Error { + return requestError(diagnostic('AB8121', 'Route manifest is not available.', status)); + } +} diff --git a/packages/agent-bundle/src/dev/routes/route-manifest.ts b/packages/agent-bundle/src/dev/routes/route-manifest.ts new file mode 100644 index 000000000..df2b3a380 --- /dev/null +++ b/packages/agent-bundle/src/dev/routes/route-manifest.ts @@ -0,0 +1,237 @@ +import type { Diagnostic } from '../../core/diagnostics.ts'; +import { deepFreeze } from '../../core/freeze.ts'; +import type { + CompiledAgentRoute, + CompiledCliCommand, + CompiledCliMode, + CompiledCliOption, + CompiledCliSurface, + CompiledProvider, + CompiledRouteGraph, + CompiledRouteKind, + CompiledServerMode, + CompiledServerSurface, +} from '../../routes/types.ts'; + +/** Mirrors {@link CompiledRouteKind}: the catalog groups by the compiler's own kinds. */ +export type RouteManifestKind = CompiledRouteKind; + +/** Mirrors {@link CompiledServerMode}. */ +export type RouteManifestServerMode = CompiledServerMode; + +/** Mirrors {@link CompiledCliMode}. */ +export type RouteManifestCliMode = CompiledCliMode; + +/** + * One statically extracted route-config property, flattened to a display pair. + * Containers report their size instead of nested JSON: this is the catalog + * summary, and whole config values belong to the schema-driven input editors + * of the next Workbench stage rather than to navigation. + */ +export interface RouteManifestConfigEntry { + readonly key: string; + readonly kind: 'array' | 'boolean' | 'null' | 'number' | 'object' | 'string'; + readonly value: string; +} + +/** + * How a route entered the graph. Only conventional filesystem discovery + * exists today; keeping the discriminant makes a later provenance additive + * rather than a wire break. + */ +export interface RouteManifestProvenance { + readonly kind: 'conventional'; +} + +/** One compiled route projected for the browser catalog. */ +export interface RouteManifestRoute { + readonly config: readonly RouteManifestConfigEntry[]; + /** `config.description` when it is a string; the catalog's human label. */ + readonly description?: string; + /** Canonical event identity; `event-route` routes only. */ + readonly event?: string; + readonly id: string; + readonly kind: RouteManifestKind; + readonly provenance: RouteManifestProvenance; + /** The owning MCP server id (`mcp:`); MCP route kinds only. */ + readonly serverId?: string; + /** + * Project-relative POSIX module path. The compiler's absolute path stays on + * the server: the relative path is the route's portable identity and the + * only location that means anything to a browser reading this catalog. + */ + readonly source: string; +} + +/** One MCP server surface with the routes its packaging mode actually compiles. */ +export interface RouteManifestServer { + readonly id: string; + readonly mode: RouteManifestServerMode; + readonly name: string; + readonly routes: readonly RouteManifestRoute[]; +} + +/** One argv projection of a CLI route's input schema, without editor defaults. */ +export interface RouteManifestCliOption { + readonly choices?: readonly string[]; + readonly description?: string; + readonly key: string; + readonly kind: CompiledCliOption['kind']; + readonly option: string; + readonly positional?: number; + readonly repeated: boolean; + readonly required: boolean; +} + +/** One executable command compiled from a `src/cli/**` route. */ +export interface RouteManifestCliCommand { + readonly aliases: readonly string[]; + readonly description?: string; + readonly exitCode: CompiledCliCommand['exitCode']; + readonly options: readonly RouteManifestCliOption[]; + readonly path: readonly string[]; + readonly routeId: string; +} + +/** The CLI surface assembled from `src/cli/**` route modules. */ +export interface RouteManifestCliSurface { + /** Present only in `generated` mode, matching the compiler surface. */ + readonly commands?: readonly RouteManifestCliCommand[]; + readonly mode: RouteManifestCliMode; + readonly routes: readonly RouteManifestRoute[]; +} + +/** One conventional `src/providers/` context provider module. */ +export interface RouteManifestProvider { + readonly id: string; + readonly name: string; + /** Project-relative POSIX module path, for the same reason routes carry one. */ + readonly source: string; +} + +/** + * The browser projection of one compiled route graph. It is the same compiler + * pass the build, `inspect`, and the test harness read — the Workbench derives + * navigation from this manifest instead of running a second discovery. + */ +export interface RouteManifest { + readonly cli?: RouteManifestCliSurface; + readonly diagnostics: readonly Diagnostic[]; + /** The graph digest over project-relative route identity. */ + readonly digest: string; + readonly events: readonly RouteManifestRoute[]; + readonly providers: readonly RouteManifestProvider[]; + readonly scripts: readonly RouteManifestRoute[]; + readonly servers: readonly RouteManifestServer[]; + /** + * The source revision of the compiler pass that produced the graph. The + * browser compares it against the published build's project revision so a + * catalog newer than the last good build is labelled, never presented as + * what that build shipped. + */ + readonly sourceRevision: string; +} + +export interface RouteManifestResponse { + readonly manifest: RouteManifest; +} + +const configEntry = (key: string, value: unknown): RouteManifestConfigEntry => { + if (value === null) return { key, kind: 'null', value: 'null' }; + if (Array.isArray(value)) { + return { key, kind: 'array', value: `${String(value.length)} ${value.length === 1 ? 'entry' : 'entries'}` }; + } + switch (typeof value) { + case 'boolean': + return { key, kind: 'boolean', value: value ? 'true' : 'false' }; + case 'number': + return { key, kind: 'number', value: String(value) }; + case 'string': + return { key, kind: 'string', value }; + default: { + const keys = Object.keys(value as Readonly>); + return { key, kind: 'object', value: `${String(keys.length)} ${keys.length === 1 ? 'key' : 'keys'}` }; + } + } +}; + +/** + * Extraction accepts only JSON literals (AB4806 rejects anything else), so the + * summary needs no escape hatch for functions, symbols, or cycles. + */ +const configSummary = (config: Readonly>): readonly RouteManifestConfigEntry[] => + Object.keys(config).sort((left, right) => left.localeCompare(right)) + .map((key) => configEntry(key, config[key])); + +const description = (config: Readonly>): string | undefined => { + const value = config['description']; + return typeof value === 'string' && value.trim().length > 0 ? value : undefined; +}; + +const manifestRoute = (route: CompiledAgentRoute): RouteManifestRoute => { + const summary = description(route.config); + return { + config: configSummary(route.config), + ...(summary === undefined ? {} : { description: summary }), + ...(route.event === undefined ? {} : { event: route.event }), + id: route.id, + kind: route.kind, + provenance: { kind: route.provenance.kind }, + ...(route.serverId === undefined ? {} : { serverId: route.serverId }), + source: route.provenance.relativePath, + }; +}; + +const manifestServer = (server: CompiledServerSurface): RouteManifestServer => ({ + id: server.id, + mode: server.mode, + name: server.name, + routes: server.routes.map(manifestRoute), +}); + +const manifestCliOption = (option: CompiledCliOption): RouteManifestCliOption => ({ + ...(option.choices === undefined ? {} : { choices: [...option.choices] }), + ...(option.description === undefined ? {} : { description: option.description }), + key: option.key, + kind: option.kind, + option: option.option, + ...(option.positional === undefined ? {} : { positional: option.positional }), + repeated: option.repeated, + required: option.required, +}); + +const manifestCliCommand = (command: CompiledCliCommand): RouteManifestCliCommand => ({ + aliases: [...command.aliases], + ...(command.description === undefined ? {} : { description: command.description }), + exitCode: command.exitCode, + options: command.options.map(manifestCliOption), + path: [...command.path], + routeId: command.routeId, +}); + +const manifestCli = (cli: CompiledCliSurface): RouteManifestCliSurface => ({ + ...(cli.commands === undefined ? {} : { commands: cli.commands.map(manifestCliCommand) }), + mode: cli.mode, + routes: cli.routes.map(manifestRoute), +}); + +const manifestProvider = (provider: CompiledProvider): RouteManifestProvider => ({ + id: provider.id, + name: provider.name, + source: provider.provenance.relativePath, +}); + +/** Projects one compiled route graph into its immutable browser manifest. */ +export const routeManifestFor = ( + graph: CompiledRouteGraph, + sourceRevision: string, +): RouteManifest => deepFreeze({ + ...(graph.cli === undefined ? {} : { cli: manifestCli(graph.cli) }), + diagnostics: graph.diagnostics.map((diagnostic) => ({ ...diagnostic })), + digest: graph.digest, + events: graph.events.map(manifestRoute), + providers: graph.providers.map(manifestProvider), + scripts: graph.scripts.map(manifestRoute), + servers: graph.servers.map(manifestServer), + sourceRevision, +}); diff --git a/packages/agent-bundle/src/dev/workbench-server.ts b/packages/agent-bundle/src/dev/workbench-server.ts index e3ee5ff6b..4e1bbfc7d 100644 --- a/packages/agent-bundle/src/dev/workbench-server.ts +++ b/packages/agent-bundle/src/dev/workbench-server.ts @@ -34,6 +34,9 @@ import { NativePlaygroundService } from './playground/native-playground-service. import { PlaygroundOrchestrationService } from './playground/playground-orchestration-service.ts'; import { PlaygroundStore as PlaygroundService } from './playground/playground-store.ts'; import { ProjectService } from './project-service.ts'; +import { emptyCompiledRouteGraph } from '../routes/graph.ts'; +import { routeManifestFor } from './routes/route-manifest.ts'; +import type { RouteManifestRouteService } from './routes/route-manifest-routes.ts'; import { DevRuntimeController } from './runtime-controller.ts'; import { RuntimeClientSurfaceProxy, @@ -700,6 +703,17 @@ export const startDevServer = async (options: StartDevServerOptions): Promise { + const prepared = latestValidPreparedProject; + if (prepared === undefined || prepared.source.revision === undefined) { + throw new Error('No valid prepared project is available for the route manifest.'); + } + return routeManifestFor(prepared.routeGraph ?? emptyCompiledRouteGraph, prepared.source.revision); + }, + }; const agentApi = agentApiEnabled ? new AgentApi({ artifacts, @@ -744,6 +758,7 @@ export const startDevServer = async (options: StartDevServerOptions): Promise> => + startRouteServer(new RouteManifestRoutes({ + authorize, + ...(service === undefined ? {} : { service }), + }), { closeMode: 'awaited' }); + +it('serves the compiled manifest without recompiling the graph', async () => { + const service = new RecordingService(); + const started = await startRoutes(service); + + try { + const read = await fetch(`${started.url}/api/routes/manifest`, { headers: headers() }); + expect(read.status).toBe(200); + await expect(read.json()).resolves.toEqual({ + manifest: { + diagnostics: [], + digest: emptyCompiledRouteGraph.digest, + events: [], + providers: [], + scripts: [], + servers: [], + sourceRevision: revision, + }, + }); + expect(service.calls).toEqual(['manifest']); + } finally { + await started.close(); + } +}); + +it('rejects invalid manifest paths, queries, and methods', async () => { + const service = new RecordingService(); + const started = await startRoutes(service); + + try { + for (const path of ['/api/routes', '/api/routes/', '/api/routes/manifest/extra', '/api/routes/unknown']) { + const rejected = await fetch(`${started.url}${path}`, { headers: headers() }); + expect(rejected.status).toBe(400); + await expect(rejected.json()).resolves.toEqual({ + diagnostic: { code: 'AB8120', message: 'Route manifest path is not valid.' }, + }); + } + + const query = await fetch(`${started.url}/api/routes/manifest?focus=cli`, { headers: headers() }); + expect(query.status).toBe(400); + await expect(query.json()).resolves.toEqual({ + diagnostic: { code: 'AB8122', message: 'Route manifest request has an invalid shape.' }, + }); + + const post = await fetch(`${started.url}/api/routes/manifest`, { headers: headers(), method: 'POST' }); + expect(post.status).toBe(405); + await expect(post.json()).resolves.toEqual({ + diagnostic: { code: 'AB8007', message: 'Route does not accept this method.' }, + }); + + const unrelated = await fetch(`${started.url}/api/other`, { headers: headers() }); + expect(unrelated.status).toBe(404); + + expect(service.calls).toEqual([]); + } finally { + await started.close(); + } +}); + +it('requires the same-session guard before reading the manifest', async () => { + const service = new RecordingService(); + const started = await startRoutes(service); + + try { + const unauthorized = await fetch(`${started.url}/api/routes/manifest`, { + headers: { origin: 'http://127.0.0.1:4567' }, + }); + expect(unauthorized.status).toBe(403); + expect(service.calls).toEqual([]); + } finally { + await started.close(); + } +}); + +it('reports an absent, unprepared, or closed manifest without leaking internals', async () => { + const absent = await startRoutes(); + try { + const unavailable = await fetch(`${absent.url}/api/routes/manifest`, { headers: headers() }); + expect(unavailable.status).toBe(404); + await expect(unavailable.json()).resolves.toEqual({ + diagnostic: { code: 'AB8121', message: 'Route manifest is not available.' }, + }); + } finally { + await absent.close(); + } + + const service = new RecordingService(); + service.failure = new Error('/private/project/path has no valid prepared project'); + const started = await startRoutes(service); + try { + const unprepared = await fetch(`${started.url}/api/routes/manifest`, { headers: headers() }); + expect(unprepared.status).toBe(409); + await expect(unprepared.json()).resolves.toEqual({ + diagnostic: { code: 'AB8121', message: 'Route manifest is not available.' }, + }); + + started.routes.close(); + const closed = await fetch(`${started.url}/api/routes/manifest`, { headers: headers() }); + expect(closed.status).toBe(503); + await expect(closed.json()).resolves.toEqual({ + diagnostic: { code: 'AB8121', message: 'Route manifest is not available.' }, + }); + } finally { + await started.close(); + } +}); + +it('preserves a request diagnostic raised by the manifest service', async () => { + const service = new RecordingService(); + service.failure = routeError('AB8004', 'A valid same-session token is required.', 403); + const started = await startRoutes(service); + + try { + const refused = await fetch(`${started.url}/api/routes/manifest`, { headers: headers() }); + expect(refused.status).toBe(403); + await expect(refused.json()).resolves.toEqual({ + diagnostic: { code: 'AB8004', message: 'A valid same-session token is required.' }, + }); + } finally { + await started.close(); + } +}); + +it('projects a compiled graph into the browser manifest with project-relative sources', async () => { + const graph = await compileRouteGraph(resolve(import.meta.dirname, '../fixtures/route-harness'), { targets: ['claude'] } as never); + const manifest = routeManifestFor(graph, revision); + + expect(manifest.digest).toBe(graph.digest); + expect(manifest.sourceRevision).toBe(revision); + expect(manifest.servers.map((server) => server.id)).toEqual(graph.servers.map((server) => server.id)); + const routes = [...manifest.events, ...manifest.scripts, ...manifest.servers.flatMap((server) => server.routes)]; + expect(routes.length).toBeGreaterThan(0); + for (const route of routes) { + expect(route.source.startsWith('/')).toBe(false); + expect(route.provenance).toEqual({ kind: 'conventional' }); + } + expect(Object.isFrozen(manifest)).toBe(true); +}); + +it('summarizes an extracted route config without leaking non-scalar shapes', async () => { + const graph = await compileRouteGraph(resolve(import.meta.dirname, '../fixtures/route-harness'), { targets: ['claude'] } as never); + const manifest = routeManifestFor(graph, revision); + const summarized = manifest.servers.flatMap((server) => server.routes).flatMap((route) => route.config); + + expect(summarized.length).toBeGreaterThan(0); + for (const field of summarized) { + expect(typeof field.value).toBe('string'); + if (field.kind === 'object') expect(field.value).toMatch(/^\d+ (?:key|keys)$/u); + if (field.kind === 'array') expect(field.value).toMatch(/^\d+ (?:entry|entries)$/u); + } +}); diff --git a/packages/agent-bundle/tests/support/route-harness.ts b/packages/agent-bundle/tests/support/route-harness.ts index 36fc65afa..ceaf7124f 100644 --- a/packages/agent-bundle/tests/support/route-harness.ts +++ b/packages/agent-bundle/tests/support/route-harness.ts @@ -3,7 +3,8 @@ import type { AddressInfo } from 'node:net'; /** The shape every dev-server route group under test exposes to the harness. */ export interface RouteHandler { - handle(request: IncomingMessage, response: ServerResponse): Promise; + /** Route groups whose evidence is already in memory answer synchronously. */ + handle(request: IncomingMessage, response: ServerResponse): boolean | Promise; close(): void | Promise; } @@ -63,7 +64,7 @@ export const startRoutes = async ( ): Promise> => { const closeMode = options.closeMode ?? 'started'; const server = createServer((request, response) => { - void routes.handle(request, response).then((handled) => { + void (async () => routes.handle(request, response))().then((handled) => { if (!handled) response.writeHead(404).end(); }).catch((error: unknown) => { const diagnostic = error as Partial<{ code: string; diagnostics: unknown; message: string; status: number }>; diff --git a/packages/workbench/src/main.tsx b/packages/workbench/src/main.tsx index 65a6e8037..8846b2483 100644 --- a/packages/workbench/src/main.tsx +++ b/packages/workbench/src/main.tsx @@ -48,6 +48,9 @@ import { createPlaygroundCatalogLifecycle, playgroundScriptsForEpoch, } from './playground/playground-page.tsx'; +import { RouteManifestClient } from './routes/route-manifest-client.ts'; +import type { RouteCatalog } from './routes/routes-model.ts'; +import { RoutesPage } from './routes/routes-page.tsx'; import { overviewFor } from './overview-model.ts'; import { downloadBlob } from './client-helpers.ts'; import { BundleWorkflow } from './overview-page.tsx'; @@ -327,6 +330,7 @@ const staleCapabilities = (state: CapabilityState): WorkbenchCapabilities | unde const navigationItems: readonly Readonly<{ glyph: string; label: string; page: WorkbenchPage }>[] = [ { glyph: '⊞', label: 'Overview', page: 'overview' }, + { glyph: '⌸', label: 'Routes', page: 'routes' }, { glyph: '⌘', label: 'Skills', page: 'skills' }, { glyph: '⌥', label: 'Hooks', page: 'hooks' }, { glyph: '⌁', label: 'MCP playground', page: 'mcp' }, @@ -625,6 +629,16 @@ const PlaygroundScreen = ({ connectionError, inspection, onNavigate, onRunChange ; }; +const RoutesScreen = ({ catalog, connectionError, onNavigate, pages, runtimeDiagnostic }: { + readonly catalog: RouteCatalog; + readonly connectionError?: string; + readonly onNavigate: (page: WorkbenchPage) => void; + readonly pages: ReadonlySet; + readonly runtimeDiagnostic: string | undefined; +}) => + +; + const LogsScreen = ({ connectionError, logClient, onNavigate, pages, runtimeDiagnostic }: { readonly connectionError?: string; readonly logClient: LogClient; @@ -776,6 +790,7 @@ const Workbench = () => { const hookClient = useRef(undefined); const logClient = useRef(undefined); const playgroundClient = useRef(undefined); + const routeManifestClient = useRef(undefined); const [connectionError, setConnectionError] = useState(); const [connection, setConnection] = useState({ state: 'connecting' }); const [capabilityRetry, setCapabilityRetry] = useState(0); @@ -852,9 +867,11 @@ const Workbench = () => { if (hookClient.current === undefined) hookClient.current = new HookClient({ foreground: foregroundClient }); if (logClient.current === undefined) logClient.current = new LogClient({ foreground: foregroundClient }); if (playgroundClient.current === undefined) playgroundClient.current = new PlaygroundClient({ foreground: foregroundClient }); + if (routeManifestClient.current === undefined) routeManifestClient.current = new RouteManifestClient({ foreground: foregroundClient }); const runtimeAvailable = runtimeCapability === 'available'; const buildId = status === undefined ? undefined : activeEpochId(status); + const epochSourceRevision = status === undefined ? undefined : activeEpochFor(status)?.projectRevision; // Serve the last loaded catalog while a new epoch's catalog loads. A build // flip must not unmount live content: a Runtime App preview keeps its // session across artifact-only changes (for example a prebuilt payload @@ -1244,7 +1261,9 @@ const Workbench = () => { void loadWorkbenchCapabilities({ artifactClient: artifactClient.current!, buildId, + ...(epochSourceRevision === undefined ? {} : { epochSourceRevision }), evalClient: evalClient.current!, + routeManifestClient: routeManifestClient.current!, signal: request.signal, skillClient: skillClient.current!, }).then( @@ -1263,7 +1282,7 @@ const Workbench = () => { }, ); return () => request.abort(); - }, [buildId, capabilityRetry]); + }, [buildId, capabilityRetry, epochSourceRevision]); useEffect(() => { if (!routesReady) return undefined; @@ -1359,6 +1378,15 @@ const Workbench = () => { status={status} />); } + if (page === 'routes') { + return withConnectionGate(); + } if (page === 'logs') { return withConnectionGate( = z.strictObject({ + code: z.string(), + generatedPath: z.string().optional(), + message: z.string(), + recovery: z.string().optional(), + severity: z.enum(['error', 'info', 'warning']), + sourcePath: z.string().optional(), + target: z.string().optional(), +}); + +const configEntrySchema: z.ZodType = z.strictObject({ + key: z.string(), + kind: z.enum(['array', 'boolean', 'null', 'number', 'object', 'string']), + value: z.string(), +}); + +const routeSchema: z.ZodType = z.strictObject({ + config: z.array(configEntrySchema), + description: z.string().optional(), + event: z.string().optional(), + id: z.string(), + kind: z.enum(['app', 'cli', 'event-route', 'prompt', 'resource', 'script', 'tool']), + provenance: z.strictObject({ kind: z.literal('conventional') }), + serverId: z.string().optional(), + source: z.string(), +}); + +const serverSchema: z.ZodType = z.strictObject({ + id: z.string(), + mode: z.enum(['command', 'conflict', 'custom', 'generated', 'remote']), + name: z.string(), + routes: z.array(routeSchema), +}); + +const cliOptionSchema: z.ZodType = z.strictObject({ + choices: z.array(z.string()).optional(), + description: z.string().optional(), + key: z.string(), + kind: z.enum(['boolean', 'enum', 'number', 'string']), + option: z.string(), + positional: z.number().int().nonnegative().optional(), + repeated: z.boolean(), + required: z.boolean(), +}); + +const cliCommandSchema: z.ZodType = z.strictObject({ + aliases: z.array(z.string()), + description: z.string().optional(), + exitCode: z.enum(['result', 'zero']), + options: z.array(cliOptionSchema), + path: z.array(z.string()), + routeId: z.string(), +}); + +const cliSchema: z.ZodType = z.strictObject({ + commands: z.array(cliCommandSchema).optional(), + mode: z.enum(['conflict', 'conventional', 'generated']), + routes: z.array(routeSchema), +}); + +const providerSchema: z.ZodType = z.strictObject({ + id: z.string(), + name: z.string(), + source: z.string(), +}); + +const manifestSchema: z.ZodType = z.strictObject({ + cli: cliSchema.optional(), + diagnostics: z.array(diagnosticSchema), + digest: z.string(), + events: z.array(routeSchema), + providers: z.array(providerSchema), + scripts: z.array(routeSchema), + servers: z.array(serverSchema), + sourceRevision: z.string(), +}); + +const responseSchema = z.strictObject({ manifest: manifestSchema }); + +const isRecord = (value: unknown): value is Readonly> => + typeof value === 'object' && value !== null && !Array.isArray(value); + +const diagnosticError = (value: unknown, status: number): RouteManifestClientError => { + if ( + isRecord(value) && isRecord(value.diagnostic) && + typeof value.diagnostic.code === 'string' && typeof value.diagnostic.message === 'string' + ) { + return new RouteManifestClientError(value.diagnostic.code, value.diagnostic.message, status); + } + return new RouteManifestClientError('AB8123', `Route manifest request failed with HTTP ${String(status)}.`, status); +}; + +const manifestBody = (value: unknown): RouteManifest => { + const result = responseSchema.safeParse(value); + if (!result.success) throw new RouteManifestClientError('AB8123', 'Route manifest route returned an invalid response.'); + return Object.freeze(result.data.manifest); +}; + +/** Reads the one compiled route graph the dev server already produced; the browser never re-discovers routes. */ +export class RouteManifestClient { + readonly #foreground: ForegroundRequestAuthority; + + constructor(options: RouteManifestClientOptions) { + this.#foreground = options.foreground; + } + + async manifest(signal?: AbortSignal): Promise { + const response = await this.#foreground.protectedRequest('/api/routes/manifest', signal === undefined ? {} : { signal }); + const body: unknown = await response.json().catch(() => undefined); + if (!response.ok) throw diagnosticError(body, response.status); + return manifestBody(body); + } +} diff --git a/packages/workbench/src/routes/routes-model.ts b/packages/workbench/src/routes/routes-model.ts new file mode 100644 index 000000000..64df35dd7 --- /dev/null +++ b/packages/workbench/src/routes/routes-model.ts @@ -0,0 +1,170 @@ +import type { Diagnostic } from '../../../agent-bundle/src/contracts/diagnostics.ts'; +import type { + RouteManifest, + RouteManifestCliCommand, + RouteManifestConfigEntry, + RouteManifestKind, + RouteManifestRoute, +} from '../../../agent-bundle/src/contracts/routes.ts'; + +/** + * The catalog's freshness against the published build. `stale` means the dev + * server has compiled newer source than the epoch the rest of the Workbench + * is scoped to; the catalog stays readable and says so rather than vanishing. + */ +export type RouteCatalogState = 'current' | 'stale' | 'unavailable'; + +/** The catalog group kinds the compiled graph can populate, in navigation order. */ +export const routeCatalogKinds = Object.freeze([ + 'tool', + 'resource', + 'prompt', + 'app', + 'event-route', + 'cli', + 'script', +] as const satisfies readonly RouteManifestKind[]); + +export interface RouteCatalogEntry { + readonly command?: RouteManifestCliCommand; + readonly config: readonly RouteManifestConfigEntry[]; + readonly description?: string; + readonly event?: string; + readonly id: string; + readonly kind: RouteManifestKind; + readonly provenance: 'conventional'; + readonly source: string; +} + +/** + * One catalog section. MCP kinds carry the owning server; `cli` and `script` + * are project-level surfaces, so their `server` stays undefined. + */ +export interface RouteCatalogGroup { + readonly entries: readonly RouteCatalogEntry[]; + readonly kind: RouteManifestKind; + readonly label: string; + readonly mode?: string; + readonly server?: string; + readonly serverId?: string; +} + +export interface RouteCatalogProvider { + readonly id: string; + readonly name: string; + readonly source: string; +} + +export interface RouteCatalog { + readonly diagnostics: readonly Diagnostic[]; + readonly digest: string; + readonly groups: readonly RouteCatalogGroup[]; + /** Present only when the catalog could not be read; `state` is `unavailable`. */ + readonly message?: string; + readonly providers: readonly RouteCatalogProvider[]; + readonly routeCount: number; + readonly sourceRevision?: string; + readonly state: RouteCatalogState; +} + +const kindLabels: Readonly> = Object.freeze({ + app: 'MCP Apps', + cli: 'CLI commands', + 'event-route': 'Event routes', + prompt: 'Prompts', + resource: 'Resources', + script: 'Scripts', + tool: 'Tools', +}); + +export const routeKindLabel = (kind: RouteManifestKind): string => kindLabels[kind]; + +const byId = (left: RouteCatalogEntry, right: RouteCatalogEntry): number => left.id.localeCompare(right.id); + +const entryFor = (route: RouteManifestRoute, command?: RouteManifestCliCommand): RouteCatalogEntry => Object.freeze({ + ...(command === undefined ? {} : { command }), + config: route.config, + ...(route.description === undefined ? {} : { description: route.description }), + ...(route.event === undefined ? {} : { event: route.event }), + id: route.id, + kind: route.kind, + provenance: route.provenance.kind, + source: route.source, +}); + +const groupFor = ( + kind: RouteManifestKind, + entries: readonly RouteCatalogEntry[], + server?: Readonly<{ id: string; mode: string; name: string }>, +): RouteCatalogGroup => Object.freeze({ + entries: Object.freeze([...entries].sort(byId)), + kind, + label: server === undefined ? kindLabels[kind] : `${server.name} · ${kindLabels[kind]}`, + ...(server === undefined ? {} : { mode: server.mode, server: server.name, serverId: server.id }), +}); + +const serverGroups = (manifest: RouteManifest): readonly RouteCatalogGroup[] => + [...manifest.servers] + .sort((left, right) => left.name.localeCompare(right.name)) + .flatMap((server) => routeCatalogKinds + .map((kind) => Object.freeze({ entries: server.routes.filter((route) => route.kind === kind).map((route) => entryFor(route)), kind })) + .filter((group) => group.entries.length > 0) + .map((group) => groupFor(group.kind, group.entries, { id: server.id, mode: server.mode, name: server.name }))); + +const cliGroups = (manifest: RouteManifest): readonly RouteCatalogGroup[] => { + const cli = manifest.cli; + if (cli === undefined || cli.routes.length === 0) return []; + const commands = new Map((cli.commands ?? []).map((command) => [command.routeId, command])); + return [Object.freeze({ + entries: Object.freeze(cli.routes.map((route) => entryFor(route, commands.get(route.id))).sort(byId)), + kind: 'cli' as const, + label: kindLabels.cli, + mode: cli.mode, + })]; +}; + +const projectGroups = (manifest: RouteManifest): readonly RouteCatalogGroup[] => [ + ...(manifest.events.length === 0 ? [] : [groupFor('event-route', manifest.events.map((route) => entryFor(route)))]), + ...cliGroups(manifest), + ...(manifest.scripts.length === 0 ? [] : [groupFor('script', manifest.scripts.map((route) => entryFor(route)))]), +]; + +/** + * Projects the compiled route manifest into the Workbench catalog. `epochSourceRevision` + * is the published build's project revision: an unequal manifest revision is normal + * mid-rebuild drift, reported as `stale` rather than an error. + */ +export const routeCatalogFor = ( + manifest: RouteManifest, + epochSourceRevision?: string, +): RouteCatalog => { + const groups = Object.freeze([...serverGroups(manifest), ...projectGroups(manifest)]); + return Object.freeze({ + diagnostics: manifest.diagnostics, + digest: manifest.digest, + groups, + providers: Object.freeze([...manifest.providers] + .map((provider) => Object.freeze({ id: provider.id, name: provider.name, source: provider.source })) + .sort((left, right) => left.name.localeCompare(right.name))), + routeCount: groups.reduce((total, group) => total + group.entries.length, 0), + sourceRevision: manifest.sourceRevision, + state: epochSourceRevision === undefined || epochSourceRevision === manifest.sourceRevision ? 'current' : 'stale', + }); +}; + +export const unavailableRouteCatalog = (message: string): RouteCatalog => Object.freeze({ + diagnostics: Object.freeze([]), + digest: '', + groups: Object.freeze([]), + message, + providers: Object.freeze([]), + routeCount: 0, + state: 'unavailable', +}); + +/** True when the compiled graph itself declares this kind, whatever configuration adds beside it. */ +export const routeCatalogHasKind = (catalog: RouteCatalog, kind: RouteManifestKind): boolean => + catalog.groups.some((group) => group.kind === kind && group.entries.length > 0); + +export const routeCatalogServerCount = (catalog: RouteCatalog): number => + new Set(catalog.groups.flatMap((group) => group.serverId === undefined ? [] : [group.serverId])).size; diff --git a/packages/workbench/src/routes/routes-page.css b/packages/workbench/src/routes/routes-page.css new file mode 100644 index 000000000..f33f1b197 --- /dev/null +++ b/packages/workbench/src/routes/routes-page.css @@ -0,0 +1,31 @@ +.routes-content { margin: 0 auto; max-width: 1180px; min-width: 0; padding: 35px 34px 64px; width: 100%; } +.routes-page-heading p { color: #596372; font-size: 15px; margin: 8px 0 0; max-width: 760px; } +.route-identity { border-top: 1px solid #d9dee7; margin-top: 24px; padding-top: 18px; } +.route-identity dl { display: grid; gap: 0; grid-template-columns: repeat(auto-fit, minmax(190px, 1fr)); margin: 0; } +.route-identity dl div { border-right: 1px solid #d9dee7; min-width: 0; padding: 12px 14px 12px 0; } +.route-identity dl div:last-child { border-right: 0; } +.route-identity dt { color: #596372; font-size: 12px; font-weight: 750; letter-spacing: .03em; margin: 0 0 6px; text-transform: uppercase; } +.route-identity dd { font-size: 14px; font-weight: 600; margin: 0; overflow-wrap: anywhere; } +.route-digest { font: 12px/1.5 "SFMono-Regular", Consolas, "Liberation Mono", monospace; font-weight: 500; } +.route-state { text-transform: capitalize; } +.route-state--current { color: #14682f; } +.route-state--stale { color: #8a5300; } +.route-state--unavailable { color: #b31b23; } +.route-diagnostics { background: #fff7f7; border-left: 3px solid #c01d26; color: #78242a; margin: 22px 0 0; padding: 9px 14px; } +.route-diagnostics h2 { color: #78242a; font-size: 15px; margin: 6px 0; } +.route-diagnostics p { align-items: baseline; display: flex; flex-wrap: wrap; font-size: 13px; gap: 8px; margin: 5px 0; } +.route-diagnostic-code { font: 12px/1.5 "SFMono-Regular", Consolas, "Liberation Mono", monospace; font-weight: 700; } +.route-group { border-top: 1px solid #d9dee7; margin-top: 26px; padding-top: 18px; } +.route-group-heading { align-items: baseline; display: flex; flex-wrap: wrap; gap: 12px; justify-content: space-between; } +.route-group-heading h2 { font-size: 17px; margin: 0; } +.route-group-heading p { color: #596372; font-size: 13px; margin: 0; text-transform: lowercase; } +.route-table { border-collapse: collapse; margin-top: 14px; table-layout: fixed; width: 100%; } +.route-table th, .route-table td { border-bottom: 1px solid #e4e8ef; padding: 11px 12px 11px 0; text-align: left; vertical-align: top; } +.route-table thead th { color: #596372; font-size: 12px; font-weight: 750; letter-spacing: .03em; text-transform: uppercase; } +.route-table tbody th { font-weight: 600; width: 34%; } +.route-id { display: block; font: 13px/1.5 "SFMono-Regular", Consolas, "Liberation Mono", monospace; font-weight: 700; overflow-wrap: anywhere; } +.route-event, .route-command { color: #345080; display: block; font: 12px/1.5 "SFMono-Regular", Consolas, "Liberation Mono", monospace; margin-top: 4px; overflow-wrap: anywhere; } +.route-description { color: #596372; display: block; font-size: 13px; font-weight: 400; margin-top: 4px; } +.route-source { font: 12px/1.5 "SFMono-Regular", Consolas, "Liberation Mono", monospace; overflow-wrap: anywhere; width: 30%; } +.route-provenance { color: #7a8492; display: block; font-family: inherit; font-size: 11px; margin-top: 4px; text-transform: uppercase; } +.route-config { color: #4f5866; font-size: 13px; overflow-wrap: anywhere; } diff --git a/packages/workbench/src/routes/routes-page.tsx b/packages/workbench/src/routes/routes-page.tsx new file mode 100644 index 000000000..1eb31fc00 --- /dev/null +++ b/packages/workbench/src/routes/routes-page.tsx @@ -0,0 +1,128 @@ +import React from 'react'; + +import type { RouteCatalog, RouteCatalogEntry, RouteCatalogGroup } from './routes-model.ts'; +import './routes-page.css'; + +export interface RoutesPageProps { + readonly catalog: RouteCatalog; +} + +const stateSummaries: Readonly> = Object.freeze({ + current: 'This catalog is the compiled route graph the published build was produced from.', + stale: 'The dev server has compiled newer source than the published build. Rebuild to publish these routes.', + unavailable: 'The compiled route manifest could not be read from the foreground server.', +}); + +const counted = (count: number, singular: string, plural = `${singular}s`): string => + `${String(count)} ${count === 1 ? singular : plural}`; + +/** + * `description` is projected as the route's own label, so repeating it here + * would print the same sentence twice on every row. + */ +const configSummary = (entry: RouteCatalogEntry): string => { + if (entry.config.length === 0) return 'No static config export'; + const fields = entry.config.filter((field) => !(field.key === 'description' && entry.description !== undefined)); + return fields.length === 0 + ? 'No config beyond the description' + : fields.map((field) => `${field.key}: ${field.value}`).join(' · '); +}; + +/** + * A usage line, so positionals lead in their argv order and flags follow — + * the compiler orders options by key, which is not the order they are typed. + */ +const commandSummary = (entry: RouteCatalogEntry): string | undefined => { + const command = entry.command; + if (command === undefined) return undefined; + const positionals = command.options.filter((option) => option.positional !== undefined) + .toSorted((left, right) => left.positional! - right.positional!) + .map((option) => option.repeated ? `[<${option.key}>…]` : `<${option.key}>`); + const flags = command.options.filter((option) => option.positional === undefined) + .map((option) => option.required ? `--${option.option}` : `[--${option.option}]`); + return [...command.path, ...positionals, ...flags].join(' '); +}; + +const RouteGroup = ({ group }: { readonly group: RouteCatalogGroup }) =>
+
+

{group.label}

+

+ {counted(group.entries.length, 'route')} + {group.mode === undefined ? '' : ` · ${group.mode}`} +

+
+ + + + + + + + + {group.entries.map((entry) => + + + + )} +
Route IDSourceConfig
+ {entry.id} + {entry.event === undefined ? undefined : {entry.event}} + {commandSummary(entry) === undefined ? undefined : {commandSummary(entry)}} + {entry.description === undefined ? undefined : {entry.description}} + + {entry.source} + {entry.provenance} + {configSummary(entry)}
+
; + +/** + * The compiled route catalog: one read of the same manifest the build, inspect, + * and test harness use. Discovery runs once in the compiler; this page renders it. + */ +export const RoutesPage = ({ catalog }: RoutesPageProps) =>
+
+
+

Routes

+

{stateSummaries[catalog.state]}

+
+
+ {catalog.state === 'unavailable' + ?

{catalog.message ?? stateSummaries.unavailable}

+ : <> +
+
+
Routes
{catalog.routeCount}
+
Graph digest
{catalog.digest === '' ? '—' : catalog.digest}
+
Source revision
{catalog.sourceRevision ?? '—'}
+
State
{catalog.state}
+
+
+ {catalog.diagnostics.length === 0 ? undefined :
+

Route diagnostics ({catalog.diagnostics.length})

+ {catalog.diagnostics.map((diagnostic, index) =>

+ {diagnostic.severity} + {diagnostic.code} + {diagnostic.message} +

)} +
} + {catalog.groups.length === 0 + ?

This project declares no conventional route modules.

+ : catalog.groups.map((group) => )} + {catalog.providers.length === 0 ? undefined :
+
+

Context providers

+

{counted(catalog.providers.length, 'provider')}

+
+ + + {catalog.providers.map((provider) => + + + )} +
Provider IDSource
{provider.id}{provider.source}
+
} + } +
; diff --git a/packages/workbench/src/workbench-capabilities.ts b/packages/workbench/src/workbench-capabilities.ts index d8e6b4fce..dffaeac02 100644 --- a/packages/workbench/src/workbench-capabilities.ts +++ b/packages/workbench/src/workbench-capabilities.ts @@ -3,6 +3,14 @@ import type { SkillDocumentTree } from '../../agent-bundle/src/contracts/skills. import type { ArtifactClient } from './artifacts/artifact-client.ts'; import type { EvalClient } from './evals/eval-client.ts'; +import type { RouteManifestClient } from './routes/route-manifest-client.ts'; +import { + routeCatalogFor, + routeCatalogHasKind, + routeCatalogServerCount, + unavailableRouteCatalog, + type RouteCatalog, +} from './routes/routes-model.ts'; import type { SkillClient } from './skill-client.ts'; import type { WorkbenchPage } from './workbench-screen.tsx'; @@ -18,13 +26,18 @@ export interface WorkbenchCapabilities { }>; readonly inspection: ArtifactInspection; readonly pages: ReadonlySet; + /** The compiled route graph this build was produced from, projected for the browser. */ + readonly routes: RouteCatalog; readonly skillTree: SkillDocumentTree; } export interface WorkbenchCapabilityClients { readonly artifactClient: Pick; readonly buildId: string; + /** The published epoch's project revision, used to detect a newer compiled manifest. */ + readonly epochSourceRevision?: string; readonly evalClient: Pick; + readonly routeManifestClient: Pick; readonly signal?: AbortSignal; readonly skillClient: Pick; } @@ -35,31 +48,66 @@ export const generalWorkbenchPages: ReadonlySet = Object.freeze(n 'logs', ])); -const pagesFor = (counts: WorkbenchCapabilities['counts']): ReadonlySet => { - const pages: WorkbenchPage[] = ['overview']; +/** + * Navigation derives from the compiled route graph wherever the graph declares + * the surface, and from the artifact catalog for everything configuration can + * declare without a route module. The union is deliberate: a project may reach + * a page through either source, and neither may hide the other. + */ +const pagesFor = ( + counts: WorkbenchCapabilities['counts'], + routes: RouteCatalog, +): ReadonlySet => { + const compiledEvents = routeCatalogHasKind(routes, 'event-route'); + const compiledScripts = routeCatalogHasKind(routes, 'script'); + const pages: WorkbenchPage[] = ['overview', 'routes']; if (counts.skills > 0) pages.push('skills'); - if (counts.hooks > 0) pages.push('hooks'); - if (counts.mcpServers > 0) pages.push('mcp'); + if (counts.hooks > 0 || compiledEvents) pages.push('hooks'); + if (counts.mcpServers > 0 || routeCatalogServerCount(routes) > 0) pages.push('mcp'); pages.push('artifacts'); - if (counts.hooks + counts.scripts > 0) pages.push('playground'); + if (counts.hooks + counts.scripts > 0 || compiledEvents || compiledScripts) pages.push('playground'); pages.push('logs'); if (counts.evalSuites > 0) pages.push('evals', 'comparisons'); return Object.freeze(new Set(pages)); }; +const errorMessage = (reason: unknown): string => + reason instanceof Error ? reason.message : 'The compiled route manifest could not be read.'; + +/** + * An absent or refused manifest route degrades this one section rather than the + * whole catalog: every page that predates the manifest keeps its artifact-derived + * evidence, so the Workbench stays usable against a dev server without the route. + */ +const routeCatalog = async ( + client: Pick, + epochSourceRevision: string | undefined, + signal: AbortSignal | undefined, +): Promise => { + try { + return routeCatalogFor(await client.manifest(signal), epochSourceRevision); + } catch (reason) { + if (reason instanceof Error && reason.name === 'AbortError') throw reason; + return unavailableRouteCatalog(errorMessage(reason)); + } +}; + /** Composes existing strict route catalogs into one build-scoped Workbench view. */ export const loadWorkbenchCapabilities = async ({ artifactClient, buildId, + epochSourceRevision, evalClient, + routeManifestClient, signal, skillClient, }: WorkbenchCapabilityClients): Promise => { signal?.throwIfAborted(); - const [inspection, skillTree, evalListing] = await Promise.all([ + const [inspection, skillTree, evalListing, routes] = await Promise.all([ artifactClient.inspect(buildId, signal), skillClient.sourceTree(), evalClient.suites(), + routeCatalog(routeManifestClient, epochSourceRevision, signal), ]); signal?.throwIfAborted(); if (inspection.epochId !== buildId) throw new Error('Capability catalog did not match the current build.'); @@ -71,5 +119,12 @@ export const loadWorkbenchCapabilities = async ({ skills: skillTree.skills.length, targets: inspection.targets.length, }); - return Object.freeze({ buildId, counts, inspection, pages: pagesFor(counts), skillTree }); + return Object.freeze({ + buildId, + counts, + inspection, + pages: pagesFor(counts, routes), + routes, + skillTree, + }); }; diff --git a/packages/workbench/src/workbench-screen.tsx b/packages/workbench/src/workbench-screen.tsx index 9149ac285..6ad11a58f 100644 --- a/packages/workbench/src/workbench-screen.tsx +++ b/packages/workbench/src/workbench-screen.tsx @@ -1,6 +1,6 @@ import React, { type ReactNode } from 'react'; -export type WorkbenchPage = 'artifacts' | 'comparisons' | 'evals' | 'hooks' | 'logs' | 'mcp' | 'overview' | 'playground' | 'skills'; +export type WorkbenchPage = 'artifacts' | 'comparisons' | 'evals' | 'hooks' | 'logs' | 'mcp' | 'overview' | 'playground' | 'routes' | 'skills'; interface NavigationItem { readonly glyph: string; @@ -14,7 +14,13 @@ interface NavigationGroup { } const navigationGroups: readonly NavigationGroup[] = [ - { items: [{ glyph: '⊞', label: 'Overview', page: 'overview' }], label: 'Build' }, + { + items: [ + { glyph: '⊞', label: 'Overview', page: 'overview' }, + { glyph: '⌸', label: 'Routes', page: 'routes' }, + ], + label: 'Build', + }, { items: [ { glyph: '⌘', label: 'Skills', page: 'skills' }, diff --git a/packages/workbench/tests/examples-real.e2e.test.ts b/packages/workbench/tests/examples-real.e2e.test.ts index 44c13dd80..f3474d3d1 100644 --- a/packages/workbench/tests/examples-real.e2e.test.ts +++ b/packages/workbench/tests/examples-real.e2e.test.ts @@ -232,6 +232,23 @@ e2e('drives Hooks, scripts, logs, diagnostics, and repair in real Chrome', { tim await expect(page.getByRole('button', { name: 'Run script' })).toBeEnabled({ timeout: browserTimeout }); await captureExampleState(page, 'hooks-and-scripts', 'script-failure'); + await page.getByRole('link', { name: 'Routes', exact: true }).click(); + await waitForSettledWorkbench(page); + await expect(page.getByRole('heading', { name: 'Routes', exact: true })).toBeVisible({ timeout: browserTimeout }); + await expect(page.getByText('This catalog is the compiled route graph the published build was produced from.', { exact: true })).toBeVisible({ timeout: browserTimeout }); + // verify-release is discovered by convention; detect-risk is configured, so + // the compiled catalog must show exactly one script route. + await expect(page.getByRole('heading', { name: 'Scripts', exact: true })).toBeVisible({ timeout: browserTimeout }); + await expect(page.locator('.route-table')).toContainText('script:verify-release', { timeout: browserTimeout }); + await expect(page.locator('.route-table')).toContainText('src/scripts/verify-release.ts', { timeout: browserTimeout }); + await expect(page.locator('.route-table')).not.toContainText('script:detect-risk'); + await expect(page.locator('.route-state')).toHaveText('current', { timeout: browserTimeout }); + // Provenance renders under its source rather than running into it, which is + // only true when the catalog stylesheet actually reached the document. + await expect(page.locator('.route-provenance')).toHaveText('conventional', { timeout: browserTimeout }); + expect(await page.locator('.route-provenance').evaluate((node) => getComputedStyle(node).display)).toBe('block'); + await captureExampleState(page, 'hooks-and-scripts', 'routes-catalog'); + await page.getByRole('link', { name: 'Logs' }).click(); await waitForSettledWorkbench(page); await expect(page.locator('.logs-entries > li').first()).toBeVisible({ timeout: browserTimeout }); @@ -344,6 +361,18 @@ e2e('drives every populated MCP App workflow surface in real Chrome', { timeout: await expect(page.locator('.comparisons-content [role="status"]')).toHaveText('At least two recorded runs are needed before a comparison can be aligned.', { timeout: browserTimeout }); await captureExampleState(page, 'mcp-app', 'comparisons-insufficient-runs'); + await page.getByRole('link', { name: 'Routes', exact: true }).click(); + await waitForSettledWorkbench(page); + await expect(page.getByRole('heading', { name: 'Routes', exact: true })).toBeVisible({ timeout: browserTimeout }); + // Every capability here is configured rather than routed: the compiled + // catalog reports an empty graph while all nine pages stay navigable. + await expect(page.getByText('This project declares no conventional route modules.', { exact: true })).toBeVisible({ timeout: browserTimeout }); + await expect(page.locator('.route-table')).toHaveCount(0); + for (const preserved of ['Overview', 'Skills', 'Hooks', 'MCP playground', 'Artifacts', 'Playground', 'Logs', 'Evals', 'Comparisons']) { + await expect(page.getByRole('link', { name: preserved, exact: true })).toBeVisible({ timeout: browserTimeout }); + } + await captureExampleState(page, 'mcp-app', 'routes-empty-graph'); + await page.getByRole('link', { name: 'MCP playground', exact: true }).click(); await waitForSettledWorkbench(page); await page.waitForFunction(() => document.querySelector('#mcp-server-name')?.value === 'status', undefined, { timeout: browserTimeout }); @@ -538,3 +567,72 @@ e2e('drives every populated MCP App workflow surface in real Chrome', { timeout: await project.release(); } }); + +e2e('renders the flagship compiled route catalog by server and kind in real Chrome', { timeout: 150_000 }, async ({ page }) => { + await buildWorkbench(); + const project = await copyExample('audiobook-curator'); + const server = await startDevServer({ + assets: createWorkbenchAssetSource({ root: workbenchAssets }), + open: false, + port: 0, + root: project.root, + }); + const ledger = createExampleErrorLedger(page, server.url); + try { + await page.goto(workbenchUrl(server.url, 'routes')); + await waitForSettledWorkbench(page); + await expect(page.getByRole('heading', { name: 'Routes', exact: true })).toBeVisible({ timeout: browserTimeout }); + await expect(page.locator('.route-state')).toHaveText('current', { timeout: browserTimeout }); + + // One generated server owns every MCP kind the curator declares, so each + // kind must appear as its own server-scoped group rather than a flat list. + for (const group of ['curator · Tools', 'curator · Resources', 'curator · Prompts']) { + await expect(page.getByRole('heading', { name: group, exact: true })).toBeVisible({ timeout: browserTimeout }); + } + await expect(page.locator('.route-group-heading').filter({ hasText: 'curator · Tools' })).toContainText('generated', { timeout: browserTimeout }); + const tools = page.getByRole('region', { name: 'curator · Tools' }); + await expect(tools.locator('tbody tr')).toHaveCount(15, { timeout: browserTimeout }); + await expect(tools).toContainText('tool:curator/convert_audiobook', { timeout: browserTimeout }); + await expect(tools).toContainText('src/mcp/curator/tools/convert_audiobook.tsx', { timeout: browserTimeout }); + // The extracted config is summarized, never inlined as nested JSON. + await expect(tools).toContainText('annotations: 2 keys', { timeout: browserTimeout }); + + const resources = page.getByRole('region', { name: 'curator · Resources' }); + await expect(resources).toContainText('resource:curator/catalog', { timeout: browserTimeout }); + await expect(resources).toContainText('uri: audiobook-curator://catalog', { timeout: browserTimeout }); + await expect(page.getByRole('region', { name: 'curator · Prompts' })).toContainText('prompt:curator/curate', { timeout: browserTimeout }); + await expect(page.locator('.route-provenance').first()).toHaveText('conventional', { timeout: browserTimeout }); + + // The generated CLI is a project surface rather than a server one, and each + // command carries the argv projection compiled from its input schema. + const cli = page.getByRole('region', { name: 'CLI commands' }); + await expect(cli.locator('tbody tr')).toHaveCount(15, { timeout: browserTimeout }); + await expect(cli).toContainText('cli:library-audit', { timeout: browserTimeout }); + await expect(cli).toContainText('src/cli/library-audit.tsx', { timeout: browserTimeout }); + await expect(cli.locator('.route-command').filter({ hasText: 'library-audit' })) + .toHaveText('library-audit […] [--concurrency] --report [--strict]', { timeout: browserTimeout }); + await expect(cli.locator('.route-command').filter({ hasText: 'inspect' })) + .toHaveText('inspect [--max-files]', { timeout: browserTimeout }); + + // 17 MCP routes plus 15 CLI routes, and nothing invented: the curator + // declares no conventional event routes, scripts, or context providers. + await expect(page.locator('.route-identity')).toContainText('32', { timeout: browserTimeout }); + await expect(page.getByRole('heading', { name: 'Event routes', exact: true })).toHaveCount(0); + await expect(page.getByRole('heading', { name: 'Scripts', exact: true })).toHaveCount(0); + await expect(page.getByRole('heading', { name: 'Context providers', exact: true })).toHaveCount(0); + await expect(page.locator('.route-diagnostics')).toHaveCount(0); + await captureExampleState(page, 'audiobook-curator', 'routes-catalog-by-server'); + + for (const preserved of ['Overview', 'Skills', 'Artifacts', 'Logs']) { + await expect(page.getByRole('link', { name: preserved, exact: true })).toBeVisible({ timeout: browserTimeout }); + } + await page.getByRole('link', { name: 'Overview', exact: true }).click(); + await waitForSettledWorkbench(page); + await expect(page.getByRole('heading', { name: 'Bundle dashboard', exact: true })).toBeVisible({ timeout: browserTimeout }); + await expectHealthyExamplePage(ledger); + await writeExampleReport(); + } finally { + await server.close(); + await project.release(); + } +}); diff --git a/packages/workbench/tests/route-manifest-client.test.ts b/packages/workbench/tests/route-manifest-client.test.ts new file mode 100644 index 000000000..6a9065040 --- /dev/null +++ b/packages/workbench/tests/route-manifest-client.test.ts @@ -0,0 +1,170 @@ +import { expect, it } from '@rstest/core'; + +import { ForegroundRouteClient } from '../src/mcp/mcp-route-client.ts'; +import { RouteManifestClient } from '../src/routes/route-manifest-client.ts'; + +interface RecordedRequest { + readonly method: string; + readonly token: string | null; + readonly url: string; +} + +const response = (body: unknown, status = 200): Response => new Response(JSON.stringify(body), { + headers: { 'content-type': 'application/json' }, + status, +}); + +const manifest = { + cli: { + commands: [{ + aliases: ['a'], + description: 'Audit the library', + exitCode: 'result', + options: [{ + choices: ['json', 'text'], + description: 'Output format', + key: 'format', + kind: 'enum', + option: 'format', + repeated: false, + required: false, + }], + path: ['library', 'audit'], + routeId: 'cli:library/audit', + }], + mode: 'generated', + routes: [{ + config: [{ key: 'description', kind: 'string', value: 'Audit the library' }], + description: 'Audit the library', + id: 'cli:library/audit', + kind: 'cli', + provenance: { kind: 'conventional' }, + source: 'src/cli/library/audit.ts', + }], + }, + diagnostics: [{ code: 'AB4800', message: 'Two routes claim the same id.', severity: 'error' }], + digest: 'd'.repeat(64), + events: [{ + config: [], + event: 'afterTool', + id: 'event:tool/after', + kind: 'event-route', + provenance: { kind: 'conventional' }, + source: 'src/events/tool/after.ts', + }], + providers: [{ id: 'provider:library', name: 'library', source: 'src/providers/library.ts' }], + scripts: [{ + config: [], + id: 'script:convert', + kind: 'script', + provenance: { kind: 'conventional' }, + source: 'src/scripts/convert.ts', + }], + servers: [{ + id: 'mcp:library', + mode: 'generated', + name: 'library', + routes: [{ + config: [{ key: 'title', kind: 'string', value: 'Echo' }], + id: 'tool:library/echo', + kind: 'tool', + provenance: { kind: 'conventional' }, + serverId: 'mcp:library', + source: 'src/mcp/library/tools/echo.ts', + }], + }], + sourceRevision: 'r'.repeat(64), +}; + +const recordingFetch = (calls: RecordedRequest[], reply: () => Response): typeof fetch => + async (input, init) => { + const url = String(input); + if (url === '/api/project/session') { + return response({ + cookieName: 'agent-bundle-foreground-session-0123456789abcdef0123456789abcdef', + instanceId: 'foreground-instance-a', + origin: 'http://127.0.0.1:5173', + token: 'foreground-token', + }); + } + calls.push({ + method: init?.method ?? 'GET', + token: new Headers(init?.headers).get('x-agent-bundle-session'), + url, + }); + return reply(); + }; + +const clientFor = (reply: () => Response, calls: RecordedRequest[] = []): RouteManifestClient => + new RouteManifestClient({ foreground: new ForegroundRouteClient({ fetch: recordingFetch(calls, reply) }) }); + +it('reads the compiled manifest over the shared foreground session', async () => { + const calls: RecordedRequest[] = []; + const client = clientFor(() => response({ manifest }), calls); + + const decoded = await client.manifest(); + + expect(decoded.digest).toBe('d'.repeat(64)); + expect(decoded.servers[0]?.routes[0]?.id).toBe('tool:library/echo'); + expect(decoded.cli?.commands?.[0]?.path).toEqual(['library', 'audit']); + expect(calls).toEqual([{ method: 'GET', token: 'foreground-token', url: '/api/routes/manifest' }]); +}); + +it('freezes the decoded manifest so no page can mutate compiled route facts', async () => { + const decoded = await clientFor(() => response({ manifest })).manifest(); + + expect(Object.isFrozen(decoded)).toBe(true); +}); + +it('rejects an unknown field on the manifest wire body', async () => { + const client = clientFor(() => response({ manifest: { ...manifest, unexpected: true } })); + + await expect(client.manifest()).rejects.toMatchObject({ + code: 'AB8123', + message: 'Route manifest route returned an invalid response.', + }); +}); + +it('rejects an unknown field on a compiled route', async () => { + const client = clientFor(() => response({ + manifest: { ...manifest, scripts: [{ ...manifest.scripts[0], extra: 1 }] }, + })); + + await expect(client.manifest()).rejects.toMatchObject({ code: 'AB8123' }); +}); + +it('rejects a route kind the compiler cannot emit', async () => { + const client = clientFor(() => response({ + manifest: { ...manifest, scripts: [{ ...manifest.scripts[0], kind: 'provider' }] }, + })); + + await expect(client.manifest()).rejects.toMatchObject({ code: 'AB8123' }); +}); + +it('rejects a sibling body key beside the manifest', async () => { + const client = clientFor(() => response({ manifest, status: 'ok' })); + + await expect(client.manifest()).rejects.toMatchObject({ code: 'AB8123' }); +}); + +it('decodes a foreground diagnostic body into a coded client error', async () => { + const client = clientFor(() => response({ + diagnostic: { code: 'AB8121', message: 'Route manifest is not available.' }, + }, 409)); + + await expect(client.manifest()).rejects.toMatchObject({ + code: 'AB8121', + message: 'Route manifest is not available.', + status: 409, + }); +}); + +it('reports a bodyless failure with the transport status', async () => { + const client = clientFor(() => new Response('', { status: 503 })); + + await expect(client.manifest()).rejects.toMatchObject({ + code: 'AB8123', + message: 'Route manifest request failed with HTTP 503.', + status: 503, + }); +}); diff --git a/packages/workbench/tests/routes-model.test.ts b/packages/workbench/tests/routes-model.test.ts new file mode 100644 index 000000000..48a7800de --- /dev/null +++ b/packages/workbench/tests/routes-model.test.ts @@ -0,0 +1,190 @@ +import { expect, it } from '@rstest/core'; + +import type { RouteManifest } from '../../agent-bundle/src/contracts/routes.ts'; +import { + routeCatalogFor, + routeCatalogHasKind, + routeCatalogServerCount, + routeKindLabel, + unavailableRouteCatalog, +} from '../src/routes/routes-model.ts'; + +const manifest: RouteManifest = { + cli: { + commands: [{ + aliases: [], + exitCode: 'zero', + options: [ + { key: 'input', kind: 'string', option: 'input', positional: 0, repeated: false, required: true }, + { key: 'verbose', kind: 'boolean', option: 'verbose', repeated: false, required: false }, + ], + path: ['library', 'audit'], + routeId: 'cli:library/audit', + }], + mode: 'generated', + routes: [{ + config: [], + id: 'cli:library/audit', + kind: 'cli', + provenance: { kind: 'conventional' }, + source: 'src/cli/library/audit.ts', + }], + }, + diagnostics: [], + digest: 'd'.repeat(64), + events: [{ + config: [{ key: 'targets', kind: 'array', value: '2 entries' }], + event: 'afterTool', + id: 'event:tool/after', + kind: 'event-route', + provenance: { kind: 'conventional' }, + source: 'src/events/tool/after.ts', + }], + providers: [ + { id: 'provider:library', name: 'library', source: 'src/providers/library.ts' }, + { id: 'provider:audio', name: 'audio', source: 'src/providers/audio.ts' }, + ], + scripts: [{ + config: [], + id: 'script:convert', + kind: 'script', + provenance: { kind: 'conventional' }, + source: 'src/scripts/convert.ts', + }], + servers: [ + { + id: 'mcp:zeta', + mode: 'custom', + name: 'zeta', + routes: [{ + config: [], + id: 'prompt:zeta/summarize', + kind: 'prompt', + provenance: { kind: 'conventional' }, + serverId: 'mcp:zeta', + source: 'src/mcp/zeta/prompts/summarize.ts', + }], + }, + { + id: 'mcp:alpha', + mode: 'generated', + name: 'alpha', + routes: [ + { + config: [{ key: 'title', kind: 'string', value: 'Echo' }], + id: 'tool:alpha/echo', + kind: 'tool', + provenance: { kind: 'conventional' }, + serverId: 'mcp:alpha', + source: 'src/mcp/alpha/tools/echo.ts', + }, + { + config: [], + id: 'tool:alpha/build', + kind: 'tool', + provenance: { kind: 'conventional' }, + serverId: 'mcp:alpha', + source: 'src/mcp/alpha/tools/build.ts', + }, + { + config: [], + id: 'resource:alpha/notes', + kind: 'resource', + provenance: { kind: 'conventional' }, + serverId: 'mcp:alpha', + source: 'src/mcp/alpha/resources/notes.ts', + }, + ], + }, + ], + sourceRevision: 'r'.repeat(64), +}; + +it('groups the compiled graph by server then by project surface', () => { + const catalog = routeCatalogFor(manifest, 'r'.repeat(64)); + + expect(catalog.groups.map((group) => group.label)).toEqual([ + 'alpha · Tools', + 'alpha · Resources', + 'zeta · Prompts', + 'Event routes', + 'CLI commands', + 'Scripts', + ]); + expect(catalog.state).toBe('current'); + expect(catalog.routeCount).toBe(7); + expect(routeCatalogServerCount(catalog)).toBe(2); +}); + +it('orders routes within a group by compiled id', () => { + const catalog = routeCatalogFor(manifest); + + expect(catalog.groups[0]?.entries.map((entry) => entry.id)).toEqual(['tool:alpha/build', 'tool:alpha/echo']); +}); + +it('carries the server packaging mode and the CLI surface mode as group metadata', () => { + const catalog = routeCatalogFor(manifest); + + expect(catalog.groups.find((group) => group.label === 'zeta · Prompts')?.mode).toBe('custom'); + expect(catalog.groups.find((group) => group.kind === 'cli')?.mode).toBe('generated'); + expect(catalog.groups.find((group) => group.kind === 'cli')?.server).toBeUndefined(); +}); + +it('attaches the compiled command to its CLI route entry', () => { + const catalog = routeCatalogFor(manifest); + const entry = catalog.groups.find((group) => group.kind === 'cli')?.entries[0]; + + expect(entry?.command?.path).toEqual(['library', 'audit']); + expect(entry?.command?.options.map((option) => option.key)).toEqual(['input', 'verbose']); +}); + +it('sorts providers by name and keeps route provenance on every entry', () => { + const catalog = routeCatalogFor(manifest); + + expect(catalog.providers.map((provider) => provider.name)).toEqual(['audio', 'library']); + expect(catalog.groups.flatMap((group) => group.entries).every((entry) => entry.provenance === 'conventional')).toBe(true); +}); + +it('reports a manifest revision ahead of the published build as stale', () => { + expect(routeCatalogFor(manifest, 'e'.repeat(64)).state).toBe('stale'); + expect(routeCatalogFor(manifest).state).toBe('current'); +}); + +it('answers kind availability from the compiled graph', () => { + const catalog = routeCatalogFor(manifest); + + expect(routeCatalogHasKind(catalog, 'event-route')).toBe(true); + expect(routeCatalogHasKind(catalog, 'script')).toBe(true); + expect(routeCatalogHasKind(catalog, 'app')).toBe(false); +}); + +it('renders an empty compiled graph without groups', () => { + const catalog = routeCatalogFor({ + diagnostics: [], + digest: 'e'.repeat(64), + events: [], + providers: [], + scripts: [], + servers: [], + sourceRevision: 'r'.repeat(64), + }); + + expect(catalog.groups).toEqual([]); + expect(catalog.routeCount).toBe(0); + expect(catalog.state).toBe('current'); +}); + +it('describes an unreadable manifest without inventing routes', () => { + const catalog = unavailableRouteCatalog('Route manifest is not available.'); + + expect(catalog.state).toBe('unavailable'); + expect(catalog.groups).toEqual([]); + expect(catalog.sourceRevision).toBeUndefined(); + expect(routeCatalogHasKind(catalog, 'tool')).toBe(false); +}); + +it('labels every compiled route kind', () => { + expect(routeKindLabel('tool')).toBe('Tools'); + expect(routeKindLabel('event-route')).toBe('Event routes'); + expect(routeKindLabel('app')).toBe('MCP Apps'); +}); diff --git a/packages/workbench/tests/routes-page.test.ts b/packages/workbench/tests/routes-page.test.ts new file mode 100644 index 000000000..ca0485af5 --- /dev/null +++ b/packages/workbench/tests/routes-page.test.ts @@ -0,0 +1,209 @@ +import { createElement } from 'react'; +import { renderToStaticMarkup } from 'react-dom/server'; + +import { expect, it } from '@rstest/core'; + +import type { RouteManifest } from '../../agent-bundle/src/contracts/routes.ts'; +import { routeCatalogFor, unavailableRouteCatalog } from '../src/routes/routes-model.ts'; +import { RoutesPage } from '../src/routes/routes-page.tsx'; + +const manifest: RouteManifest = { + cli: { + commands: [{ + aliases: [], + exitCode: 'zero', + options: [ + { key: 'input', kind: 'string', option: 'input', positional: 0, repeated: false, required: true }, + { key: 'verbose', kind: 'boolean', option: 'verbose', repeated: false, required: false }, + ], + path: ['library', 'audit'], + routeId: 'cli:library/audit', + }], + mode: 'generated', + routes: [{ + config: [], + id: 'cli:library/audit', + kind: 'cli', + provenance: { kind: 'conventional' }, + source: 'src/cli/library/audit.ts', + }], + }, + diagnostics: [{ code: 'AB4801', message: 'Two MCP tool routes claim the same name.', severity: 'error' }], + digest: 'd'.repeat(64), + events: [{ + config: [{ key: 'targets', kind: 'array', value: '2 entries' }], + event: 'afterTool', + id: 'event:tool/after', + kind: 'event-route', + provenance: { kind: 'conventional' }, + source: 'src/events/tool/after.ts', + }], + providers: [{ id: 'provider:library', name: 'library', source: 'src/providers/library.ts' }], + scripts: [], + servers: [{ + id: 'mcp:library', + mode: 'generated', + name: 'library', + routes: [{ + config: [{ key: 'title', kind: 'string', value: 'Echo' }], + description: 'Echo the request back', + id: 'tool:library/echo', + kind: 'tool', + provenance: { kind: 'conventional' }, + serverId: 'mcp:library', + source: 'src/mcp/library/tools/echo.ts', + }], + }], + sourceRevision: 'r'.repeat(64), +}; + +const render = (catalog: Parameters[0]['catalog']): string => + renderToStaticMarkup(createElement(RoutesPage, { catalog })); + +it('renders the compiled catalog grouped by server and project surface', () => { + const markup = render(routeCatalogFor(manifest, 'r'.repeat(64))); + + expect(markup).toContain('library · Tools'); + expect(markup).toContain('>Event routes<'); + expect(markup).toContain('>CLI commands<'); + expect(markup).toContain('tool:library/echo'); + expect(markup).toContain('src/mcp/library/tools/echo.ts'); + expect(markup).toContain('title: Echo'); + expect(markup).toContain('Echo the request back'); + expect(markup).toContain('>Context providers<'); + expect(markup).toContain('provider:library'); +}); + +it('shows the argv projection of a compiled CLI command', () => { + const markup = render(routeCatalogFor(manifest)); + + expect(markup).toContain('library audit <input> [--verbose]'); +}); + +it('leads the usage line with positionals in argv order regardless of option order', () => { + const reordered: RouteManifest = { + ...manifest, + cli: { + ...manifest.cli!, + commands: [{ + aliases: [], + exitCode: 'result', + options: [ + { key: 'concurrency', kind: 'number', option: 'concurrency', repeated: false, required: false }, + { key: 'report', kind: 'string', option: 'report', repeated: false, required: true }, + { key: 'sources', kind: 'string', option: 'sources', positional: 0, repeated: true, required: true }, + ], + path: ['library', 'audit'], + routeId: 'cli:library/audit', + }], + }, + }; + + const markup = render(routeCatalogFor(reordered)); + + expect(markup).toContain('library audit [<sources>…] [--concurrency] --report'); +}); + +it('shows the canonical event beside an event route', () => { + const markup = render(routeCatalogFor(manifest)); + + expect(markup).toContain('>afterTool<'); + expect(markup).toContain('targets: 2 entries'); +}); + +it('surfaces route graph diagnostics alongside the catalog', () => { + const markup = render(routeCatalogFor(manifest)); + + expect(markup).toContain('Route diagnostics (1)'); + expect(markup).toContain('AB4801'); + expect(markup).toContain('Two MCP tool routes claim the same name.'); +}); + +it('reports a manifest ahead of the published build without hiding routes', () => { + const markup = render(routeCatalogFor(manifest, 'e'.repeat(64))); + + expect(markup).toContain('Rebuild to publish these routes.'); + expect(markup).toContain('route-state--stale'); + expect(markup).toContain('tool:library/echo'); +}); + +it('names the empty compiled graph rather than an error', () => { + const markup = render(routeCatalogFor({ + diagnostics: [], + digest: 'e'.repeat(64), + events: [], + providers: [], + scripts: [], + servers: [], + sourceRevision: 'r'.repeat(64), + })); + + expect(markup).toContain('This project declares no conventional route modules.'); + expect(markup).not.toContain('role="alert"'); +}); + +it('reports an unreadable manifest as an alert', () => { + const markup = render(unavailableRouteCatalog('Route manifest is not available.')); + + expect(markup).toContain('role="alert"'); + expect(markup).toContain('Route manifest is not available.'); + expect(markup).not.toContain('route-table'); +}); + +it('reports no static config export instead of an empty cell', () => { + const markup = render(routeCatalogFor(manifest)); + + expect(markup).toContain('No static config export'); +}); + +it('does not repeat the description it already renders as the route label', () => { + const described: RouteManifest = { + ...manifest, + servers: [{ + ...manifest.servers[0]!, + routes: [{ + config: [ + { key: 'description', kind: 'string', value: 'Echo the request back' }, + { key: 'title', kind: 'string', value: 'Echo' }, + ], + description: 'Echo the request back', + id: 'tool:library/echo', + kind: 'tool', + provenance: { kind: 'conventional' }, + serverId: 'mcp:library', + source: 'src/mcp/library/tools/echo.ts', + }], + }], + }; + + const markup = render(routeCatalogFor(described)); + + expect(markup).toContain('title: Echo'); + expect(markup).not.toContain('description: Echo the request back'); + expect(markup.match(/Echo the request back/gu)).toHaveLength(1); +}); + +it('names a config carrying only the description rather than claiming there is none', () => { + const onlyDescription: RouteManifest = { + ...manifest, + servers: [{ + ...manifest.servers[0]!, + routes: [{ + config: [{ key: 'description', kind: 'string', value: 'Curate a library' }], + description: 'Curate a library', + id: 'prompt:library/curate', + kind: 'prompt', + provenance: { kind: 'conventional' }, + serverId: 'mcp:library', + source: 'src/mcp/library/prompts/curate.ts', + }], + }], + }; + + const markup = render(routeCatalogFor(onlyDescription)); + + expect(markup).toContain('No config beyond the description'); + // The CLI route in the same manifest exports no config at all, so the two + // empty-config summaries must stay distinguishable rather than collapse. + expect(markup).toContain('No static config export'); +}); diff --git a/packages/workbench/tests/support/example-acceptance.ts b/packages/workbench/tests/support/example-acceptance.ts index ea9740af8..b4a5e8c0d 100644 --- a/packages/workbench/tests/support/example-acceptance.ts +++ b/packages/workbench/tests/support/example-acceptance.ts @@ -8,7 +8,7 @@ import type { Page, Request } from 'playwright-core'; import { workspaceRoot } from './workbench-e2e.ts'; import { timeScale } from '../../../agent-bundle/tests/support/time-scale.ts'; -export type ExampleName = 'hooks-and-scripts' | 'mcp-app' | 'skills-starter'; +export type ExampleName = 'audiobook-curator' | 'hooks-and-scripts' | 'mcp-app' | 'skills-starter'; export interface ExampleCapture { readonly example: ExampleName; diff --git a/packages/workbench/tests/workbench-capabilities.test.ts b/packages/workbench/tests/workbench-capabilities.test.ts index 9af36c122..f0f037738 100644 --- a/packages/workbench/tests/workbench-capabilities.test.ts +++ b/packages/workbench/tests/workbench-capabilities.test.ts @@ -1,6 +1,7 @@ import { expect, it } from '@rstest/core'; import type { ArtifactInspection } from '../../agent-bundle/src/contracts/artifacts.ts'; +import type { RouteManifest } from '../../agent-bundle/src/contracts/routes.ts'; import { loadWorkbenchCapabilities } from '../src/workbench-capabilities.ts'; const digest = '0'.repeat(64); @@ -65,15 +66,63 @@ const skill = { targets: ['portable'], }; +const route = (id: string, kind: RouteManifest['events'][number]['kind'], relativePath: string) => ({ + config: [], + id, + kind, + provenance: { kind: 'conventional' as const }, + source: relativePath, +}); + +const manifest = ({ + cliRoutes = 0, + events = 0, + routeScripts = 0, + servers = 0, + sourceRevision = digest, +} = {}): RouteManifest => ({ + ...(cliRoutes === 0 ? {} : { + cli: { + mode: 'generated' as const, + routes: Array.from({ length: cliRoutes }, (_, index) => + route(`cli:command-${String(index)}`, 'cli', `src/cli/command-${String(index)}.ts`)), + }, + }), + diagnostics: [], + digest, + events: Array.from({ length: events }, (_, index) => ({ + ...route(`event:after-tool-${String(index)}`, 'event-route', `src/events/tool/after-${String(index)}.ts`), + event: 'afterTool', + })), + providers: [], + scripts: Array.from({ length: routeScripts }, (_, index) => + route(`script:task-${String(index)}`, 'script', `src/scripts/task-${String(index)}.ts`)), + servers: Array.from({ length: servers }, (_, index) => ({ + id: `mcp:server-${String(index)}`, + mode: 'generated' as const, + name: `server-${String(index)}`, + routes: [route(`tool:server-${String(index)}/echo`, 'tool', `src/mcp/server-${String(index)}/tools/echo.ts`)], + })), + sourceRevision, +}); + const clientsFor = ({ + cliRoutes = 0, evalSuites = 0, + events = 0, hooks = 0, mcpServers = 0, + routeManifest = undefined as RouteManifest | undefined, + routeScripts = 0, + routeServers = 0, scripts = 0, skills = 0, targets = 1, } = {}) => ({ artifactClient: { inspect: async () => inspection({ hooks, mcpServers, scripts, targets }) }, + routeManifestClient: { + manifest: async () => routeManifest ?? manifest({ cliRoutes, events, routeScripts, servers: routeServers }), + }, evalClient: { suites: async () => ({ diagnostics: [], @@ -97,9 +146,11 @@ it('derives the Skills Starter routes from its validated catalogs', async () => }); expect([...capabilities.pages]).toEqual([ - 'overview', 'skills', 'artifacts', 'logs', 'evals', 'comparisons', + 'overview', 'routes', 'skills', 'artifacts', 'logs', 'evals', 'comparisons', ]); expect(capabilities.counts).toEqual({ evalSuites: 1, hooks: 0, mcpServers: 0, scripts: 0, skills: 1, targets: 3 }); + expect(capabilities.routes.state).toBe('current'); + expect(capabilities.routes.routeCount).toBe(0); expect(Object.isFrozen(capabilities)).toBe(true); expect(Object.isFrozen(capabilities.counts)).toBe(true); }); @@ -111,7 +162,7 @@ it('derives Hooks and Playground without advertising unrelated capabilities', as }); expect([...capabilities.pages]).toEqual([ - 'overview', 'hooks', 'artifacts', 'playground', 'logs', + 'overview', 'routes', 'hooks', 'artifacts', 'playground', 'logs', ]); }); @@ -122,7 +173,7 @@ it('derives the complete route set for a full bundle', async () => { }); expect([...capabilities.pages]).toEqual([ - 'overview', 'skills', 'hooks', 'mcp', 'artifacts', 'playground', 'logs', 'evals', 'comparisons', + 'overview', 'routes', 'skills', 'hooks', 'mcp', 'artifacts', 'playground', 'logs', 'evals', 'comparisons', ]); expect(capabilities.inspection.epochId).toBe('build-a'); }); @@ -133,3 +184,49 @@ it('rejects an inspection from a different build', async () => { ...clientsFor({ skills: 1 }), })).rejects.toThrow('Capability catalog did not match the current build.'); }); + +it('opens Hooks, MCP, and Playground from the compiled route graph alone', async () => { + const capabilities = await loadWorkbenchCapabilities({ + buildId: 'build-a', + ...clientsFor({ cliRoutes: 1, events: 1, routeScripts: 2, routeServers: 2 }), + }); + + expect([...capabilities.pages]).toEqual([ + 'overview', 'routes', 'hooks', 'mcp', 'artifacts', 'playground', 'logs', + ]); + expect(capabilities.counts.hooks).toBe(0); + expect(capabilities.counts.mcpServers).toBe(0); + expect(capabilities.routes.routeCount).toBe(6); + expect(capabilities.routes.groups.map((group) => group.label)).toEqual([ + 'server-0 · Tools', + 'server-1 · Tools', + 'Event routes', + 'CLI commands', + 'Scripts', + ]); +}); + +it('reports a manifest compiled from newer source than the published build as stale', async () => { + const capabilities = await loadWorkbenchCapabilities({ + buildId: 'build-a', + epochSourceRevision: '1'.repeat(64), + ...clientsFor({ events: 1 }), + }); + + expect(capabilities.routes.state).toBe('stale'); + expect(capabilities.pages.has('hooks')).toBe(true); +}); + +it('keeps every artifact-derived page when the manifest route is unavailable', async () => { + const capabilities = await loadWorkbenchCapabilities({ + buildId: 'build-a', + ...clientsFor({ evalSuites: 1, hooks: 1, mcpServers: 1, scripts: 1, skills: 1, targets: 3 }), + routeManifestClient: { manifest: async () => { throw new Error('Route manifest is not available.'); } }, + }); + + expect([...capabilities.pages]).toEqual([ + 'overview', 'routes', 'skills', 'hooks', 'mcp', 'artifacts', 'playground', 'logs', 'evals', 'comparisons', + ]); + expect(capabilities.routes.state).toBe('unavailable'); + expect(capabilities.routes.message).toBe('Route manifest is not available.'); +}); diff --git a/packages/workbench/tests/workbench-screen.test.ts b/packages/workbench/tests/workbench-screen.test.ts index f9aaceb4e..729de95e3 100644 --- a/packages/workbench/tests/workbench-screen.test.ts +++ b/packages/workbench/tests/workbench-screen.test.ts @@ -11,9 +11,10 @@ it('renders only available routes in grouped navigation', () => { const markup = renderToStaticMarkup(createElement(Navigation, { onNavigate: () => undefined, page: 'skills', - pages: pages('overview', 'skills', 'artifacts', 'logs', 'evals', 'comparisons'), + pages: pages('overview', 'routes', 'skills', 'artifacts', 'logs', 'evals', 'comparisons'), })); + expect(markup).toContain('>Routes<'); expect(markup).toContain('>Build<'); expect(markup).toContain('>Capabilities<'); expect(markup).toContain('>Quality<'); @@ -26,9 +27,11 @@ it('renders only available routes in grouped navigation', () => { }); it('resolves unsupported and unknown hashes to Overview', () => { - const available = pages('overview', 'skills', 'artifacts', 'logs'); + const available = pages('overview', 'routes', 'skills', 'artifacts', 'logs'); expect(pageForHash('#skills', available)).toBe('skills'); + expect(pageForHash('#routes', available)).toBe('routes'); + expect(pageForHash('#routes', pages('overview', 'skills'))).toBe('overview'); expect(pageForHash('#hooks', available)).toBe('overview'); expect(pageForHash('#unknown', available)).toBe('overview'); expect(pageForHash('', available)).toBe('overview');