Skip to content

feat(test): route-unit consumer test harness (#103 stage 1) - #151

Merged
ScriptedAlchemy merged 1 commit into
mainfrom
wave3/103-test-harness
Sep 1, 2026
Merged

feat(test): route-unit consumer test harness (#103 stage 1)#151
ScriptedAlchemy merged 1 commit into
mainfrom
wave3/103-test-harness

Conversation

@ScriptedAlchemy

Copy link
Copy Markdown
Owner

Closes the stage-1 slice of #103: route modules can now be tested without building an artifact.

Shipped

agent-bundle/rstestagentBundleRstest(options?) returns a plain Rstest configuration object (thin defineConfig input; no plugin surface yet):

import { defineConfig } from '@rstest/core';
import { agentBundleRstest } from 'agent-bundle/rstest';

export default defineConfig(await agentBundleRstest());

It runs one compiler pass — the same ProjectService.prepare() the build and inspect use, which owns route-graph compilation — and projects that graph into a test manifest. No second compile, no artifact build. The manifest and per-route loaders are written to a generated setup module under the project's gitignored .agent-bundle/test/, so the bundler transforms the route .tsx modules the same way the build does.

agent-bundle/testrenderRoute(routeId | module, { input, args, context, limits, progress, signal }) executes a route through the runtime's public final-only Flight dispatcher (#144) and request store (#113) and resolves to the final Agent Document (#140), the ordered render events, the resolved provenance, and the resultSchema-parsed value. expectDocument(...) matchers: toHaveStatus, toContainMarkdown, toContainText, toHaveValue, toHaveError, toHaveNodeKinds. testManifest() exposes the compiled route inventory so per-route assertions iterate in process. Every failure names the route id, target kind, module, project root, and route-graph digest.

Proof level discipline. One level ships, labeled route-unit in the manifest, in every provenance record, and in the failure text. No placeholders for the transport, packed, or browser levels.

Packaging. @agent-bundle/runtime, @rstest/core, and react are optional peers. The existing packed boundary test now asserts a skills-only consumer installs none of them (and that no directory for them exists on disk); its tree walk previously counted unmet optional peers as installed, which is fixed here. Declaring the peers is also what keeps them external in the published bundle — before this, dist/ vendored a 410 KB copy of the Flight server whose bundler-injected RSC manifest was not the consumer project's, which is exactly how the audiobook adoption test failed until the peer was declared.

Pools. The level needs Node's react-server condition, a per-process flag, so it runs as its own pool (pnpm test:route-unit, built from the shipped helper) and the workspace pools exclude tests/route-unit/**. No new build or pack is added to any pool; the adoption example reuses the single examples:check build.

Adoption proof

  • examples/audiobook-curator/tests/route-unit/routes.test.ts renders the migrated prompt:curator/curate route (feat(routes): migrate audiobook curator to public authoring #150) with renderRoute and asserts status, text, and structured value, plus the manifest compile with no build. 2 tests.
  • packages/agent-bundle/fixtures/route-harness/ is the framework's own fixture project (tool, event, resource, app routes) driving 10 route-unit tests: context overrides, progress capture, refused kinds, unknown routes, render errors, and result-schema rejection.

Stage-1 follow-ups (deliberately not here)

  • In-memory MCP transport, packed-artifact, and browser proof levels (Generate an Agent Bundle framework test harness on top of Rstest #103 later stages).
  • Event-stream matchers (assertions today read the final document; the events are returned but unmatched).
  • A real Rstest plugin surface, alias/environment plumbing beyond the react-server condition and JSX runtime, and any docs beyond the README section.
  • Broader audiobook coverage: only the prompt route is asserted; the 15 tool routes need fixture roots or fakes for their filesystem and ffprobe operations.

Scoped results

  • pnpm test:unit — 141 files, 1972 tests, 0 failures
  • pnpm test:route-unit — 10 passed
  • examples/audiobook-curator: pnpm test:routes — 2 passed; pnpm test 35 passed; pnpm typecheck clean
  • rsc-runtime-optional-packaging (packed) — 1 passed
  • public-api (integration, loads every public subpath) — 7 passed
  • pnpm lint, pnpm typecheck, pnpm lint:package (publint x3) — clean

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c34feb3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
agent-bundle Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T08:49:46.431001Z 7b07c9d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Route modules had no way to be tested except through a full artifact build.
This adds the route-unit proof level as two public subpaths.

`agent-bundle/rstest` exposes `agentBundleRstest()`: it runs the project
preparation the build and `inspect` already use — one route-graph compile, no
artifact — and returns a plain Rstest configuration carrying the compiled test
manifest, the route loaders, React's `react-server` resolution on both the
process and the bundler, and the automatic JSX runtime.

`agent-bundle/test` exposes `renderRoute`, which executes a route by compiled id
or by module through the runtime's real final-only Flight dispatcher and request
store and resolves to the final Agent Document, plus `expectDocument` matchers
over the Agent Document contracts and `testManifest()` for iterating the route
inventory in process. Failures name the route, target kind, module provenance,
and manifest digest.

`@agent-bundle/runtime`, `@rstest/core`, and `react` are optional peers: a
project that does not test routes installs none of them, and declaring them is
also what keeps them external in the published bundle instead of vendoring a
second React server copy whose RSC manifest is not the consumer's.
@ScriptedAlchemy
ScriptedAlchemy merged commit ce55a67 into main Sep 1, 2026
2 of 9 checks passed
@ScriptedAlchemy
ScriptedAlchemy deleted the wave3/103-test-harness branch September 1, 2026 08:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b07c9d237

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/agent-bundle/src/test/render.ts
Comment thread packages/agent-bundle/src/test/matchers.ts
Comment thread packages/agent-bundle/src/rstest/setup-module.ts
Comment thread packages/agent-bundle/src/test/render.ts
Comment thread packages/agent-bundle/README.md
ScriptedAlchemy added a commit that referenced this pull request Sep 1, 2026
…API (wave 3.5 stage 1) (#154)

- sqlite driver: connection owned by a Layer + per-store scoped runtime;
  BEGIN IMMEDIATE / COMMIT / ROLLBACK as Effect.acquireUseRelease with
  rollback-on-failed-commit; SqliteConnection as a Context.Service
- typed AgentStateError error channel end to end; sqlite/system throws map to
  the existing corrupt/unavailable codes, unexpected throws stay defects and
  still roll the transaction back
- memory driver: same stateEffect channel; store close is the scope finalizer
- driver close now settles pending opens with store-closed before resolving
  (new race tests pin it for both drivers)
- packaging test additionally proves root/plugin entries ship no effect bytes
- host-artifacts example test: allow the optional react peer that #151 added
  (pre-existing failure on main; hard deps stay banned)
- agent-patterns/effect-scope.md: document the transaction idiom the rewrite
  settled on (dogfood feedback)
ScriptedAlchemy added a commit that referenced this pull request Sep 1, 2026
Bind route loaders to the manifest that produced them, so an explicit
manifest cannot silently execute the registered project's module for a
colliding route id and report the other manifest's provenance.

Separate an absent document value from an emitted null in toHaveValue,
record request-scoped progress even when the caller supplies its own
reporter, validate the generated registry version where the helpers read
it, and correct the README's request-context example.
ScriptedAlchemy added a commit that referenced this pull request Sep 1, 2026
…ots, workspace-free published manifest (#183)

Three independent breakages kept hosted CI red on main:

- scripts/run-examples-check.mjs launched npm_execpath through the current
  Node executable, but pnpm/setup on hosted runners exposes a native shim
  (bare command name), so `node pnpm` failed with MODULE_NOT_FOUND. The
  script now execs non-JavaScript entrypoints directly.
- The sqlite migration (#149) made the example kernel return the durable
  prefix at the original commit revision on idempotent replays; the retired
  JSONL kernel returned the current head. Re-running a fixture after a reset
  therefore reported the pre-reset state version and failed the runtime
  playground e2e on every Node line. Replays now read the head again.
- #151 added `"@agent-bundle/runtime": "workspace:*"` to the published
  agent-bundle devDependencies, tripping the release audit (npm refuses
  tarball manifests with workspace ranges). The optional peer is now
  satisfied through a pnpm-workspace override instead of a shipped range.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant