fix(test,workbench): closed-issue audit G1 follow-ups — harness provider mounting and deterministic examples-real edits

.changeset/test-harness-conventional-providers.md

@@ -0,0 +1,5 @@
+---
+"agent-bundle": patch
+---
+
+Mount conventional request context providers (`src/providers/*`) in the `agent-bundle/test` harness for every manifest-backed call — `renderRoute`, `renderRouteEvents`, `invokeCli` (plain, rendered, and projected MCP commands), `openInMemoryMcpServer`, and `invokeMcpTool` — exactly as the generated request scopes do: same deterministic key order, same surface-specific `invocation`, same fail-closed factory errors, and a `providers.processLifetime` scoped like the artifact's (fresh per `invokeCli` call and per `renderRoute` render, shared across one open in-memory MCP session). Pass `context.providers` to mount an explicit fixture map instead; `context` and its `providers` stay optional even once the generated `.agent-bundle/routes.d.ts` augmentation declares provider keys (`HarnessOptionsArguments`, `RenderRouteContextInit`), while an explicit map must carry every declared key and a direct `runAgentRequest` still requires `providers`. Provider modules are evaluated once per test worker, so module-level provider state is shared across simulated executables; prove cold state through the proof levels that spawn the artifact. Hand `renderRoute` providers and the request scope the executable surface the artifact records (a routed CLI command's space-joined path, a script's path-derived name) instead of the route id, and mount an event route's compiled id (`event:tool/after`) as `invocation.operationId` in the generated Flight worker, matching the hook shell, lifecycle replay, and harness. The test manifest gains `providers`, the generated Rstest setup registers provider loaders (test registry version 4), and a project whose setup predates that registration fails with the `manifest-unavailable` harness error naming the provider. (#399)

.gitignore

@@ -5,8 +5,8 @@ dist/
 coverage/
 artifacts/
 *.log
-examples/audiobook-curator/artifact/
-examples/worktree-proximity/artifact/
+# Every example's `pnpm build` (and `pnpm examples:check`) writes here.
+examples/*/artifact/
 
 # Build-time copies of the root LICENSE and NOTICE (scripts/sync-license-files.mjs)
 packages/*/LICENSE

docs/entry-conventions.md

@@ -174,14 +174,25 @@ of throwing. `invocation.kind` stays surface-specific (`tool`, `event`, `cli`,
 `processLifetime` is reserved for the framework-owned process identity and hit
 counter, so provider filenames must not derive that key.
 
-Route-unit and CLI-dispatch tests inject provider values through the same
-`context` seam as identity axes (`renderRoute(id, { context: { providers:
-{ library: fixture } } })`); the harness never executes conventional provider
-modules, so a test chooses exactly the values a component observes. Once the
-generated `.agent-bundle/routes.d.ts` augmentation declares provider keys, the
-harness `options` and its `context.providers` become required (as does
-`providers` on a direct `runAgentRequest`), so omitting a fixture the route's
-types promise is a compile error rather than a runtime `undefined`.
+The `agent-bundle/test` harness mounts the same providers, in the same order
+and with the same fail-closed semantics, for every manifest-backed helper
+(`renderRoute`, `renderRouteEvents`, `invokeCli`, and the in-memory MCP
+helpers), so a route test observes what the artifact would mount. A test that
+wants to choose the values instead injects them through the same `context`
+seam as identity axes (`renderRoute(id, { context: { providers: { library:
+fixture } } })`): an explicit map is mounted verbatim and no conventional
+provider module executes. A module rendered directly (no compiled manifest)
+has no project to discover, so it observes only `processLifetime`. Once the
+generated `.agent-bundle/routes.d.ts` augmentation declares provider keys, an
+explicit `context.providers` map must carry every declared key (as must
+`providers` on a direct `runAgentRequest`), so a fixture that omits a value the
+route's types promise is a compile error rather than a runtime `undefined`;
+omitting `context.providers` altogether stays legal and mounts the real
+providers. The harness reproduces the per-executable process identity, not
+per-executable module evaluation: provider modules are evaluated once per test
+worker, so module-level provider state is shared across the simulated
+executables of that worker and is only proven cold by the proof levels that
+spawn the artifact.
 
 ### Handler request context
 

docs/framework-mode.md

@@ -136,13 +136,19 @@ and augments `@agent-bundle/runtime`'s `AgentProviderValues`, so
 `(await agent()).providers.library` is a `LibraryContext` with no cast once
 the file is part of the project's TypeScript program (add
 `".agent-bundle/routes.d.ts"` to `tsconfig.json` `include`). Undeclared keys
-stay `unknown`. Route-unit and CLI-dispatch tests inject fixture values through
-`renderRoute(id, { context: { providers: { library } } })`; the harness never
-executes provider modules on a test's behalf. Because the augmentation makes
-declared keys required, the same program also requires `context.providers`
-(and the harness `options` argument) on every `renderRoute`, `invokeCli`, and
-in-memory MCP call, and `providers` on a direct `runAgentRequest`: a handler
-typed against `providers.library` can never observe an unchecked `undefined`.
+stay `unknown`. The `agent-bundle/test` harness (`renderRoute`, `invokeCli`,
+the in-memory MCP helpers) mounts the project's providers automatically, in the
+same order and with the same fail-closed semantics as the generated request
+scopes, so a route-unit test observes what the artifact would mount — including
+a provider that reaches the network or the file system. To stub one, inject
+fixture values through `renderRoute(id, { context: { providers: { library } } })`:
+an explicit map is mounted verbatim and no provider module executes. Because
+the augmentation makes declared keys required, an explicit `context.providers`
+must carry every declared key, and a direct `runAgentRequest` (where nothing
+else supplies providers) requires `providers` outright: a handler typed against
+`providers.library` can never observe an unchecked `undefined`. See the
+[harness section](../packages/agent-bundle/README.md#testing-routes) for the
+module-evaluation caveat that applies to provider-level state.
 
 ### What reaches the MCP wire
 

docs/local-ci.md

@@ -175,6 +175,18 @@ machine is exactly the contention that scale exists for. Exporting
 running other heavy work) overrides the default; the integration config
 never lets it drop below what its own pool shape requires.
 
+Load-sensitive failures are fixed at their cause, never absorbed with a
+per-test `retry`. The recurring shape is a test that acts before the product
+has published the state it is about to assert on; the fix is to wait on the
+product's own readiness signal. Precedents: the dev watcher's stat-signature
+dedupe (#122/#329), content-identity reload announcements (#200/#332), and
+the `examples-real.e2e` source edits, which used to pair a file write with an
+immediate manual rebuild and so raced the watcher's own rebuild of the same
+write for a second epoch. Those edits now go through
+`replaceWatchedSourceAndAwaitRebuild` (`packages/agent-bundle/tests/support/watched-files.ts`):
+one atomic replacement, then a wait on the coordinator's published build
+attempt, so one edit is exactly one build.
+
 ## What is deliberately not covered
 
 - **dependency-review** runs as a GitHub-side action against the GitHub

examples/audiobook-curator/tests/route-unit/context.test.ts

@@ -38,7 +38,10 @@ it('renders the catalog from injected library context with its contents envelope
 });
 
 it('renders an honest degraded catalog when library context is absent', async () => {
+  // The harness mounts `src/providers/library.ts` automatically, so the
+  // degraded path needs an explicit empty provider map to keep it absent.
   const rendered = await renderRoute('resource:curator/catalog', {
+    context: { providers: {} },
     input: { uri: 'audiobook-curator://catalog' },
   });
   const value = rendered.document.value as {

packages/agent-bundle/README.md

@@ -411,6 +411,48 @@ than paying for a build per route. Every failure — an unknown route, a refused
 route kind, a rejected input, a render error — names the route id, the target
 kind, and the module provenance.
 
+Conventional request context providers (`src/providers/*`, see
+[entry conventions](../../docs/entry-conventions.md#request-context-providers-power-tier))
+are mounted automatically for every manifest-backed helper — `renderRoute`,
+`renderRouteEvents`, `invokeCli`, and the in-memory MCP helpers — exactly as the
+generated request scopes mount them: discovered from the compiled manifest,
+executed once per request in the same deterministic key order, handed the same
+surface-specific `invocation` (`tool`, `event`, `cli`, `script`), and failing the
+request closed when a factory throws. `providers.processLifetime` is scoped the
+way the artifact scopes it: each `invokeCli` call and each `renderRoute` render
+is a fresh simulated executable (hit 1, new `instanceId`), while one open
+`openInMemoryMcpServer` session shares a single identity across every request
+it handles, like the artifact's warm Flight worker. Pass `context.providers` to opt out: an explicit map is mounted
+verbatim and no conventional provider runs, which is how a test stubs a provider
+that would otherwise reach the network or the file system.
+
+```ts
+// Real providers, as the artifact would mount them.
+const real = await renderRoute('tool:library/summarize', { input: { title: 'Dune' } });
+
+// Stubbed providers: nothing under src/providers/ executes.
+const stubbed = await invokeCli(['library', 'audit', './books'], {
+  context: { providers: { libraryTooling: { tool: 'ffprobe 6.1' } } },
+});
+```
+
+`context` (and `context.providers`) stays optional even once the generated
+`.agent-bundle/routes.d.ts` augmentation declares provider keys: omitting it
+runs the real providers, which is what the artifact does, while an explicit map
+must carry every declared key, so a fixture cannot leave a promised value
+`undefined`. Only a direct `runAgentRequest` requires `providers` in that case,
+because nothing else would supply them.
+
+The harness simulates the process identity per executable, not module
+evaluation: one Rstest worker evaluates each provider module once, so
+module-level state in a provider is shared across every simulated CLI
+invocation, render, and in-memory server in that worker (as it is for the route
+modules themselves), whereas a real artifact evaluates the module afresh in
+every CLI process and Flight worker. A provider's module-level cache, counter,
+or singleton is therefore only proven by the packed and projected proof levels
+that spawn the artifact; a route-unit test that needs cold state should stub
+the provider through `context.providers` or reset that state between calls.
+
 Matchers over the Agent Document contracts: `toHaveStatus`, `toContainMarkdown`,
 `toContainText`, `toHaveValue`, `toHaveError`, and `toHaveNodeKinds`.
 

packages/agent-bundle/fixtures/route-harness/src/cli/tooling/inspect.ts

@@ -0,0 +1,24 @@
+import { agent } from '@agent-bundle/runtime';
+import type { CliRouteConfig, CliRouteProps } from 'agent-bundle';
+import { z } from 'zod';
+
+export const config = {
+  description: 'Reports the request providers a plain command observes.',
+} satisfies CliRouteConfig;
+
+export const inputSchema = z.object({}).strict();
+
+export const resultSchema = z.object({
+  keys: z.array(z.string()),
+  libraryTooling: z.unknown().optional(),
+  processLifetime: z.object({ hits: z.number().int().min(1), instanceId: z.string(), pid: z.number().int() }).strict(),
+}).strict();
+
+export default async function inspect(_props: CliRouteProps<typeof inputSchema>) {
+  const { providers } = await agent();
+  return {
+    keys: Object.keys(providers).sort(),
+    libraryTooling: providers['libraryTooling'],
+    processLifetime: providers['processLifetime'],
+  };
+}

packages/agent-bundle/fixtures/route-harness/src/cli/tooling/report.tsx

@@ -0,0 +1,24 @@
+import { Agent, agent, type JsonValue } from '@agent-bundle/runtime';
+import type { CliRouteConfig, CliRouteProps } from 'agent-bundle';
+import { z } from 'zod';
+
+export const config = {
+  description: 'Renders the request providers a rendered command observes.',
+} satisfies CliRouteConfig;
+
+export const inputSchema = z.object({}).strict();
+
+export const resultSchema = z.object({
+  keys: z.array(z.string()),
+  libraryTooling: z.unknown().optional(),
+}).strict();
+
+export default async function ToolingReport(_props: CliRouteProps<typeof inputSchema>) {
+  const { providers } = await agent();
+  const value = { keys: Object.keys(providers).sort(), libraryTooling: providers['libraryTooling'] as JsonValue };
+  return (
+    <Agent.Result value={value}>
+      <Agent.Text>{`tooling: ${JSON.stringify(providers['libraryTooling'])}`}</Agent.Text>
+    </Agent.Result>
+  );
+}

packages/agent-bundle/fixtures/route-harness/src/mcp/harness/tools/tooling.tsx

@@ -0,0 +1,34 @@
+import { Agent, agent, type JsonValue } from '@agent-bundle/runtime';
+import { z } from 'zod';
+
+export const config = {
+  annotations: { readOnlyHint: true },
+  description: 'Reports the request providers an MCP tool observes.',
+  title: 'Tooling',
+};
+
+export const inputSchema = z.object({
+  /** Makes the `library-tooling` provider throw, to prove the request fails closed. */
+  failProvider: z.boolean().optional(),
+}).strict();
+
+export const resultSchema = z.object({
+  keys: z.array(z.string()),
+  libraryTooling: z.unknown().optional(),
+  processLifetime: z.object({ hits: z.number(), instanceId: z.string(), pid: z.number() }).optional(),
+}).strict();
+
+export default async function Tooling() {
+  const { providers } = await agent();
+  const { processLifetime } = providers;
+  const value = {
+    keys: Object.keys(providers).sort(),
+    libraryTooling: providers['libraryTooling'] as JsonValue,
+    ...(processLifetime === undefined ? {} : { processLifetime: { ...processLifetime } }),
+  };
+  return (
+    <Agent.Result value={value}>
+      <Agent.Text>{`tooling: ${JSON.stringify(providers['libraryTooling'])}`}</Agent.Text>
+    </Agent.Result>
+  );
+}

packages/agent-bundle/fixtures/route-harness/src/providers/library-tooling.ts

@@ -0,0 +1,25 @@
+import type { AgentProviderContext } from 'agent-bundle';
+
+/**
+ * A conventional request context provider. The harness mounts it for every
+ * manifest request scope exactly as the generated entries do, so routes on
+ * every surface observe `providers.libraryTooling` with the surface-specific
+ * invocation kind the provider saw.
+ */
+export default async function libraryTooling({ invocation, signal }: AgentProviderContext) {
+  if (signal.aborted) throw new DOMException('aborted', 'AbortError');
+  const input = invocation.kind === 'tool' ? invocation.props.input : undefined;
+  if (typeof input === 'object' && input !== null && (input as { readonly failProvider?: unknown }).failProvider === true) {
+    throw new Error('ffprobe is not installed');
+  }
+  const surface = invocation.kind === 'tool'
+    ? invocation.props.operationId
+    : invocation.kind === 'cli'
+      ? invocation.props.command
+      : invocation.kind === 'script'
+        ? invocation.props.name
+        : invocation.kind === 'event'
+          ? invocation.props.event
+          : invocation.props.view;
+  return { kind: invocation.kind, surface, tool: 'ffprobe 6.1' };
+}

packages/agent-bundle/fixtures/route-harness/src/scripts/tooling-summary.tsx

@@ -0,0 +1,26 @@
+import { Agent, agent, type JsonValue } from '@agent-bundle/runtime';
+import { z } from 'zod';
+
+export const resultSchema = z.object({
+  arguments: z.number().int().nonnegative(),
+  keys: z.array(z.string()),
+  libraryTooling: z.unknown().optional(),
+}).strict();
+
+export default async function ToolingSummary({ argv, signal }: {
+  readonly argv: readonly string[];
+  readonly signal: AbortSignal;
+}) {
+  if (signal.aborted) throw new DOMException('aborted', 'AbortError');
+  const { providers } = await agent();
+  const value = {
+    arguments: argv.length,
+    keys: Object.keys(providers).sort(),
+    libraryTooling: providers['libraryTooling'] as JsonValue,
+  };
+  return (
+    <Agent.Result value={value}>
+      <Agent.Text>{`Summarized ${String(argv.length)} arguments.`}</Agent.Text>
+    </Agent.Result>
+  );
+}