Scaffold: npm test tests the plugin — every shipped pool, each as its own labeled run (#749)

.changeset/749-scaffold-default-test.md

@@ -0,0 +1,5 @@
+---
+"create-agent-bundle": patch
+---
+
+Make the scaffolded `npm test` test the plugin. In the `mcp-server` template the default `test` script now runs the plain module tests (`test:unit`, new), the route-unit pool (`test:routes`), and the in-memory MCP projection pool (`test:projection`) in turn; in `cli-tool` it runs `test:unit` and the `cli-dispatch`/`script-dispatch` projection pool (`test:projection`). Each pool is its own labeled run, so a route that stops rendering or a command that stops dispatching fails the ordinary test command while domain tests stay green. `check` in both templates is now `build && typecheck && npm test`. The focused scripts remain for a tight loop and take Rstest flags after `--`; packed and native proof stays opt-in. The `minimal` template still ships no route pool; its README shows the same aggregate wiring to add with the first route (#763)

packages/create-agent-bundle/README.md

@@ -52,8 +52,11 @@ harness. `mcp-server` ships a route-unit pool (`agentBundleRstest()` from
 `agent-bundle/rstest`, `renderRoute` and `expectDocument` from
 `agent-bundle/test`) and a separate in-memory MCP projection pool; `cli-tool`
 ships one projection pool at the `cli-dispatch` (`invokeCli`, `cliJson`) and
-`script-dispatch` (`runScript`) levels. Each pool is labeled with the proof
-level it carries and run by `check`. The `minimal` template compiles no route
+`script-dispatch` (`runScript`) levels. In both, `npm test` runs every pool
+the template ships, each as its own labeled run, so a broken route or
+projection fails the ordinary test command; the focused scripts remain for a
+tight loop — `test:unit`, `test:routes`, and `test:projection` in
+`mcp-server`, `test:unit` and `test:projection` in `cli-tool`. The `minimal` template compiles no route
 modules, so it ships no harness pool that would pass without addressing
 anything; its README documents the wiring to add with the first route.
 

packages/create-agent-bundle/templates/cli-tool/README.md

@@ -15,10 +15,11 @@ schemas. `src/index.ts` is the library export with declarations, and
 ```sh
 npm run dev              # local workbench with live rebuilds
 npm run build            # dist/ package build + host artifacts in artifact/
-npm run check            # build + typecheck + both test pools
+npm test                 # every test pool: unit, projection
+npm run check            # build + typecheck + npm test
 npm run typecheck        # validate (writes .agent-bundle/routes.d.ts) + tsc
-npm run test             # plain module tests
-npm run test:projection  # cli-dispatch + script-dispatch pool
+npm run test:unit        # plain module tests only
+npm run test:projection  # cli-dispatch + script-dispatch pool only
 
 # after a build
 node dist/bin/my-agent-plugin.mjs greet World
@@ -52,13 +53,16 @@ Validate and publish the generated npm root with
 
 ## Tests
 
-Two pools ship, and each one names the proof level it carries. A pass at one
-level is never a receipt for another, so they run — and are reported —
-separately. `npm run check` runs both.
+`npm test` tests the plugin: it runs both pools below in turn, so a command
+that stops dispatching fails the ordinary test command even while the plain
+module tests stay green. Each pool names the proof level it carries and is
+reported as its own run — a pass at one level is never a receipt for another.
+The focused scripts run one pool for a tight loop, and take Rstest's own flags
+after `--` (`npm run test:projection -- --watch`).
 
 | pool | command | files | what a pass proves |
 | --- | --- | --- | --- |
-| plain | `npm run test` | `tests/*.test.ts` | ordinary module tests over `src/index.ts`; no framework involved |
+| unit | `npm run test:unit` | `tests/*.test.ts` | ordinary module tests over `src/index.ts`; no framework involved |
 | projection (`cli-dispatch`) | `npm run test:projection` | `tests/projection/cli-dispatch.test.ts` | argv resolved and executed through the routed CLI's own shell over the compiled command graph — help, grammar, validation, exit codes — in-process; not the spawned executable |
 | projection (`script-dispatch`) | `npm run test:projection` | `tests/projection/script-dispatch.test.ts` | `src/scripts/hello.ts` run through its generated executable's `main` envelope contract with captured stdout/stderr and exit code — as a Node process of its own over the source, not the bundled `scripts/hello.mjs` |
 

packages/create-agent-bundle/templates/cli-tool/package_json

@@ -14,11 +14,12 @@
   },
   "scripts": {
     "build": "agent-bundle build --json --output artifact",
-    "check": "npm run build && npm run typecheck && npm run test && npm run test:projection",
+    "check": "npm run build && npm run typecheck && npm test",
     "dev": "agent-bundle dev",
     "pack:check": "agent-bundle prepack --json --output artifact",
-    "test": "rstest tests --exclude \"tests/projection/**\"",
+    "test": "npm run test:unit && npm run test:projection",
     "test:projection": "rstest --config rstest.projection.config.ts",
+    "test:unit": "rstest tests --exclude \"tests/projection/**\"",
     "typecheck": "npm run validate && tsc -p tsconfig.json --noEmit",
     "validate": "agent-bundle validate --json"
   },

packages/create-agent-bundle/templates/mcp-server/README.md

@@ -10,11 +10,12 @@ the `status` server; no handwritten server factory or server config is needed.
 ```sh
 npm run dev
 npm run build
-npm run check            # build + typecheck + all three test pools
+npm test                 # every test pool: unit, route-unit, projection
+npm run check            # build + typecheck + npm test
 npm run typecheck        # validate (writes .agent-bundle/routes.d.ts) + tsc
-npm run test             # plain module tests
-npm run test:routes      # route-unit pool
-npm run test:projection  # in-memory MCP projection pool
+npm run test:unit        # plain module tests only
+npm run test:routes      # route-unit pool only
+npm run test:projection  # in-memory MCP projection pool only
 npx --no-install agent-bundle mcp list --server status --target portable --artifact artifact
 
 # after publishing/removing "private" and installing the package
@@ -38,13 +39,16 @@ Validate and publish the generated npm root with
 
 ## Tests
 
-Three pools ship, and each one names the proof level it carries. A pass at one
-level is never a receipt for another, so they run — and are reported —
-separately. `npm run check` runs all three.
+`npm test` tests the plugin: it runs the three pools below in turn, so a route
+that stops rendering or a server that stops registering it fails the ordinary
+test command even while the plain module tests stay green. Each pool names the
+proof level it carries and is reported as its own run — a pass at one level is
+never a receipt for another. The focused scripts run one pool for a tight loop,
+and take Rstest's own flags after `--` (`npm run test:routes -- --watch`).
 
 | pool | command | files | what a pass proves |
 | --- | --- | --- | --- |
-| plain | `npm run test` | `tests/*.test.ts` | ordinary module tests over `src/status.ts`; no framework involved |
+| unit | `npm run test:unit` | `tests/*.test.ts` | ordinary module tests over `src/status.ts`; no framework involved |
 | route-unit | `npm run test:routes` | `tests/route-unit/**` | the route module renders to the Agent Document it claims, through the real renderer — no artifact, no transport |
 | projection | `npm run test:projection` | `tests/projection/**` | the real generated MCP server registers the route and projects its document to protocol content, over the SDK's in-memory transport — not a process, not the packed artifact |
 

packages/create-agent-bundle/templates/mcp-server/package_json

@@ -15,12 +15,13 @@
   },
   "scripts": {
     "build": "agent-bundle build --json --output artifact",
-    "check": "npm run build && npm run typecheck && npm run test && npm run test:routes && npm run test:projection",
+    "check": "npm run build && npm run typecheck && npm test",
     "dev": "agent-bundle dev",
     "pack:check": "agent-bundle prepack --json --output artifact",
-    "test": "rstest tests --exclude \"tests/{route-unit,projection}/**\"",
+    "test": "npm run test:unit && npm run test:routes && npm run test:projection",
     "test:projection": "rstest --config rstest.projection.config.ts",
     "test:routes": "rstest --config rstest.route-unit.config.ts",
+    "test:unit": "rstest tests --exclude \"tests/{route-unit,projection}/**\"",
     "typecheck": "npm run validate && tsc -p tsconfig.json --noEmit",
     "validate": "agent-bundle validate --json"
   },

packages/create-agent-bundle/templates/minimal/README.md

@@ -18,11 +18,11 @@ npm run check      # validate + build + typecheck + test
 - `src/skills/getting-started/` — a Skill: `SKILL.md` frontmatter plus optional
   `references/` and `assets/`. Every `src/skills/<name>/SKILL.md` directory is
   discovered automatically; add a folder and it ships.
-- `tests/` — run with `npm run test`.
+- `tests/` — run with `npm test`.
 
 ## Tests
 
-`npm run test` runs ordinary module tests — here, one test that keeps the
+`npm test` runs ordinary module tests — here, one test that keeps the
 Skill's frontmatter aligned with its directory — and `npm run check` runs them
 after validate, build, and typecheck.
 
@@ -43,10 +43,15 @@ export default defineConfig(await agentBundleRstest());
 ```
 
 ```json
-"test": "rstest tests --exclude \"tests/route-unit/**\"",
+"test": "npm run test:unit && npm run test:routes",
+"test:unit": "rstest tests --exclude \"tests/route-unit/**\"",
 "test:routes": "rstest --config rstest.route-unit.config.ts"
 ```
 
+`npm test` then runs both pools, each reported as its own run, so the route
+pool is part of the ordinary test command rather than a separate step to
+remember.
+
 Then `renderRoute` and `expectDocument` from `agent-bundle/test` assert the
 document the route renders, at the `route-unit` proof level. Rendering also
 needs `react` and `@agent-bundle/runtime`, which a project with route modules

packages/create-agent-bundle/tests/scaffold-packed-matrix.e2e.test.ts

@@ -1,5 +1,5 @@
 import { execFile as executeFile } from 'node:child_process';
-import { mkdtemp, readFile, rm, stat } from 'node:fs/promises';
+import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { pathToFileURL } from 'node:url';
@@ -19,6 +19,9 @@ import {
 
 const execFile = promisify(executeFile);
 
+/** npm's banner for one script of the scaffolded project, the label each pool runs under. */
+const poolBanner = (script: string): string => `status-plugin@0.1.0 ${script}\n`;
+
 afterAll(cleanupScaffoldFixture);
 
 /**
@@ -74,6 +77,30 @@ it.concurrent('scaffolds the mcp-server template and serves the conventional ent
       structuredContent: { service: 'docs', status: 'healthy' },
     },
   });
+
+  // The ordinary test command tests the plugin (#749): `npm test` runs every
+  // pool as its own labeled run, and a route that stops rendering what it
+  // claims fails it while the plain module tests stay green.
+  const tested = await npmRun(projectRoot, 'test');
+  for (const pool of ['test:unit', 'test:routes', 'test:projection']) expect(tested).toContain(poolBanner(pool));
+  const route = join(projectRoot, 'src', 'mcp', 'status', 'tools', 'report-status.tsx');
+  const source = await readFile(route, 'utf8');
+  expect(source).toContain('<Agent.Text>{report.summary}</Agent.Text>');
+  await writeFile(route, source.replace('<Agent.Text>{report.summary}</Agent.Text>', '<Agent.Text>unreachable</Agent.Text>'));
+  const broken = await npmRun(projectRoot, 'test').then(
+    () => { throw new Error('`npm test` passed with a route that renders the wrong text.'); },
+    (error: unknown) => error as { readonly code?: number; readonly stderr?: string; readonly stdout?: string },
+  );
+  // npm writes each script's banner to stderr, so the run is read as a whole:
+  // the unit pool passed, the route-unit pool failed on the changed render,
+  // and the projection pool never ran behind it.
+  const brokenOutput = `${broken.stdout ?? ''}${broken.stderr ?? ''}`;
+  expect(broken.code).toBe(1);
+  expect(brokenOutput).toContain(poolBanner('test:unit'));
+  expect(brokenOutput).toContain(poolBanner('test:routes'));
+  expect(brokenOutput).toContain('tests/route-unit/report-status.test.ts');
+  expect(brokenOutput).not.toContain(poolBanner('test:projection'));
+  await npmRun(projectRoot, 'test:unit');
 }, 600_000);
 
 it.concurrent('scaffolds the cli-tool template with a routed bin, lib, and artifact script', async () => {

website/docs/en/guide/authoring/reuse-framework.mdx

@@ -194,6 +194,11 @@ second native-cache manager, receipt database, or lifecycle parser. See
 | Packaging, paths or installation | Actual packed/source-free execution and receipt-owned install tests. |
 | Host behavior | A separately labelled native host test with real authorization; mocks are not an authenticated model turn. |
 
+In a scaffolded project the first three rows are the deterministic pools `npm test` runs together,
+each reported as its own run — `test:unit`, `test:routes`, and `test:projection` in the
+`mcp-server` template, `test:unit` and `test:projection` in `cli-tool`, unit tests alone in the
+route-less `minimal` template; the remaining rows are explicit, separately labelled gates.
+
 Keep independent expected identities and safety outcomes in tests. Deriving every expectation
 from the same output under test cannot detect a wrong rename or missing annotation. Conversely,
 do not copy the framework's whole transport implementation into a test fixture merely to test

website/docs/en/guide/development/troubleshooting.mdx

@@ -54,7 +54,7 @@ their numeric versions must be equal.
 | Installation rejects a manifest revision or changed digest | Align the emitting compiler and consuming lifecycle implementation. Keep public packages coherent; do not independently update a private source reader or edit the closed manifest version/digests. |
 | A read-only installed plugin fails on a write | Keep code/assets immutable and use the observed framework state location plus an explicit domain-data policy. An acquired session or cache is not a packaged asset or a reason to rewrite the installed `.env`. |
 | MCP stdio fails to parse output | Keep application logs off the JSON-RPC stdout channel. Use the generated lifecycle or a correctly scoped custom-server entry, and send diagnostics to stderr. |
-| A green default test did not catch a broken route | Check which tests the script collects. The current MCP starter's `check` includes route/projection pools, while its `npm test` alone excludes them. Use the full documented check until the script contract changes. |
+| A green default test did not catch a broken route | Check which tests the script collects. The MCP starter's `npm test` runs its unit, route-unit, and projection pools in turn; a project whose `test` script excludes a pool must add it back or run that pool's script explicitly. |
 | A native test was skipped or an account is unavailable | Record the missing prerequisite and the proof not performed. A successful mock, build, or install is not authenticated tool/hook execution; unverified is not the same as unsupported. |
 
 ## Generated types on a clean checkout
@@ -73,9 +73,11 @@ warning is not a complete audit of every project reference. For solution-style c
 verify each consuming program rather than adding a file to an unrelated root program merely to
 silence a warning. Do not import server implementations into browser runtime code to get types.
 
-The starter's current complete check is:
+The starter's ordinary test command runs every pool it ships; its complete check adds build and
+typecheck:
 
 ```sh
+npm test
 npm run check
 ```
 

website/docs/en/guide/start/quick-start.mdx

@@ -127,16 +127,18 @@ it('returns the greeting as structured data', async () => {
 });
 ```
 
-Run the existing aggregate command:
+Run the ordinary test command:
 
 ```sh
-npm run check
+npm test
 ```
 
-In the current MCP starter, `npm test` alone excludes route and projection tests. `check` runs
-validation, build, typecheck, and all three test groups. Use it for the complete starter gate;
-`npm run test:routes` is the focused route loop. More proof levels, including real MCP processes
-and browser Apps, are covered in [testing](../development/testing.mdx).
+In the MCP starter, `npm test` runs every test pool the project ships — the plain module tests,
+the route-unit pool, and the in-memory MCP projection pool — each as its own labeled run, so a
+route that stops rendering fails it even while the domain tests stay green. `npm run check` adds
+build and typecheck in front of it for the complete starter gate; `npm run test:routes` is the
+focused route loop. More proof levels, including real MCP processes and browser Apps, are covered
+in [testing](../development/testing.mdx).
 
 The generated `.agent-bundle/routes.d.ts` must be included in the TypeScript project consuming
 route/provider/App types. It is generated, not an authored registry. For a standalone typecheck

website/docs/zh/guide/authoring/reuse-framework.mdx

@@ -165,6 +165,10 @@ notice 使用框架 ledger 及[notice 矩阵](../../reference/notices.md)中的
 | 打包、路径或安装 | 真实打包后/删除源码运行，以及基于 receipt 所有权的安装测试。 |
 | 宿主行为 | 单独标注的、使用真实授权的原生宿主测试；mock 不是已认证的模型调用。 |
 
+在脚手架项目中，前三行就是 `npm test` 一起运行的确定性测试池，各自作为独立运行报告——`mcp-server`
+模板中是 `test:unit`、`test:routes` 与 `test:projection`，`cli-tool` 中是 `test:unit` 与
+`test:projection`，没有路由的 `minimal` 模板只有单元测试；其余各行是显式的、单独标注的门禁。
+
 测试应保留独立的预期身份和安全结果。所有预期都从被测输出本身推导，无法发现错误改名或缺失 annotation。
 反过来，也不应仅为测试插件领域行为，就把框架整个传输实现复制进测试 fixture。
 

website/docs/zh/guide/development/troubleshooting.mdx

@@ -52,7 +52,7 @@ npx --no-install agent-bundle validate --artifact artifact
 | 安装拒绝清单版本或变动的摘要 | 对齐发射编译器与消费它的生命周期实现，保持公开包配对一致。不要独立升级私有源码 reader，也不要编辑封闭的清单版本或 digest。 |
 | 只读安装插件在写入时失败 | 保持代码/assets 不可变，使用观测到的框架 state 位置和明确的领域数据策略。取得的 session 或 cache 不是打包 asset，也不是改写已安装 `.env` 的理由。 |
 | MCP stdio 无法解析输出 | 不要向 JSON-RPC stdout 通道写应用日志。使用生成生命周期或正确限定的自定义服务器入口，诊断写 stderr。 |
-| 默认测试通过，却没发现路由损坏 | 检查脚本收集哪些测试。当前 MCP starter 的 `check` 包含 route/projection pools，但单独 `npm test` 不包含。在脚本契约更新前使用完整的文档 check。 |
+| 默认测试通过，却没发现路由损坏 | 检查脚本收集哪些测试。MCP starter 的 `npm test` 会依次运行 unit、route-unit 与 projection 池；若项目的 `test` 脚本排除了某个池，需要把它加回来或显式运行该池的脚本。 |
 | 原生测试被跳过，或账户不可用 | 记录缺失前提和未完成证明。mock、构建或安装成功不是已认证的工具/hook 执行；unverified 与 unsupported 不同。 |
 
 ## 全新工作区中的生成类型
@@ -70,9 +70,10 @@ npm run typecheck
 对于 solution-style 配置，验证每个实际消费程序；不要只把文件加进无关根程序以消除警告。
 不要为获取类型而把服务器实现导入浏览器运行时代码。
 
-starter 当前完整检查命令是：
+starter 的普通测试命令会运行它附带的每个池；完整检查在其前面加上构建与类型检查：
 
 ```sh
+npm test
 npm run check
 ```