Skip to content

Scaffold: npm test tests the plugin — every shipped pool, each as its own labeled run (#749) - #763

Merged
ScriptedAlchemy merged 3 commits into
mainfrom
feat/749-scaffold-default-test
Sep 8, 2026
Merged

Scaffold: npm test tests the plugin — every shipped pool, each as its own labeled run (#749)#763
ScriptedAlchemy merged 3 commits into
mainfrom
feat/749-scaffold-default-test

Conversation

@ScriptedAlchemy

Copy link
Copy Markdown
Owner

Closes #749.

What

An aggregate over the existing helpers, no new runner and no flattened module conditions (Rstest's projects cannot carry a per-project pool, so the route pools keep their own react-server fork conditions):

  • mcp-server: test = npm run test:unit && npm run test:routes && npm run test:projection; test:unit is the former test. check = build && typecheck && npm test.
  • cli-tool: test = npm run test:unit && npm run test:projection; check likewise.
  • minimal: unchanged (no route pool — a skills-only project has nothing to render); its README shows the aggregate wiring to add with the first route.
  • Each pool still runs under npm's own per-script banner and every agent-bundle/test helper stamps its proof level, so output keeps distinguishing unit, route-unit, and projection claims; nothing in-memory is labelled native/packed. Focused scripts take Rstest flags after -- (npm run test:routes -- --watch).

Acceptance (in scaffold-packed-matrix.e2e.test.ts, on the packed tarball in a clean consumer)

npm test passes and shows all three pool banners; then the route's rendered text is changed and npm test exits 1 with the unit pool green, the route-unit pool naming tests/route-unit/report-status.test.ts, and the projection pool never reached; npm run test:unit alone still passes.

Docs

quick-start.mdx (leads with npm test), troubleshooting.mdx (rewrote the "green default test" row and the check block, as the #754 coordination note asked), reuse-framework.mdx (one sentence mapping the proof table to the scaffold's pools), en + zh; create-agent-bundle/README.md and the three template READMEs. Changeset for create-agent-bundle (patch).

Local gate (on current origin/main)

  • pnpm build ✅ · pnpm typecheck ✅ · pnpm lint ✅ · pnpm test:unit 4496/4496 ✅
  • node scripts/run-packed-tests.mjs --release …/scaffold-packed-matrix.e2e.test.ts ✅ (mcp-server + cli-tool templates, including the break-the-route case)
  • node scripts/run-packed-tests.mjs …/scaffold-packed.e2e.test.ts
  • pnpm docs:site:build ✅ (parity ok)

@changeset-bot

changeset-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d8fd904

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
create-agent-bundle Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-08T03:58:31.304538Z e6ce162 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pkg-pr-new

pkg-pr-new Bot commented Sep 8, 2026

Copy link
Copy Markdown
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle@763
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/create-agent-bundle@763
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/rsc-markdown-stream@763
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/@agent-bundle/runtime@763

commit: 9b833bd

@ScriptedAlchemy
ScriptedAlchemy force-pushed the feat/749-scaffold-default-test branch from e6ce162 to 9b833bd Compare September 8, 2026 03:55

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6ce162b37

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .changeset/749-scaffold-default-test.md Outdated
"create-agent-bundle": patch
---

Make the scaffolded `npm test` test the plugin: in the `mcp-server` and `cli-tool` templates the default `test` script now runs every deterministic pool the project ships — the plain module tests (`test:unit`, new), the route-unit pool (`test:routes`), and the in-memory projection pool (`test:projection`) — each as its own labeled run, so a route that stops rendering or a server that stops registering it fails the ordinary test command while domain tests stay green. `check` is now `build && typecheck && npm test`. The focused scripts remain for a tight loop and take Rstest flags after `--`; packed and native proof stays opt-in. The `minimal` template still ships no route pool; its README shows the same aggregate wiring to add with the first route (#TBD)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Replace the placeholder with the actual PR reference

When Changesets consumes this file, the literal (#TBD) will be copied into the public changelog even though this change already identifies itself as PR #749. Replace the placeholder with (#749) so the release note has a valid reference.

AGENTS.md reference: AGENTS.md:L167-L168

Useful? React with 👍 / 👎.

Comment on lines +197 to +199
In a scaffolded project the first three rows are the deterministic pools `npm test` runs together
(`test:unit`, `test:routes`, `test:projection`, each reported as its own run); the remaining rows
are explicit, separately labelled gates.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restrict the three-pool claim to the MCP scaffold

This applies the MCP template's script layout to every scaffolded project, but the CLI template only defines test:unit and test:projection, while the minimal template defines none of the three named focused scripts. Users of those templates therefore cannot run the documented set; scope this statement to the MCP starter or describe each template separately.

AGENTS.md reference: AGENTS.md:L139-L142

Useful? React with 👍 / 👎.

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Local Sol Max review (change-risk-reviewer): script behavior, README drift markers, focused watch/filter wiring, host catalog/tests, locale coverage, and changeset package/format all checked out; three prose-accuracy findings, fixed in d8fd904:

  • create-agent-bundle/README.md implied test:routes exists in cli-tool — now names each template's focused scripts.
  • reuse-framework.mdx (en + zh) said every scaffold runs all three pools — now per template (mcp-server: unit/routes/projection; cli-tool: unit/projection; minimal: unit only).
  • Changeset assigned the route-unit pool and "in-memory MCP" wording to both runtime templates — split into the MCP and CLI descriptions.

pnpm docs:site:build and pnpm lint rerun green.

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Re the automatic Codex review (two P1 threads, on the 4c543f3 commit):

  • r3954184219 — changeset (#TBD) placeholder: fixed in 9b833bd; the summary ends with (#763) (the PR number, per .changeset/README.md; the issue is referenced from the PR body).
  • r3954184225 — reuse-framework.mdx three-pool claim applied to every scaffold: fixed in d8fd904 (en + zh) — the sentence now names each template's pools: mcp-server unit/routes/projection, cli-tool unit/projection, minimal unit only. The same scoping was applied to create-agent-bundle/README.md and the changeset.

Resolving both.

@ScriptedAlchemy
ScriptedAlchemy merged commit 795507a into main Sep 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Make the scaffold's default test command test the plugin; keep proof-level separation inside the harness, not in the user's command choreography

1 participant