Compiler evidence: prove self-containment from the Rspack module graph and Artifact IR; delete the generated-JS load scanners (replaces #591/#602)

[ScriptedAlchemy]

Replaces #591 and PR #602 (closed unmerged, owner decision). Shared-compiler-policy step of #592; #604 (Artifact IR / manifest v2) records the compile provenance this issue produces; #555 is the composite root the compiler serves.

Why (owner rationale, 2026-09-05)

The approach in #591/#602 was the wrong layer. A meta-framework must not reverse-engineer emitted JavaScript to discover what its own bundler did. Rspack already knows module identity, resolved resource, external modules, dependency type, issuer, chunk membership, and runtime requirements. A bespoke scanner over emitted output (require(…), createRequire(…), bound loaders, aliases, templates, regex literals, shadowing, minified forms) is a second JavaScript parser maintained in parallel with Rspack and Node syntax, and it is fragile by construction.

Dependency-direction lesson from #602. The scanner flagged Agent Bundle's own createRequire(…).resolve(…) in src/core/dependency-manifest.ts; the runtime was rewritten to an ancestor node_modules walk to satisfy the scanner; Yarn PnP broke (locateFrameworkCli → framework-not-installed). A validator implementation detail must never force runtime architecture. The direction is: runtime requirements → the compiler resolves them → the validator checks the compiler's evidence.

Correct flow

source
  → Agent Bundle compiler service
  → Rspack compilation / module graph
  → external / module resolution report
  → Artifact IR
  → self-containment validation
  → emit artifact

Not: generated JS → reparse → infer → AB6005.

Compiler API

compiler.compile({ entry, runtime: 'node', selfContained: true })
  → CompileResult { assets: AssetIR[]; modules: ModuleIR[]; externals: ExternalIR[]; diagnostics }

selfContained: true means any runtime dependency not explicitly permitted by the artifact contract is a compiler error. Enforcement is one loop over compiler evidence:

for (const external of compilation.externals) {
  if (!isAllowedBuiltin(external.request)) diagnostic('AB6005', …);
}

Deterministic, and indifferent to whether Rspack emits require(...), __rspack_createRequire_require(...), or any other shim: the external is known as an external, not inferred from its emitted spelling.

Closed-world by default

• bundleDependencies: 'all', allowedExternals: ['node-builtins'].
• tools.rspack.externals: ['left-pad'] is rejected by the configuration layer before compilation, or the compilation result shows the external and fails. Either way, no scanner.
• The AGENTS.md "Generated plugin output" rule (no autoExternal, the framework never adds externals, the author's tools hatch cannot externalize a dependency on the author's behalf) is unchanged; the compiler policy is what enforces it, at configuration time and again on the compilation result.

Evidence adapter

A tiny Rspack plugin — ArtifactDependencyAuditPlugin — taps thisCompilation, captures external modules and unresolved runtime loads, and attaches structured evidence to CompileResult. Alternatives, in preference order if the plugin cannot see something: compilation modules / external-module metadata (ExternalModule.request, externalType, issuer via module reasons), then stats module reasons.

Opaque content

Content Agent Bundle did not compile — assets/, prebuilt native executables, verbatim-copied JavaScript, external command integrations — is either fully self-contained by contract or a declared opaque dependency:

definePrebuilt({ path, runtimeDependencies })
  → Artifact IR { kind: 'prebuilt', path, dependencies }

Never "parse arbitrary JavaScript harder".

AB7014 (unused dependency) from evidence

Settled (owner decision, 2026-09-05 16:20 UTC, option A): bundled ≠ used. A package.json dependencies/optionalDependencies/peerDependencies entry counts as used only when it must exist at consumer run time: (1) a definePrebuilt runtimeDependencies declaration, (2) a shipped .d.ts reference read with TypeScript's preProcessFile, (3) the install-script grammar (package.json scripts, bin commands, files, preloads), or (4) the framework's process-dependency record — which is empty today: no runtime module bundled into a generated executable resolves or spawns a package. A package the compiler merely inlined stays AB7014; the graph evidence (ModuleIR.package) upgrades the message to name the bundle that inlined it. No regex or token scanner over packed files.

Invariant

For every compiler-owned executable, all runtime code is one of:

1. a bundled module,
2. a Node built-in,
3. an artifact-relative emitted module,
4. an explicitly declared opaque/prebuilt dependency.

Anything else is a build error, proven before artifact emission. Artifact validation is then defense in depth only: files exist, hashes match, paths are inside the root, manifests reference declared executables — no JavaScript syntax understanding at all.

Implementation order

1. Central compiler service.
2. Rspack compilation evidence adapter (ArtifactDependencyAuditPlugin).
3. External dependency policy (closed-world configuration layer + compilation-result check).
4. Artifact IR records compile provenance (with feat(manifest): authoritative agent-bundle.manifest.json v2 — consumers read the composite root through the manifest (#592 step 3, #555 W2/S3) #604).
5. AB6005 evaluates the external / module graph evidence.
6. Delete the generated-JS load scanners (inventory below), and rewrite the docs that describe them.

PR-sized steps: 1–3, then 4–5, then 6 (which also carries the AGENTS.md, docs/diagnostics.md, and website en+zh validation / entry-conventions updates). One minor changeset per PR.

Reordered by the owner audit of 2026-09-05 17:11 UTC (see comments): step 4 lands first as a persisted compile evidence record (agent-bundle.compile-evidence.json, #638); step 6 deletes a scanner only where compiler evidence or an explicit declaration replaces it, per the coverage matrix posted below — everything else stays, gated so it skips what the record proves.

Behavioral tests to preserve from #602

Each of these must hold on compiler evidence, with a fixture that actually builds:

• An externalized package must fail: tools.rspack.externals: ['left-pad'] (import form) and externalsType: 'node-commonjs' (the createRequire shim form) both fail AB6005 in a host pack and in a package build's dist.
• A Node built-in must pass, with and without the node: prefix, imported or required.
• A relative emitted module must pass: an artifact-relative .js/.mjs reached from an entry is fine; a relative target that is missing, outside the root, or not in the manifest fails.
• An opaque prebuilt remains opaque: a prebuilt payload module that loads a bare package neither fails AB6005 nor loses the dependency under AB7014; the dependency is kept by the prebuilt declaration.
• tools.rspack cannot escape the bundling policy: externals, externalsType, autoExternal, and alias tricks on reserved specifiers are rejected at configuration time or on the compilation result.
• Framework-generated modules (entry-shell.ts bins and MCP entries, install bins, hook wrappers, the composite root's bin/ and mcp/ entries, agent-bundle/meta, agent-bundle/mcp-apps) compile with zero externals other than built-ins — the audit AB6005: also refuse bare createRequire/require/import.meta.resolve loads in compiled modules #591 asked for, now as a compile-evidence assertion instead of a scan of rendered source.

The salvaged fixtures and test files from #602 are in the branch history of feat/591-ab6005-module-loads (commit 9ab94d19d) and were extracted with a README to /tmp/602-salvage/ on the owner's machine; the useful ones are the package-build.test.ts externals fixtures (node-commonjs shim, createRequire literal/computed), the prepack.test.ts prebuilt-payload fixture, and the generated-code audit list in generated-module-loads.test.ts (the list of generators, not the scan).

Scanners to delete (inventory, current main)

Scanner	Where	Replaced by
Emitted-JS import walk for AB6005 (host packs)	src/build/validate-artifact-modules.ts — validateJavaScriptModules, resolveJavaScriptImport; called from src/build/validate-artifact.ts (validateArtifact, ~L612)	CompileResult.externals/modules checked by the external dependency policy (step 3/5); the relative-target file checks stay as the manifest/file-table check (defense in depth, no JS parsing)
#588 dist walk for AB6005 (package builds)	src/build/package-build.ts — the selfContainment block in buildPackageOutputs (~L383) calling validateJavaScriptModules over the staged dist	The same compiler evidence for the package build's compilations (dist/bin/*, Flight workers, lib)
Residual reserved-import lex	src/build/rslib.ts — assertNoResidualReservedImports (readModuleImports over emitted bundles)	Reserved specifiers appear in externals evidence or not at all; the config-layer rejection (reservedExternalsViolation, guardReservedExternals) stays
AB7014 import/require token scanning over packed files	src/build/pack-dependencies.ts — moduleLoads, javaScriptEvidence, fileEvidence, literalLoad, computedLoad, loaderReference, loaderBinding, factoryNames, loaderNames, codeOnly, decodeLiteral, declarationSpecifiers, importedPackageNames; consumed by src/build/pack-inventory.ts dependencyDiagnostics	Dependency usage from CompileResult.modules (which packages the graph references) plus prebuilt declarations (definePrebuilt({ runtimeDependencies })); install-script and bin command evidence stays declarative (package.json), not scanned from JS
ES-module lexer cache	src/build/module-imports.ts — readModuleImports, bundleSyntaxCheckFor (es-module-lexer + acorn parse)	No consumer once the three walks above go; routes/syntax.ts (TypeScript AST over source, not emitted output) is unaffected
#602's leaf scanner (never merged)	src/build/module-loads.ts — scanModuleLoads	Nothing; do not land

AGENTS.md

"Generated plugin output" currently says: "Proof is bytes and processes, not config: every artifact build walks the compiled host-pack modules (AB6005 fails a bare package specifier there …)". When the scanners go (step 6), that sentence must be rewritten to "proof is compiler evidence + packed-process tests": the compilation's external/module report is the proof of self-containment, and the packed pool (pnpm test:packed, packed-deleted-source) remains the process-level proof. The rest of the section — autoExternal: false, bundle: true, splitChunks: false, no framework externals, MCP App views inlined — is unchanged and is the policy the compiler enforces.

[ScriptedAlchemy]

Design (implementer's comment before PR 1)

Evidence behind this design: a prototype Rslib 1.0.0 / Rspack 2.2.2 harness under the framework's exact profile (bundle: true, format: 'esm', output.autoExternal: false, target: 'node', splitChunks: false) run over twelve fixtures, plus a read of every module on the current build and prepack paths.

What the compiler already proves

Shape in source	Rspack's view under the framework profile
import 'node:fs'	ExternalModule, identifier external module "node:fs", two incoming connections from the issuer (esm import, esm import specifier)
import 'left-pad' with externals: ['left-pad'] (array, object, or function form; with or without externalsType: 'node-commonjs')	ExternalModule, identifier external module "left-pad" / external node-commonjs "left-pad"; stats.toJson({ modules: true }) names it external "left-pad". The __rspack_createRequire_require shim is an emit detail of the same record.
import 'left-pad' not installed	Module not found: Can't resolve 'left-pad' on compilation.errors; rslib.build() throws
import './sibling.js' kept external	external module "./sibling.js" — an artifact-relative external is a first-class record
require('x') inside a bundled CJS module	resolved and bundled (NormalModule), or Module not found
import(expr) / require(expr)	context module or Critical dependency: the request of a dependency is an expression (Rslib's profile sets requireDynamic: false, so a computed require in an ESM entry is left as a free identifier and is a run-time ReferenceError, not a dependency)
createRequire(import.meta.url)('x'), free require('x') in an ESM entry	not a graph edge: no module, no external, no warning

The last row is the honest boundary, and it is the one the issue names: content the compiler did not resolve is opaque and must be declared, never parsed harder. compilation.externalModules does not exist in Rspack 2.x's JS API; externals are compilation.modules filtered by instanceof rspack.ExternalModule, and module.request / module.externalType are undefined on the JS object — the real fields are userRequest, identifier(), readableIdentifier(), nameForCondition().

Module boundaries (packages/agent-bundle/src/build/)

• compile-result.ts (leaf types, no imports from the build graph):

  interface AssetIR { path: string; sourceInputs: readonly string[] }            // today's BundledOutputEvidence
  interface ModuleIR { identifier: string; resource?: string; kind: 'authored' | 'dependency' | 'generated'; package?: string }
  interface ExternalIR { request: string; externalType: string; kind: 'builtin' | 'artifact-relative' | 'package' | 'unresolved'; issuers: readonly string[]; asset: string }
  interface CompileResult { assets: readonly AssetIR[]; modules: readonly ModuleIR[]; externals: readonly ExternalIR[]; diagnostics: readonly Diagnostic[] }

• dependency-audit-plugin.ts — ArtifactDependencyAuditPlugin: taps compiler.hooks.thisCompilation, then compilation.hooks.afterOptimizeModules (externals are complete there; finishModules runs before resolution errors are attached). Records every ExternalModule (userRequest, external type parsed from identifier(), issuers via compilation.moduleGraph.getIncomingConnections(module)), every Critical dependency warning as an unresolved external, and the resolution errors, keyed by the lib id composeEntryLibConfig already assigns. Appended by enforceInvariants after the tools hatch so no hatch layer can remove it; its name joins the framework-owned registrations AB4724 guards.
• external-policy.ts — one predicate, two enforcement points:
  • allowedExternal(external): Node built-ins (node: prefix or module.isBuiltin) and pnpapi (Rslib's node target already leaves it external; AGENTS.md names it); artifact-relative only when the target is an emitted asset of the same compilation (checked against CompileResult.assets); everything else, including unresolved, is AB6005.
  • configuration layer: validateTools (config/validate.ts) rejects statically visible tools.rsbuild.output.externals, tools.rsbuild.output.autoExternal, object-form tools.rspack.externals / externalsType / externalsPresets with a new AB4725 (the AB4720–AB4724 family). enforceInvariants re-forces output.autoExternal: false from the invariant layer (today it only pins cleanDistPath) and generalises reservedExternalsViolation from "reserved specifiers" to "any resolved externals the profile did not put there" — mutator and function forms it cannot see statically surface as externals in the compilation and fail there.
• compiler.ts — the service. compile({ cwd, meta, outputRoot, tools?, surfaces, selfContained: true }) wraps today's buildRslibSurfaces batching (one Rslib instance, many libs) and returns one CompileResult per surface: assets from collectBundledOutputEvidence (kept, extended to emit ModuleIR from the stats.toJson({ modules: true }) it already reads), externals from the plugin. With selfContained: true the policy runs before the result is returned; a violation is DiagnosticError([AB6005 …]) naming the asset, the request, its external type, and the issuer module — the same message whether Rspack emitted import, require, or the createRequire shim. Both build.ts (host pack) and package-build.ts (dist/bin, Flight workers, lib) call the service; the package build's Rsbuild browser stage (mcp-apps.ts) inlines everything into one HTML file and stays as is.

composeEntryLibConfig becomes closed-world

Profile unchanged (autoExternal: false, bundle: true, splitChunks: false, no externals). Layer order stays profile → tools.rsbuild → tools.rspack → invariants; the invariant layer gains output.autoExternal: false and the plugin, and its mutator rejects any externals entry rather than only reserved ones. allowedExternals is therefore not a config key: the artifact contract is "Node built-ins", and the only way to widen it is a declaration, not a bundler option.

ExternalIR / ModuleIR relative to #604 (PR 2)

#604's manifest v2 file record is { bytes, kind, mode?, path, sha256, sourceInputs } — no compile slot; the only provenance there belongs to routes. Proposed additions once #604 lands: an optional compile: { externals: { request, type }[]; dependencies: string[] } on kind: 'bundle' rows (built-in externals and the package names the graph reached), dependencies: string[] on kind: 'prebuilt' rows (from definePrebuilt), and producer.compiler: { name: 'rspack', version }. sourceInputs is already asset evidence and stays. validateArtifact then re-derives AB6005 from the manifest — a bundle row whose external is not a built-in fails — and keeps the file-table checks (relative targets exist, resolve inside the root, are listed); it never opens a JavaScript file. Schema source, manifest-schema.ts, emit.ts::createArtifactManifestFiles, and build.ts::manifestFor change together.

definePrebuilt surface

Today payload: { <dest>: '<dir>' | { source, targets? } } declares opaque trees (AgentBundlePayloadEntry, NormalizedPayload). Add, exported from agent-bundle beside defineConfig:

definePrebuilt({ path: './built/tools', targets?: ['claude'], runtimeDependencies: ['sharp'] })
// → AgentBundlePayloadEntry & { runtimeDependencies }

normalizePayloads carries runtimeDependencies onto NormalizedPayload; the Artifact IR row is { kind: 'prebuilt', path, dependencies }. Validation: names must be bare package names present in dependencies/optionalDependencies (else a config diagnostic). A compiled entry that shells out to a dependency's bin (spawnSync('tsc')) is the one case neither the graph nor a payload covers; it gets the same runtimeDependencies field on the CLI/MCP entry config in PR 3 if a fixture needs it — flagged below.

AB6005 / AB7014 re-derivation

• AB6005 (PR 1 at compile time, PR 2 also from the manifest): every ExternalIR not allowed by the policy; every resolution error is already a hard compile failure. Relative-target findings (is missing, resolves outside the artifact root, is not listed in the artifact manifest) survive as file-table checks. Syntax findings disappear: the compiler produced the bytes.
• AB7014 (PR 3): a declared dependency is used when (a) any ModuleIR of the package build's compilations resolves under node_modules/<name>, (b) a definePrebuilt (or entry) runtimeDependencies names it, (c) the lib declaration compilation's emitted .d.ts references it — read with TypeScript's own preProcessFile and resolveTypeReferenceDirective over the framework's own dts output, never a regex, or (d) a package.json install script or imports-map target names it (package-manifest grammar, unchanged). Otherwise unused. The withhold state is deleted: a computed load in compiled code is a compile error under the invariant, and an opaque file contributes only its declaration. AB7015's "survivable" downgrade keeps only install-script command evidence plus the declared dependencies of prebuilt files a script names.
• The framework's own serve-app-command → dependency-manifest.ts (createRequire(...).resolve(...), Yarn PnP-aware) is not a module load of the executable; it is a process dependency on the installed agent-bundle. The compiler records it as a declared process dependency when that runtime module is in the graph (the runtimeModulePath registry entry-shell.ts already keeps), so the runtime is never rewritten for the validator.

Deletion list (PR 3, each with what replaces it)

Delete	Replaced by
validate-artifact-modules.ts (validateJavaScriptModules, resolveJavaScriptImport) and its call in validate-artifact.ts	CompileResult.externals under the policy; manifest compile rows; file-table relative-target checks
package-build.ts selfContainment block (the #588 dist walk)	the same compiler service the package build now calls
rslib.ts::assertNoResidualReservedImports	reserved specifiers either appear in externals (rejected) or were bundled
module-imports.ts (readModuleImports, bundleSyntaxCheckFor) and the bundleSyntaxCheck plumbing in build.ts / validate-artifact.ts / api.ts; acorn if nothing else imports it	no consumer
pack-dependencies.ts literal/loader machinery (moduleLoads, javaScriptEvidence, fileEvidence, literalLoad, computedLoad, loaderReference, loaderBinding, factoryNames, loaderNames, codeOnly, decodeLiteral, declarationSpecifiers, typeDirectivePackages, packedMains, packedModule, installScriptFiles, installScriptModuleDependencies); importedPackageNames reshaped to take CompileResult + declarations	(a)–(d) above; installedDependencyFields, declaredDependencies, packageNameOf, classifyDependency, shellWords, the shell/node/npm grammar, executableCommands stay
pack-inventory.ts imported.complete ? … : [] and the "withholds this check" recovery sentence	no withhold state
Docs: AGENTS.md "Proof is bytes and processes" → "proof is compiler evidence + packed-process tests"; docs/diagnostics.md AB60xx summary row, AB6005 row, AB7014/AB7015 rows and evidence paragraphs, the AB4837 aside; docs/entry-conventions.md tools and serve-app-command sections; website/docs/{en,zh}/guide/distribution/validation.mdx	rewritten to the evidence model

PR plan

1. PR 1 — steps 1–3: compile-result.ts, dependency-audit-plugin.ts, external-policy.ts, compiler.ts; build.ts and package-build.ts rewired to the service; closed-world composeEntryLibConfig; AB4725; compile-time AB6005 from evidence (the old walk still runs until PR 3 and the tests assert both agree). Behavioural tests from the issue: externalized package fails (array, object, function, node-commonjs), built-in passes, artifact-relative emitted module passes, opaque prebuilt stays opaque, tools.rsbuild/tools.rspack cannot escape (config-time and compile-time). Docs: AB4725 row, AB6005 message form, tools section, website en+zh. One minor changeset.
2. PR 2 — steps 4–5 (after feat(manifest): authoritative agent-bundle.manifest.json v2 — consumers read the composite root through the manifest (#592 step 3, #555 W2/S3) #604 merges): manifest compile / dependencies rows, producer.compiler, definePrebuilt export, validateArtifact AB6005 from the manifest, the framework process-dependency record. minor changeset.
3. PR 3 — step 6: AB7014 from evidence + declarations, the deletion list, AGENTS.md and docs rewrite. minor changeset.

Each PR: lanes in parallel, a deslop pass over the full diff before each reviewer pass (Sol or Fable -high, never Grok), origin/main merged before every gate.

Decisions I am proceeding on unless overruled

• Declaration references for AB7014 come from TypeScript's parser over the framework's own dts compilation output (option (c) above); the alternative is an explicit author list.
• A compiled entry that spawns a dependency's bin declares it with runtimeDependencies on the entry (PR 3, only if a fixture needs it).
• AB4725 is the config-time code for hatch externalization; the compile-time finding stays AB6005.

[ScriptedAlchemy]

Scope note from the #623 review (Codex P2): compileMcpApps (Rsbuild, AB477x) does not go through the compiler service, so a function-form or mutator externals in an MCP App build escapes both AB4725 and evidence today. PR 2 (steps 4–5) installs ArtifactDependencyAuditPlugin in composeMcpAppsRsbuildConfig's invariant layer and fails a view on any recorded ExternalModule — a browser document has no allowable external — replacing the script/link URL heuristic in assertSelfContainedViews.

[ScriptedAlchemy]

PR 1 merged: #623 (8b0a6b9) — compiler service (build/compiler.ts), Rspack evidence adapter (build/dependency-audit-plugin.ts), external policy (build/external-policy.ts, AB4725 config-time, compile-time AB6005 judged on external type + run-time request). Steps 4–5 next (PR 2): every surface's Artifact IR records compile provenance; AB6005 from evidence for MCP App views too (see scope note above); definePrebuilt.

[ScriptedAlchemy]

PR 2a merged: #627 (02b8972) — MCP App views judged from compile evidence: ArtifactDependencyAuditPlugin in composeMcpAppsRsbuildConfig's invariant layer, any recorded ExternalModule in a view is AB6005 (viewSelfContainmentDiagnostics). Remaining step 4–5 work (manifest compile/dependencies rows, producer.compiler, validateArtifact re-deriving AB6005 from the manifest) waits on #604; definePrebuilt + runtimeDependencies on payload entries proceeds now as PR 2b since it touches only config/normalization.