diff --git a/.changeset/591-ab6005-module-loads.md b/.changeset/591-ab6005-module-loads.md new file mode 100644 index 000000000..52de19378 --- /dev/null +++ b/.changeset/591-ab6005-module-loads.md @@ -0,0 +1,5 @@ +--- +"agent-bundle": minor +--- + +Hold every emitted JavaScript module to `AB6005` for what it loads, not only for what it imports: `agent-bundle build`, `agent-bundle validate`, and `agent-bundle prepack` now fail a host-pack module or a package build `dist` bundle whose code calls `require("pkg")`, `require.resolve("pkg")`, `createRequire(…)("pkg")` or `.resolve("pkg")` — the factory written out, namespace-qualified, aliased (`import { createRequire as mk }`), or bound to a name first (`const load = createRequire(import.meta.url); load("pkg")`, the shim Rspack emits for a `tools.rspack` `externalsType: 'node-commonjs'` external) — or `import.meta.resolve("pkg")` with a bare specifier that is not a Node built-in; a non-literal argument to any of those calls and a loader passed on as a value rather than called (`const l = require`, `fn(load)`) fail the same way. Node built-ins under either spelling still pass; a relative or `file:` target must be a listed regular `.js`/`.mjs` file inside the tree, which is walked, or, in a host pack, listed valid JSON, which is accepted; prebuilt payload modules stay untouched. Each diagnostic names the call — `… uses unsupported specifier "left-pad" in require("left-pad").`, `… loads a non-literal specifier through load(…), a createRequire(…) loader.` — and the import messages are unchanged. The prepack gate's `require`/`createRequire`/`import.meta.resolve` evidence for `AB7014` comes from the same scanner, which reads code only: a `require("pkg")` inside a comment, string, regular-expression literal, or the text of a template no longer counts (a load inside a template's `${…}` substitution does), so a dependency a packed file mentions only in a docblock is reported as unused, and the `AB7014` recovery names where load evidence can still come from (a prebuilt payload module, packed JavaScript the `files` allowlist adds from outside the artifact and `dist`, a packed declaration reference, an install script, or a `bin` command). `agent-bundle/serve-app-command` and the build's dependency-root discovery locate a dependency only through `node_modules` at the project root or an ancestor; `NODE_PATH`, Node's global module folders, and Yarn Plug'n'Play are no longer consulted. (#602) diff --git a/AGENTS.md b/AGENTS.md index 7b42d4448..16ef9b0db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,34 +71,64 @@ every dependency of a generated executable — `output.autoExternal: false`, `bundle: true`, `splitChunks: false`, no `externals`. Rslib's `node` target leaves only Node built-ins (and `pnpapi`) external, and the only bare - import specifiers `AB6005` accepts in a host-pack module are Node built-ins. The + specifiers `AB6005` accepts in a host-pack module are Node built-ins. The package build's `dist` bundles are walked by the same `AB6005` rule (`src/build/package-build.ts` reuses `validateJavaScriptModules` from `src/build/validate-artifact-modules.ts`), so a generated executable in a - host pack or in `dist` imports nothing but Node built-ins from outside its - tree. The walk reads import specifiers, static and literal dynamic; a - `createRequire(…)(…)` or `import.meta.resolve(…)` call is not an import and - is outside `AB6005` in either output — the prepack gate reads those calls - as dependency evidence. MCP App views (`src/build/mcp-apps.ts`) inline every script and style - into one HTML file. The framework never adds `externals` to a plugin build; + host pack or in `dist` loads nothing but Node built-ins from outside its + tree. The walk reads the recognised load forms in an emitted + `.js`/`.mjs` module (prebuilt payload modules excepted, `.d.ts` never + walked): its ES import records — static, literal dynamic `import()`; a + non-literal one is a finding — and `require(…)`, `require.resolve(…)`, + `createRequire(…)(…)` and `.resolve(…)` with the factory written inline, + namespace-qualified, or imported/destructured under an alias, a loader + declared with `const`/`let`/`var` from `createRequire(…)` and then called + directly or through `.resolve(…)`, and `import.meta.resolve(…)`; optional + calls and a trailing comma after the literal count the same. Comments, + strings, regular-expression literals, and template text are stepped over; + `${…}` template substitutions are code and are scanned; binding and alias + names are read from code only. A Node built-in passes; a relative or + `file:` target must be a listed regular `.js`/`.mjs` file inside the tree + and is walked in turn (in a host pack only, listed valid JSON is accepted + as a terminal and not walked). A bare package name fails, a non-literal + argument fails, and a loader used as a value rather than called fails — + a binding position (parameter, `catch`, destructuring pattern, import + specifier) is not a use as a value; a default initializer (`x = require`) + is. One scanner, + `src/build/module-loads.ts`, reads those loads for `AB6005` and for the + prepack gate alike; its header states the approximations it makes (a `/` + after `)` or an identifier is division, and the hand-authored forms it + does not recognise: `.call`/`.apply`, `globalThis.require`, + `module.require`, `import.meta["resolve"]`, assignment-bound or + second-hop loader aliases). MCP App + views (`src/build/mcp-apps.ts`) inline every script and style into one + HTML file. The framework never adds `externals` to a plugin build; the `externals` handling in `rslib.ts` (`reservedExternalsViolation`, `guardReservedExternals`) only rejects reserved specifiers in the resolved externals, which come from the author's `tools` hatch and Rslib's built-in list, never from the profile. - No refactor, toolchain upgrade, or "leaner install" change may enable `autoExternal` or externalize a dependency on the author's behalf. A package - a consumer must install is the author's explicit decision, and an import - kept external through the `tools` hatch is not a way to make it anywhere: - `AB6005` fails such an import in a host pack and in `dist` alike. What - legitimately puts a package under `dependencies` is a packed declaration - reference, a prebuilt payload module that imports it, an install script, - or a `bin` command packed JavaScript runs — and the prepack gate judges - those: `AB7014` demands that evidence, `AB7015` a specifier a consumer's - npm can install. + a consumer must install is the author's explicit decision, and keeping a + dependency external through the `tools` hatch is not how that decision is + made, in any emitted form: an ES `import` external and the + `node-commonjs` `createRequire` shim both fail `AB6005` in a host pack and + in `dist` alike; a direct `require("pkg")` call, however emitted, is + rejected the same way. `AB7014` lexes every packed + `.js`/`.mjs`/`.cjs` file, including `dist` and artifact files. Because + `AB6005` has already refused a bare load in every walked emitted module + before that inventory runs, the evidence that can still keep a dependency + in a build that passed comes from a prebuilt payload module, packed + JavaScript the `files` allowlist adds from outside the artifact and + `dist`, a packed declaration reference, an install script, or a `bin` + command. The prepack gate judges those: `AB7014` demands that evidence, + `AB7015` a specifier a consumer's npm can install. - Proof is bytes and processes, not config: every artifact build walks the compiled host-pack modules and every package build walks its emitted `dist` - bundles (`AB6005` fails a bare package specifier in either), the prepack - gate then judges what remains declared, and the packed pool + bundles (`AB6005` fails a bare package specifier in either, imported or + loaded through `require`, a `createRequire(…)` loader, or + `import.meta.resolve`), the prepack gate then judges what remains + declared, and the packed pool (`pnpm test:packed`) installs the packed tarball into a clean consumer, builds, removes the project source, and spawns the generated entry as a real process (`packed-deleted-source`). diff --git a/docs/diagnostics.md b/docs/diagnostics.md index 310925cd5..3cae0dc48 100644 --- a/docs/diagnostics.md +++ b/docs/diagnostics.md @@ -32,7 +32,7 @@ even when no error diagnostic was reported. | `AB490x`/`AB492x` | Conventional host components (#100 stage 2): rules `src/rules/*.mdc` (`AB4900`–`AB4908`) and commands `src/commands/*.md` (`AB4920`–`AB4928`), including per-host feature-set enforcement (`AB4907`/`AB4908`, `AB4927`/`AB4928`); see below. | | `AB48xx`/`AB494x` | Route graph, state, layout (`AB4830`–`AB4832`), generated route declarations outside the TypeScript program (`AB4834`), route render budgets (`AB4835`), tool task support (`AB4836`), a route module that value-imports a compiler-carrying framework entry (`AB4837`), a CLI route `inputSchema` reference the static resolver cannot follow (`AB4838`) or that cycles (`AB4839`), and provider conventions (see below). | | `AB5000` | General CLI and adapter failures (see below). | -| `AB60xx` | Built-artifact validation, including schema documents and referenced files (`AB6005`: an emitted JavaScript module — a host-pack module or a package build `dist` bundle (`dist/bin/*.js`, the Flight workers, the `lib` entry), prebuilt payloads excepted — has an import that is neither a Node built-in nor a relative or `file:` specifier resolving to a listed regular file inside its tree, or a non-literal dynamic import; a `dist` finding names `dist/`; `AB6011`/`AB6012`: a target's required pinned-schema document is missing or invalid; `AB6025`: a manifest-declared `logo` path is missing from the artifact or escapes the deploy tree; `AB6034`: emitted Skill Markdown has no instruction body; `AB6035`–`AB6038`: Agent Plugins portable validation, see below). | +| `AB60xx` | Built-artifact validation, including schema documents and referenced files (`AB6005`: an emitted `.js`/`.mjs` module — a host-pack module or a package build `dist` bundle (`dist/bin/*.js`, the Flight workers, the `lib` entry), prebuilt payloads excepted, `.d.ts` never walked — `cannot be read.`, `has invalid syntax.`, or loads something that is neither a Node built-in nor a relative or `file:` specifier resolving to a listed regular `.js`/`.mjs` file inside its tree (a host pack may also accept a relative target to listed valid JSON without traversing it). Messages include `uses invalid specifier`, `uses unsupported specifier`, `uses invalid file URL`, `is missing`, `does not resolve to a regular file`, `resolves outside the artifact root`, `is not listed in the artifact manifest`, `references invalid JSON`, and `uses unsupported target`; the recognised forms are imports, `require(…)`/`require.resolve(…)`, direct or aliased `createRequire(…)` loaders and their `.resolve(…)`, and `import.meta.resolve(…)`. Comments, strings, regular-expression literals where an operand is expected, and template text are stepped over, while `${…}` substitutions are scanned. A non-literal dynamic import is reported, a non-literal loader call says `loads a non-literal specifier through `, and a loader reference says `passes on as a value instead of calling it`; load messages append `in ` when a concrete call is available (for example `… in require("left-pad").`), and a `dist` finding names `dist/`; `AB6011`/`AB6012`: a target's required pinned-schema document is missing or invalid; `AB6025`: a manifest-declared `logo` path is missing from the artifact or escapes the deploy tree; `AB6034`: emitted Skill Markdown has no instruction body; `AB6035`–`AB6038`: Agent Plugins portable validation, see below). | | `AB6200`–`AB6202` | Workbench artifact inspection over published epochs: `AB6200` the epoch does not validate or its provenance is inconsistent, `AB6201` an epoch reference could not be released, `AB6202` unsafe runtime metadata (see below). | | `AB700x` | Host installation and uninstallation: bundle identity, host availability, scope, command failure, and collision checks (`AB7000`–`AB7004`: unsupported host, unreadable bundle identity, missing host, scope or mode refusal, host command failure — the same five codes are also the development project service's preparation failures; `AB7005`: version collision, pre-receipt content collision, or foreign install; `AB7006`: the host lists the installed copy with load errors; see below), plus the `uninstall` refusals `AB7007`–`AB7009` (ownership or content mismatch, unconfirmed data purge, missing receipt; see below). | | `AB7010`–`AB7015` | npm prepack inventory, artifact freshness, package bin targets, release-version agreement, and installed-dependency hygiene (`AB7014`: a dependency no packed file references; `AB7015`: a git, remote-tarball, path, or unrewritten workspace-protocol dependency specifier). | @@ -426,16 +426,20 @@ Validation happens at three moments, all fail-closed: | `AB7011` | An on-disk artifact file no longer matches its manifest SHA-256. Rebuild and do not modify generated host packs. | | `AB7012` | A `package.json` bin points outside the packed `dist` output (including `src/`) or names a file npm omitted. Point it at the generated `dist/bin` file. | | `AB7013` | `package.json`, normalized plugin metadata, a host manifest, or artifact provenance reports a different release version. Make every release identity agree. | -| `AB7014` | A `package.json` `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses: no packed JavaScript imports, requires, or resolves them, or runs one of their `bin` commands, no packed declaration file references them, no `#subpath` import reaches them through the manifest's `imports` map, and no consumer-side install script (or script it delegates to) runs them (one diagnostic per field; the full evidence list follows this table). Peers `peerDependenciesMeta` marks optional are never installed and are not inspected here (their specifier is still checked by `AB7015`), and a name under both `dependencies` and `optionalDependencies` is judged by its optional entry, which npm lets override. The build inlines every dependency into `dist` and the host packs, and `AB6005` fails a compiled bundle that imports a bare specifier other than a Node built-in (`prepack` runs that build before this inventory), so a compiled bundle can never supply `import` evidence here and such an entry only makes every consumer's `npm install` fetch build-time packages; the packed-JavaScript `import` evidence class is for modules the framework copied rather than compiled — prebuilt payload modules and other packed scripts — while `require`, `createRequire`, and `import.meta.resolve` calls, which `AB6005` does not walk, count from any packed file. Move build-only packages to `devDependencies`; keep a runtime dependency only for what a prebuilt payload module imports, a packed file requires or resolves, a packed declaration references, a `#subpath` import reaches, or an install script or packed file runs. For `peerDependencies` the diagnostic is a warning: a required peer nothing imports may be a deliberate compatibility contract with the host that loads the package, though npm 7+ still installs it for every consumer — keep it, mark it optional in `peerDependenciesMeta`, or move a build-only package to `devDependencies`. | +| `AB7014` | A `package.json` `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses: no packed JavaScript imports, requires, or resolves them, or runs one of their `bin` commands, no packed declaration file references them, no `#subpath` import reaches them through the manifest's `imports` map, and no consumer-side install script (or script it delegates to) runs them (one diagnostic per field; the full evidence list follows this table). Peers `peerDependenciesMeta` marks optional are never installed and are not inspected here (their specifier is still checked by `AB7015`), and a name under both `dependencies` and `optionalDependencies` is judged by its optional entry, which npm lets override. The inventory lexes every packed `.js`/`.mjs`/`.cjs` file, including `dist` and artifact files. Because `AB6005` has already refused a bare load in every walked emitted module before the inventory runs, the evidence that can still keep a dependency in a build that passed comes from a prebuilt payload module, packed JavaScript the `files` allowlist adds from outside the artifact and `dist`, a packed declaration reference, an install script, or a `bin` command. Move build-only packages to `devDependencies`; keep a runtime dependency only when one of those packed surfaces supplies evidence. For `peerDependencies` the diagnostic is a warning: a required peer nothing imports may be a deliberate compatibility contract with the host that loads the package, though npm 7+ still installs it for every consumer — keep it, mark it optional in `peerDependenciesMeta`, or move a build-only package to `devDependencies`. | | `AB7015` | A `package.json` `dependencies`, `optionalDependencies`, or `peerDependencies` entry that a consumer's npm cannot resolve through a registry. Each entry — name and specifier together, the value exactly as written (a leading space makes `" npm:bar@1"` an invalid dist-tag, not an alias) — is read with `npm-package-arg`, the parser npm, Arborist, and pacote share, so the verdict is npm's own rather than an imitation of its grammar: **registry** (a version, range, or dist-tag, or an `npm:` alias of one — the only kind a published package can rely on), **fetched** (parseable, but a `git`/`github:`/`gitlab:`/`bitbucket:`/`gist:` source or `owner/repo` shorthand, an `http(s):` tarball, or a `file:`/relative/bare path or tarball filename — npm 12 refuses git and remote fetches by default (`allow-git=none`, `allow-remote=none`) and a path never exists on the consumer's disk), or **unparseable** (npm rejects the manifest before fetching anything: `EINVALIDPACKAGENAME` for a name such as `bad name`, `.hidden`, or `node_modules`; `EUNSUPPORTEDPROTOCOL` for `link:`, `portal:`, `jsr:`, a `git+` transport npm lacks, or a typo; `EINVALIDTAGNAME` for a selector that is neither a range nor a URL-safe dist-tag, such as `"not a valid spec"`; an `npm:` alias without a name or with a non-registry target, since aliases only work for registry dependencies; or an invalid URL such as `http:%zz`). A fetched specifier is reported on installed entries only; an unparseable one is reported on every entry, even an optional peer npm would never install, because the manifest read itself fails. A peer that `dependencies` or `optionalDependencies` also names is judged by that concrete entry alone: npm resolves the concrete declaration and never reads the duplicate peer's selector. For an `optionalDependencies` entry that is fetched, the diagnostic is a warning, not an error (`agent-bundle prepack` prints it and exits 0): npm continues an install without such a dependency, but every consumer still tries and fails to fetch it. It stays an error when the entry is unparseable, or when a consumer-side install script needs the skipped package — runs one of its `bin` commands in command position (`setup-tool --init`, `npx setup-tool`, `cross-env CI=1 setup-tool`, `./node_modules/.bin/setup-tool`; a mention elsewhere, `echo setup-tool`, proves nothing), runs one of its files (`node node_modules/setup-tool/install.js`), loads it from an inline program (`node -e "require('setup-tool')"`, `node --input-type=module -e "await import('setup-tool')"`, also `-p`, `-pe`, `--eval=…`, `--print=…`; the program is read as a packed file is — `require`, `createRequire`, and `import()` — and a computed load there, or a program the lexer rejects, may need any declared package), preloads it (`node -r setup-tool/register install.js`; `-r`/`--require`, `--import`, `--loader`/`--experimental-loader`, with a space or `=` before the module — read as Node does, `node [options] script [arguments]`: options end at the first positional (the script, or an argument when `-e`/`-p` supply the program) or a `--`, valued options such as `--conditions x` or `--env-file x` taking their word with them, so `node install.js --require x` passes `--require x` to `install.js` and preloads nothing; a `NODE_OPTIONS` assignment on the same command — `NODE_OPTIONS=--require=setup-tool/register node install.js`, `cross-env NODE_OPTIONS="-r setup-tool/register" node .` — supplies options Node applies before the command line's, while one `export`ed by an earlier command is not read), or runs a packed file (`node install.cjs`, `node scripts/install` resolving `scripts/install.js`, `node "scripts/my install.cjs"`, `node install.js&&echo done`, `node .` or `node ./` running the root `main`, `node --import ./setup.mjs .` running a packed preload) that imports it — every word of the script that names a packed JavaScript file counts as run, deliberately, so that runners this gate does not model (`tsx`, `ts-node`, `zx`, `bun`, `deno run`, `npx `) still have the dependencies their file loads traced; the cost is a rare escalation for a word that names a packed file without running it (`echo install.js`), which the diagnostic makes visible by naming the file — directly, through relative imports inside the tarball (`require("./lib")` following `lib/package.json`'s `main` before `lib/index.js`, as Node does), or through the `imports` map resolved as Node does (`"#setup": "./setup.js"`; `#setup/foo` through `"#setup/*": "./scripts/*.js"`, a preloaded `#setup` included): npm continues past the failed fetch, then the script fails on the missing command or module. Each command of a script is read on its own: after a shell operator (`&&`, `;`, and the rest) or a newline — the second line of a script, and each lifecycle script after the first, starts a new command — and Node's options belong to `node` alone (`rm -r dist` preloads nothing, `npm --prefix . run setup` runs no `main`). Depend on a published registry version, or bundle the package and declare it under `devDependencies`. Entries the tarball itself carries are never reported, since a consumer does not fetch them: `bundleDependencies` (by name or `true`; never a peer, which npm cannot bundle; only when the pack inventory contains `node_modules//package.json`, since npm silently packs nothing for a bundled name absent from `node_modules`), and a `file:` or bare path inside the package (`file:vendor/foo`, `file:vendor/foo.tgz`) whose packed source npm can install from — a directory whose packed `package.json` parses to an object, or a packed tarball (gzipped or plain tar, ustar headers with valid checksums and payloads inside the archive) whose `/package.json` entry parses to an object — since npm installs it from the consumer's own copy. A path that escapes the package (`file:../sibling`), whose source is not packed, or whose packed source is not installable (a `.tgz` that is not an archive, or is malformed or truncated, fails the consumer's install with `TAR_BAD_ARCHIVE`; a manifest that does not parse, on disk or inside the archive, fails it with `EJSONPARSE`) is reported. `workspace:` and `catalog:` count as registry specifiers only when the `prepack` lifecycle runs under pnpm, Yarn, or Bun (`npm_config_user_agent`), which rewrite them in the tarball they pack; `npm publish` publishes them verbatim and consumers fail with `EUNSUPPORTEDPROTOCOL`, so under npm — or when `agent-bundle prepack` runs outside any package-manager lifecycle — they are reported. The `npm pack --dry-run` that `prepack` itself spawns is only the file inventory; the tarball consumers receive is the lifecycle's packer's, which is what the user agent identifies. | -Compiled bundles reach this gate without bare imports: `prepack` builds before it packs, and `AB6005` -fails any `dist` bundle or host-pack module whose import is neither a Node built-in nor a listed file inside -its tree, so the lexed `import` evidence below describes prebuilt payload modules and other packed scripts -the framework did not compile. `AB6005` walks import specifiers only; a `require`, `createRequire(…)(…)`, or -`import.meta.resolve(…)` call is not an import, so that evidence is read from every packed file, compiled -bundles included. The dependency evidence is read from the packed bytes themselves: every `.js`/`.mjs`/`.cjs` file -`npm pack --dry-run` lists is lexed for static and dynamic `import` specifiers and scanned for +Emitted modules reach this gate without bare loads: `prepack` builds before it packs, and `AB6005` +fails any `dist` bundle, host-pack module, or copied artifact script whose `import`, `require`, +`createRequire(…)` loader call, or `import.meta.resolve` names anything but a Node built-in or a listed file +inside its tree — the same scanner (`src/build/module-loads.ts`) reads those loads for `AB6005` and for this +gate. The inventory nevertheless lexes every packed `.js`/`.mjs`/`.cjs` file, including `dist` +and artifact files. In a build that passed, bare-load evidence can still come from a prebuilt +payload module, JavaScript the `files` allowlist packs from outside the artifact and `dist`, or an +inline `node -e` install program; declaration, install-script, and `bin` evidence can also keep a +dependency. A computed load through a recognised loader call in packed code still withholds +`AB7014`, as described at the end of this section. Every packed JavaScript file is lexed for static +and dynamic `import` specifiers and scanned for literal `require("…")` and `.resolve("…")` calls (`require.resolve`, `createRequire(…).resolve`, `import.meta.resolve`: a package located only to find an asset is still a runtime dependency; a binding such as `const load = createRequire(import.meta.url)` is a loader and `load("…")` counts like @@ -452,9 +456,11 @@ their package name (`@scope/name` or `name`), string escapes decoded first (`req and Node built-ins are ignored. A dependency packed JavaScript runs rather than loads — a string literal that is one of the `bin` commands its manifest under `node_modules` declares, bare or followed by arguments (`spawnSync("tsc", ["--version"])`, `execSync("tsc --noEmit")`) — counts as used too; a dependency not installed -at pack time has no known commands, so its bare name in a string proves nothing here. A mention inside a -comment or string can only keep a dependency, never report one, and `devDependencies` are never -inspected. A packed `#subpath` import counts for every package the `imports` entry Node would pick for it targets +at pack time has no known commands, so its bare name in a string proves nothing here. A +`require` or import mention inside a comment, string, template text, or regular-expression literal +is neither evidence nor a load; only install-script command text and `bin` command strings are text +evidence. `devDependencies` are never inspected. A packed `#subpath` import counts for every +package the `imports` entry Node would pick for it targets (the exact key, or the wildcard key with the longest matching prefix, its `*` substituted — every conditional target of that entry, since conditions are not settled here), and a dependency named — anywhere in the text, since a mention can only keep a declaration — by a consumer-side `preinstall`/`install`/`postinstall` script (not `prepare`, which npm @@ -1733,7 +1739,7 @@ names the host target namespace when the check is per target. | `AB6001` | error | `Artifact manifest is not a strict canonical manifest.` — `agent-bundle.manifest.json` does not parse as a strict canonical artifact manifest. `Artifact manifest changed during validation.` — its bytes or identity differ between the first read and the re-read after validation. | Regenerate the strict canonical manifest without concurrent writes, then rerun validation. | | `AB6002`–`AB6003` | error | Reserved: both codes are declared in the artifact diagnostic registry, but no validator emits either today. | `AB6002`: Rebuild the artifact from complete project source, then rerun validation. `AB6003`: Rebuild the artifact with canonical generated output, then rerun validation. | | `AB6004` | error | `Artifact files do not match the manifest.` — the regular files on disk differ from the manifest file table (a path, byte length, mode, or SHA-256; a missing or unmanifested file). `Artifact file changed during validation: "".` — a file differed between the initial and final inspection, or between a validated staging tree and its re-check after `build` renamed it into place. `Artifact file table changed during validation.` — the final inspection could not be taken. | Rebuild the artifact so its file table and contents match the manifest. | -| `AB6005` | error | `Generated JavaScript import from "" .` — an emitted JavaScript module — a host-pack module or a package build `dist` bundle (`dist/bin/*.js`, the Flight workers, the `lib` entry), prebuilt payloads excepted — has an import that is neither a Node built-in nor a relative or `file:` specifier resolving to a listed regular file inside its tree (`uses unsupported specifier`, `is missing`, `resolves outside the artifact root`, `is not listed in the artifact manifest`, `does not resolve to a regular file`, `references invalid JSON`, `uses unsupported target`), or the module cannot be read, has invalid syntax, or has a non-literal dynamic import; a `dist` finding names `dist/`. The walk covers `import` specifiers only — `require`, `createRequire`, and `import.meta.resolve` calls are the prepack inventory's business (`AB7014`). | Bundle every JavaScript dependency into the artifact, then rebuild it. | +| `AB6005` | error | `Generated JavaScript import from "" .` — an emitted JavaScript module — a host-pack module or a package build `dist` bundle (`dist/bin/*.js`, the Flight workers, the `lib` entry), prebuilt payloads excepted — has an import or a load that is neither a Node built-in nor a relative or `file:` specifier resolving to a listed regular `.js`/`.mjs` file inside its tree (`uses invalid specifier`, `uses unsupported specifier`, `uses invalid file URL`, `is missing`, `resolves outside the artifact root`, `is not listed in the artifact manifest`, `does not resolve to a regular file`, `references invalid JSON`, `uses unsupported target`; a load's message appends `in `, for example `in require("left-pad")`), or the module `cannot be read`, `has invalid syntax`, `has a non-literal dynamic import`, `loads a non-literal specifier through `, or `passes on as a value instead of calling it`; a `dist` finding names `dist/`. The recognised loads are `require(…)`, `require.resolve(…)`, `createRequire(…)(…)` and `.resolve(…)` — the factory bare, qualified, or aliased, or bound to a `const`/`let`/`var` and called later — and `import.meta.resolve(…)`, read by `src/build/module-loads.ts` from code only: comments, strings, template text, and regular-expression literals are stepped over, `${…}` substitutions are scanned. `.d.ts` files are never walked; in a host pack a relative target to listed valid JSON is accepted and not walked. | Bundle every JavaScript dependency into the artifact, then rebuild it. | | `AB6006` | error | `Generated JSON cannot be parsed.` — a `.json` file in the artifact is not valid JSON (prebuilt payload files are exempt). Doctor's Claude document lane reports the same code inside an `AB7319` message for a Claude bundle document that is unreadable or not valid JSON. | Regenerate the affected JSON document as valid JSON, then rebuild the artifact. | | `AB6007` | error | `MCP manifest references missing generated server "".` — a root-level MCP manifest (pre-manifest pass) or a target's MCP manifest names a local server entry that the artifact does not contain. | Repair MCP manifest references to generated servers, then rebuild the artifact. | | `AB6008` | error | `Artifact Agent Skills provenance does not match the pinned schema contract.` — the manifest's `agentSkills` schema SHA-256, source revision, or specification differs from the framework's pinned Agent Skills revision. | Rebuild the artifact with the pinned Agent Skills contract. | diff --git a/docs/entry-conventions.md b/docs/entry-conventions.md index aef00c10b..8adbd0b9c 100644 --- a/docs/entry-conventions.md +++ b/docs/entry-conventions.md @@ -1224,11 +1224,15 @@ hatch customizes *how code compiles*, never *what the artifact promises*. The framework's own profile keeps the same promise: `output.autoExternal` is `false`, `bundle: true`, `splitChunks: false`, and no `externals` are added, so Rslib's `node` target leaves only Node built-ins (and `pnpapi`) external, and -`AB6005` fails any bare import specifier that is not a Node built-in in every +`AB6005` fails any bare specifier that is not a Node built-in in every compiled module — host-pack modules and the package build's `dist` bundles -alike — so the hatch cannot externalize an import on the author's behalf (a -`require`, `createRequire`, or `import.meta.resolve` call is not an import and -is outside that walk; the prepack gate reads those as dependency evidence). Run-time +alike — whether Rspack emitted it as an ES `import` or as the `require`, +`createRequire(…)` loader, or `import.meta.resolve` call. An ES `import` +external and the `node-commonjs` `createRequire` shim are both rejected; a +direct `require("pkg")` call, however emitted, is rejected the same way, so +the hatch cannot keep a dependency external in any emitted form (one scanner, +`src/build/module-loads.ts`, reads those loads for `AB6005` and for the +prepack gate alike). Run-time path references are kept the same way: a `new URL(…, import.meta.url)` or `new Worker(new URL(…))` in consumer or generated code names a file beside the artifact, so the invariant layer turns the bundler's URL and worker asset diff --git a/packages/agent-bundle/src/build/module-imports.ts b/packages/agent-bundle/src/build/module-imports.ts index d9f9e249c..e06845d47 100644 --- a/packages/agent-bundle/src/build/module-imports.ts +++ b/packages/agent-bundle/src/build/module-imports.ts @@ -1,6 +1,7 @@ import { parse as parseJavaScript } from 'acorn'; import { init, parse } from 'es-module-lexer'; +import { DigestCache } from '../core/digest.ts'; import type { AgentBundleToolsConfig } from '../core/types.ts'; /** @@ -43,22 +44,12 @@ const importKind = (dynamic: number): ModuleImport['kind'] => /** * Imports already read from bytes with a known SHA-256, keyed by check level - * and digest. Within one process the same emitted bundle is scanned by the - * post-compile self-containment check and then by artifact validation, twice - * (before and after the manifest is written); the bytes never change between - * those passes, so the imports of a multi-megabyte bundle are lexed once. - * The records are a few dozen specifiers per module; the map stays bounded. + * and digest (a full parse is a stronger claim than a lex, so each level is + * remembered on its own); see `DigestCache` for why the same bundle is read + * several times per process. The records are a few dozen specifiers per + * module; the cache stays bounded. */ -const importsByDigest = new Map(); -const importsByDigestLimit = 512; - -const remember = (key: string, imports: readonly ModuleImport[]): void => { - if (importsByDigest.size >= importsByDigestLimit) { - const oldest = importsByDigest.keys().next(); - if (!oldest.done) importsByDigest.delete(oldest.value); - } - importsByDigest.set(key, imports); -}; +const importsByDigest = new DigestCache(512); /** * Reads the imports of one ES module source, throwing on invalid syntax @@ -82,6 +73,6 @@ export const readModuleImports = async ( kind: importKind(record.d), specifier: record.n, }))); - if (options.sha256 !== undefined) remember(`${options.check}:${options.sha256}`, imports); + if (options.sha256 !== undefined) importsByDigest.set(`${options.check}:${options.sha256}`, imports); return imports; }; diff --git a/packages/agent-bundle/src/build/module-loads.ts b/packages/agent-bundle/src/build/module-loads.ts new file mode 100644 index 000000000..55ac5e512 --- /dev/null +++ b/packages/agent-bundle/src/build/module-loads.ts @@ -0,0 +1,463 @@ +import { DigestCache } from '../core/digest.ts'; + +/** + * The module loads a JavaScript source makes outside the `import` syntax the + * ES-module lexer reports, for two gates that agree on what a load is while + * disagreeing on what it means. `AB6005` (`validate-artifact-modules.ts`) + * walks every load of every compiled or generated module it validates and + * fails a bare package name, a computed argument, or a loader passed on as a + * value, exactly as it fails the same shapes of `import`. `AB7014` + * (`pack-dependencies.ts`) reads the literal specifiers as evidence that a + * declared dependency is used, and a computed load or a passed-on loader as a + * reason to withhold the finding, since the file may then load a package no + * literal names. + * + * The calls recognised, each a literal load when its argument is one string + * literal — a trailing comma allowed, `require("x",)` — and a computed load + * otherwise (`require(name)`, `require("driver/" + v)`, `require("a", "b")`, + * `require(("x"))`, and a template literal `` require(`x`) ``, static or not): + * `require("x")` and `require.resolve("x")`; a `createRequire(…)` factory + * called at once, `createRequire(…)("x")` and `createRequire(…).resolve("x")`, + * the factory bare, qualified (`Module.createRequire(…)`, + * `ns.default.createRequire(…)`, `require("node:module").createRequire(…)`), + * or under an alias bound by `import { createRequire as mk }` or + * `{ createRequire: mk }`; a loader bound from a factory by `const`, `let`, + * or `var` whose initializer ends with the factory call — `const load = + * createRequire(import.meta.url); load("x")`, `load.resolve("x")`, the shim + * Rspack emits for an external kept as `node-commonjs`, and `const require = + * createRequire(…)` alike, but not `const pad = createRequire(u)("x")`, which + * binds a module and is the factory load itself; and + * `import.meta.resolve("x")`. Whitespace and comments may separate the callee + * from its parentheses and the argument from either parenthesis, and `?.` may + * precede the argument list or `resolve` (`require?.("x")`, + * `load?.resolve("x")`, `import.meta.resolve?.("x")`). A loader or factory + * name is matched whole and + * never after `.`, `#`, or an identifier character — `host.require(…)`, + * `this.#require(…)`, `__webpack_require__(…)`, `require_fast_uri()` are not + * loads — an argument list followed by `{` is a method or function being + * defined (`require(id) {`), not a call, and only `require`, `import.meta`, + * and a `createRequire` factory resolve modules, so `path.resolve("x")`, + * `Promise.resolve("x")`, and `typeof require` never match. `require("")`, + * which Node rejects, reports nothing. + * + * A loader reference is a loader name passed on as a value rather than called: + * after `=`, `(`, `,`, `[`, `{`, `:`, `?`, `|`, `&`, `=>`, or `return` and + * before `;`, `,`, `)`, `]`, `}`, or a line end — `const l = require`, + * `fn(load)`, `[require]`, `{ require }`, `{ key: require }`, `x ? y : require`, + * `return load`, `=> load`, `use({ require })`, `export { require }` — or the + * consequent of a ternary (`x ? require : y`). A binding position introduces + * a name and passes nothing on, so it is not a reference: a name inside a + * parenthesised list that `{` or `=>` follows — a parameter list + * (`function f(module, require) {`, `function (require) {`, `(require) => x`, + * a method `m(require) {`, `catch (require) {`) or the head of an `if`, + * `while`, or `switch`, which merely tests the name — a bare arrow parameter + * (`require => x`), a destructuring pattern declared or assigned + * (`const { require } = host`, `let [a, require] = xs`, `({ require } = host)`, + * `const { a: { require } } = host`), a later declarator (`let a, require;`), + * and an import or re-export specifier list (`import { require } from "./x"`, + * `export { require } from "./x"`). A default initializer is a value, not a + * binding: `function f(x = require) {`, `(x = load) => x`, and + * `const { x = require } = host` are references. The list walk reads a + * parameter list nested two calls deep and a pattern nested one level deep, + * each to about a thousand characters past the name; deeper or longer, the + * name is reported as a value. `export { require as r }` and + * `export default require` are not read. + * + * The scan reads code, not text. One pass over the source treats block and + * line comments, string literals, template literals, and regular-expression + * literals as tokens it steps over, so `require("x")` in a bundled docblock, + * an ajv code template, or an error message is never a load. A template's + * quasis — the text outside its `${…}` substitutions — are text; every + * substitution body is code, scanned with the same names as the source, so + * `` `${require("x")}` `` and `` `${`${require("x")}`}` `` report a literal + * load where the template appears, `` `require("x")` `` reports nothing, and a + * loader name that is a whole substitution (`${require}`) is not a reference. + * A substitution may hold braces two deep and one flat template + * (`${JSON.stringify({ a: { b } })}`); a template nesting deeper is not + * recognised, and its text is scanned as code, quasis included. The + * `createRequire` aliases and bound loaders tracked by name are read from + * the code between those tokens and from substitution bodies, so a binding + * written in a comment or a string binds nothing and one written inside + * `${…}` binds like any other. + * + * Two approximations remain. A `/` is read as a regular-expression literal + * where an operand is expected — after an operator other than `++` and `--`, + * after `(`, `,`, `[`, `{`, `;`, `:`, `?`, a line start, or `return`, + * `typeof`, `case`, `do`, `else`, `in`, `of`, `instanceof`, `new`, `delete`, + * `void`, `throw` — and as division after `)` or an identifier. So + * `count++ / require("x") / d` reports the load, and a regex written directly + * after `)` (`if (x) /require("y")/.test(s)`) is scanned as code: a load + * written inside it is reported, and a quote inside it can misalign the + * string tokens until the next quote on that line. Bindings are tracked by + * name in the three forms above, wherever they appear in the code, which + * assumes the unminified output the framework's bundler emits: a loader + * bound another way — `r = createRequire(u)`, `r ??= createRequire(u)`, + * `const [r] = [createRequire(u)]`, a second-hop alias `const s = r`, a + * loader imported from another module — is not one, and its calls are not + * loads. Out of scope likewise, hand-authored rather than emitted: + * `require.call(…)`, `require.apply(…)`, `Reflect.apply(require, …)`, + * `globalThis.require(…)`, `module.require(…)`, and `import.meta["resolve"](…)`. + */ + +/** The call a load is made through, the shape `AB6005` names in its message. */ +export type ModuleLoadForm = + | 'require' // require("x") + | 'require.resolve' // require.resolve("x") + | 'createRequire' // createRequire(…)("x"), Module.createRequire(…)("x"), mk(…)("x") for an alias mk + | 'createRequire.resolve' // createRequire(…).resolve("x") + | 'bound-loader' // load("x") after const load = createRequire(…) + | 'bound-loader.resolve' // load.resolve("x") after const load = createRequire(…) + | 'import.meta.resolve'; // import.meta.resolve("x") + +interface ModuleLoadSite { + readonly form: ModuleLoadForm; + /** The identifier as written: `require`, the bound name (`load`), the factory or its alias (`createRequire`, `mk`), or `import.meta`. */ + readonly loader: string; +} + +/** A load whose argument is one string literal; `specifier` is the decoded value (`"\x6ceft-pad"` is `left-pad`). */ +export interface LiteralModuleLoad extends ModuleLoadSite { + readonly kind: 'literal'; + readonly specifier: string; +} + +/** A load whose argument is not a single string literal: `require(name)`, `require("driver/" + v)`, a template literal. */ +export interface ComputedModuleLoad extends ModuleLoadSite { + readonly kind: 'computed'; +} + +/** + * A loader passed on as a value rather than called — `const l = require`, + * `fn(load)`, `[require]`, `{ require }`, `{ key: require }`, `x ? require : y`, + * `return load`, `=> load`, `export { require }` — after which packages may be + * loaded under a name the scan never sees. A binding position (a parameter, + * a destructuring pattern, an import specifier) is not one. + */ +export interface LoaderReference { + readonly kind: 'reference'; + readonly form: 'require' | 'bound-loader'; + readonly loader: string; +} + +export type ModuleLoad = LiteralModuleLoad | ComputedModuleLoad | LoaderReference; + +const identifier = String.raw`[A-Za-z_$][\w$]*`; + +/** + * A parenthesised argument list with calls nested up to two deep — + * `(new URL("./entry.js", import.meta.url))`, `(join(dirname(x), "y"))` — the + * shapes a `createRequire` argument takes. + */ +const nestedArguments = (() => { + const flat = String.raw`[(][^()]*[)]`; + return String.raw`[(](?:[^()]|${flat})*[)]`; +})(); +/** The rest of an argument list whose `(` was consumed: up to and including its `)`. */ +const argumentsRest = String.raw`(?:[^()]|${nestedArguments})*[)]`; +const callArguments = String.raw`[(]${argumentsRest}`; + +// Whitespace and comments, the trivia JavaScript allows around a call's parentheses: `require /* x */ ("y")`. +const trivia = String.raw`(?:\s|/\*[\s\S]*?\*/|//[^\n]*\n)*`; +// An optional-chaining `?.` before an argument list: `require?.("x")`, `createRequire(…)?.("x")`. +const optionalCall = String.raw`(?:\?\.${trivia})?`; +// The `.resolve` member of a loader or a factory call, `?.resolve` included. +const resolveMember = String.raw`\s*\??\.\s*resolve`; + +/* + * The tokens the scan steps over, each matched whole where it starts: a block + * or line comment; a single- or double-quoted string, a backslash escaping any + * character, a newline included; a template literal, whose `${…}` substitutions + * may hold one flat template and up to two levels of braces + * (`${JSON.stringify({ a })}`, `${xs.map((x) => { if (x) { … } })}`) — deeper + * nesting leaves the template unrecognised and its text scanned as code; and a + * regular-expression literal where an operand is expected. + */ +const blockComment = String.raw`/\*[\s\S]*?\*/`; +const lineComment = String.raw`//[^\n]*`; +const doubleQuoted = String.raw`"(?:[^"\\\n]|\\[\s\S])*"`; +const singleQuoted = String.raw`'(?:[^'\\\n]|\\[\s\S])*'`; +const flatTemplate = String.raw`\x60(?:[^\x60\\]|\\[\s\S])*\x60`; +const substitutionBraces = String.raw`\{(?:[^{}\x60]|\{[^{}\x60]*\})*\}`; +/** The code of one `${…}` substitution: anything but a brace or a backtick, braces two deep, one flat template. */ +const substitutionBody = String.raw`(?:[^{}\x60]|${substitutionBraces}|${flatTemplate})*`; +const templateLiteral = String.raw`\x60(?:[^\x60\\$]|\\[\s\S]|\$(?!\{)|\$\{${substitutionBody}\})*\x60`; +// A `/` where an operand is expected starts a regular-expression literal, never a division. After `++` or `--` it is +// the division the operator's operand takes part in (`count++ / require("x")`). The `/` is matched before the +// lookbehind that reads what precedes it: an alternative that opens with a literal character lets the engine +// dispatch on that character, one that opens with a lookbehind runs the lookbehind at every position. +const regexLiteral = String.raw`/(?<=(?:^|[\n(,=:[!&|?{};+\-*%<>~^])\s*(?${blockComment}|${lineComment}|${doubleQuoted}|${singleQuoted}|${templateLiteral}|${regexLiteral})`; +/** The tokens alone, for blanking them out of a source before its binding names are read. */ +const skippedTokens = new RegExp(skippedToken, 'gu'); +/** + * The substitutions of a recognised template literal, each body captured; a + * backslash escapes the character after it, so `\${` is text. The body grammar + * is `templateLiteral`'s own, and reads the same text the same way. + */ +const templateSubstitution = new RegExp(String.raw`\\[\s\S]|\$\{(?${substitutionBody})\}`, 'gu'); + +/** + * A single- or double-quoted string literal: `quote` is the opening quote and + * `body` the text between the quotes, escapes as written (`decodeLiteral` + * reads the value). A fragment for callers' own expressions — + * `declarationSpecifiers` in `pack-dependencies.ts` — whose groups are also + * numbered from the fragment's position: 1 and 2 when it opens the expression. + */ +export const quotedLiteral = String.raw`(?["'])(?(?:(?!\k)[^\\\n]|\\.)+)\k`; + +const escapeSequence = /\\(?:x(?[0-9A-Fa-f]{2})|u\{(?[0-9A-Fa-f]+)\}|u(?[0-9A-Fa-f]{4})|(?.))/gsu; +const controlEscapes: Readonly> = { 0: '\0', b: '\b', f: '\f', n: '\n', r: '\r', t: '\t', v: '\v' }; + +/** + * The string a JavaScript literal's body denotes: `\x66oo` is `foo`, + * `foo\u002fsubpath` is `foo/subpath`, `\/` is `/`. Node resolves the value, + * not the source text, so a package name compared textually has to be + * decoded first. + */ +export const decodeLiteral = (body: string): string => body.replace(escapeSequence, (...args) => { + const groups = args.at(-1) as Record; + if (groups.hex !== undefined) return String.fromCharCode(Number.parseInt(groups.hex, 16)); + if (groups.unit !== undefined) return String.fromCharCode(Number.parseInt(groups.unit, 16)); + if (groups.point !== undefined) { + const point = Number.parseInt(groups.point, 16); + // Beyond Unicode the literal is a syntax error; the file never loads anything. + return point > 0x10_ff_ff ? '' : String.fromCodePoint(point); + } + const other = groups.other ?? ''; + return controlEscapes[other] ?? other; +}); + +const escapeIdentifier = (name: string): string => name.replace(/\$/gu, String.raw`\$`); + +/** + * `createRequire` renamed on import or destructuring: `import { createRequire + * as makeRequire } from "node:module"` or `const { createRequire: makeRequire } + * = require("node:module")`. Each alias is a factory like `createRequire` itself. + */ +const createRequireAlias = /\bcreateRequire\s*(?:as|:)\s*([A-Za-z_$][\w$]*)/gu; + +const factoryNames = (code: string): readonly string[] => [...new Set([ + 'createRequire', + ...Array.from(code.matchAll(createRequireAlias), (match) => escapeIdentifier(match[1] ?? '')), +])]; + +/** + * What may qualify a factory in a loader binding: nothing (`createRequire(…)` + * after a named import), a dotted namespace (`Module.createRequire(…)` after + * `import * as Module from "node:module"`, `ns.default.createRequire(…)`), or a + * CommonJS load (`require("node:module").createRequire(…)`, + * `require('module')…`; any argument). No capture group: `loaderBinding` + * counts its own by number. + */ +const factoryQualifier = String.raw`(?:(?:${identifier}\s*\.\s*)*|\brequire\s*${callArguments}\s*\.\s*)`; + +/** + * `const load = (…)` — `let` and `var` too — the factory qualified as + * `factoryQualifier` allows or not: the binding is a loader, called like + * `require` from then on. The factory call must end the initializer: with a + * call, a member, an index, or `?.` after it — `const pad = + * createRequire(u)("left-pad")`, `const where = createRequire(u).resolve(…)` — + * the binding holds a module or a path, not a loader, and the load is the + * factory call itself, reported where it stands. + */ +const loaderBinding = (factories: readonly string[]): RegExp => new RegExp( + String.raw`\b(?:const|let|var)\s+(${identifier})\s*=\s*${factoryQualifier}(? [...new Set([ + 'require', + ...Array.from(code.matchAll(loaderBinding(factories)), (match) => escapeIdentifier(match[1] ?? '')), +])]; + +interface BindingNames { + readonly factories: readonly string[]; + readonly loaders: readonly string[]; +} + +const backtick = '\x60'; +/** Whether a template literal token has substitutions to scan: one with none is text throughout. */ +const hasSubstitution = (template: string): boolean => template.startsWith(backtick) && template.includes('${'); + +/** + * The code of a source with every token the scan steps over — comment, string, + * template text, regular expression — replaced by one space, and the `${…}` + * substitution bodies of a template kept as the code they are, projected the + * same way in turn. + */ +const codeProjection = (source: string): string => source.replace(skippedTokens, (token) => + (hasSubstitution(token) ? codeProjection(substitutionBodies(token)) : ' ')); + +/** The substitution bodies of one template literal token, in order, one per line. */ +const substitutionBodies = (template: string): string => Array.from( + template.matchAll(templateSubstitution), + (match) => match.groups?.substitution ?? '', +).join('\n'); + +/** + * The factory and loader names of a source, read from its code alone + * (`codeProjection`), so a `const load = createRequire(…)` in a comment or a + * string binds nothing, one inside a template substitution binds like any + * other, and `require("node:module").createRequire(…)` still reads as the + * qualified factory it is. Without the text `createRequire` anywhere there is + * no alias and no bound loader, and the token pass is spared. + */ +const bindingNames = (source: string): BindingNames => { + if (!source.includes('createRequire')) return { factories: ['createRequire'], loaders: ['require'] }; + const code = codeProjection(source); + const factories = factoryNames(code); + return { factories, loaders: loaderNames(code, factories) }; +}; + +/* + * The positions that bind a loader name rather than pass it on. Each is read + * from the name outwards, over the rest of its enclosing list: `parameterList` + * to the `)` — past calls nested two deep, never past `;` — and then `=>` or + * `{`, which makes the list a parameter list, a `catch` clause, or an `if`, + * `while`, or `switch` head; `patternTail` to the `}` or `]` of a pattern — + * past one nested pattern, one inner close — and then `=` (not `==`, `=>`), + * a destructuring assignment, or `from`, an import or re-export specifier + * list; `declaredPattern`, read backwards, to the `{` or `[` a `const`, `let`, + * or `var` opens, past one inner open; `declaratorList`, read backwards, to + * the `,` of a declaration whose earlier declarators sit on the same + * statement (`let a, require;`). + */ +/** + * How far a list walk reads from a loader name: a parameter list, pattern, or + * declarator list in emitted code is a few hundred characters at most, and a + * walk to the end of every list from every loader name in it is quadratic in + * a list that names a loader thousands of times. Past the bound the name is + * reported as a value. + */ +const listReach = 1024; +const parameterList = String.raw`(?:[^();]|${nestedArguments}){0,${listReach}}[)]\s*(?:=>|\{)`; +const patternContent = String.raw`(?:[^{}[\]();]|\{[^{}[\]();]{0,${listReach}}\}|\[[^{}[\]();]{0,${listReach}}\]){0,${listReach}}`; +const patternTail = String.raw`${patternContent}(?:[}\]]${patternContent})?[}\]]\s*(?:=(?![=>])|from\b)`; +const declaredPattern = String.raw`\b(?:const|let|var)\s*[{[]${patternContent}(?:[{[]${patternContent})?`; +const declaratorList = String.raw`\b(?:const|let|var)\s+[^;{}()[\]]{0,${listReach}},\s*`; + +/** + * One pass over a source: the tokens the scan steps over, then the calls and + * loader references it reports, each alternative anchored so that it matches + * where JavaScript would read a call or a value and nowhere else. + */ +const loadScanner = (loaders: readonly string[], factories: readonly string[]): RegExp => { + // A loader name whole: not after `.` (a member, `host.require`), `#` (a private name), or an identifier character + // (`__webpack_require__`), and not before one (`require_fast_uri`). A factory may follow `.`: that is the qualified + // form, `Module.createRequire(…)` or `require("node:module").createRequire(…)`, whose qualifier needs no matching. + const loaderCall = String.raw`(?${loaders.join('|')})(?![\w$])(?${resolveMember})?`; + const metaCall = String.raw`(?import\s*\.\s*meta${resolveMember})`; + const factoryCall = String.raw`(?${factories.join('|')})(?![\w$])${trivia}${optionalCall}${callArguments}(?${resolveMember})?`; + const call = String.raw`(?:${loaderCall}|${metaCall}|${factoryCall})${trivia}${optionalCall}[(]${trivia}`; + const literal = String.raw`${quotedLiteral}${trivia}(?:,${trivia})?[)]`; + // Anything but a lone literal: a template literal whole (its substitutions are code, read like any template's), + // an identifier, an operator after a literal (`"driver/" + v`). A comment is trivia the call prefix already + // consumed, not the start of a computed argument. + const computed = String.raw`(?${templateLiteral}|(?!/[*/])[^"'\s)]|"[^"\n]*"${trivia}(?!/[*/])[^)\s]|'[^'\n]*'${trivia}(?!/[*/])[^)\s])`; + // `require(id) {` is a method or function named `require` being defined, not a call: the lookahead sees the block + // after a balanced argument list. An argument list nesting deeper than `argumentsRest` reads is a computed call. + const notDefinition = String.raw`(?!${argumentsRest}\s*\{)`; + // A loader passed on as a value: after `=`, `(`, `,`, `[`, `{`, `:`, `?`, `|`, `&`, `=>`, or `return`, and before + // `;`, `,`, `)`, `]`, `}`, or a line end (`fn(require)`, `{ key: require }`, `x ? y : require`, `return load`); or + // the consequent of a ternary (`x ? require : y`). A `:` after the name makes it an object key (`{ require: x }`), + // not the loader, unless a `?` precedes it. A binding position — a parameter, a declared or assigned pattern, a + // later declarator, an import specifier — is excluded by the list it sits in. The name is matched first, whole, + // and every lookbehind reads back over it: as with `regexLiteral`, the engine then dispatches on the name's first + // character, and the list walks run only where a loader name stands after an operator — a walk at every position + // after an operator is quadratic in a stretch of source without brackets. + const names = loaders.join('|'); + // After `=` the name is an initializer or an assignment's right-hand side — `const l = require`, `x = load`, + // and a default, `function f(x = require) {`, `const { x = require } = host` — a value wherever the list is. + const value = String.raw`(?${names})(?![\w$])(?=\s*(?:[;,)\]}]|\n|$))(?:(?<=(?])=\s*(?:${names}))|(?<=(?:=>|\breturn|[(,[{:?|&])\s*(?:${names}))(?${names})(?![\w$])(?=\s*:)(?<=\?\s*(?:${names}))`; + return new RegExp(`${skippedToken}|${call}(?:${literal}|${notDefinition}${computed})|${value}|${consequent}`, 'gu'); +}; + +/** The named groups of `loadScanner`; each match sets those of one alternative. */ +interface ScanGroups { + readonly skipped?: string; + readonly loader?: string; + readonly loaderResolve?: string; + readonly meta?: string; + readonly factory?: string; + readonly factoryResolve?: string; + readonly quote?: string; + readonly body?: string; + readonly computed?: string; + readonly reference?: string; + readonly ternary?: string; +} + +const loadForm = (groups: ScanGroups): ModuleLoadForm => { + if (groups.meta !== undefined) return 'import.meta.resolve'; + if (groups.factory !== undefined) return groups.factoryResolve === undefined ? 'createRequire' : 'createRequire.resolve'; + if (groups.loader === 'require') return groups.loaderResolve === undefined ? 'require' : 'require.resolve'; + return groups.loaderResolve === undefined ? 'bound-loader' : 'bound-loader.resolve'; +}; + +/** + * The loads and loader references of one stretch of code, appended in source + * order. A template literal token — stepped over or a computed argument — has + * each substitution body scanned as code in turn, by the same scanner, where + * the template appears. + */ +const scanCode = (code: string, scanner: RegExp, loads: ModuleLoad[]): void => { + for (const match of code.matchAll(scanner)) { + const groups: ScanGroups = match.groups ?? {}; + if (groups.skipped !== undefined) { + if (hasSubstitution(groups.skipped)) scanTemplate(groups.skipped, scanner, loads); + continue; + } + const referenced = groups.reference ?? groups.ternary; + if (referenced !== undefined) { + loads.push(Object.freeze({ form: referenced === 'require' ? 'require' : 'bound-loader', kind: 'reference', loader: referenced })); + continue; + } + const site: ModuleLoadSite = { form: loadForm(groups), loader: groups.loader ?? groups.factory ?? 'import.meta' }; + if (groups.computed === undefined) { + loads.push(Object.freeze({ ...site, kind: 'literal', specifier: decodeLiteral(groups.body ?? '') })); + continue; + } + loads.push(Object.freeze({ ...site, kind: 'computed' })); + // A template argument matched whole; a lone backtick is one the template grammar did not recognise. + if (groups.computed.length > 1 && hasSubstitution(groups.computed)) scanTemplate(groups.computed, scanner, loads); + } +}; + +const scanTemplate = (template: string, scanner: RegExp, loads: ModuleLoad[]): void => { + for (const match of template.matchAll(templateSubstitution)) { + const body = match.groups?.substitution; + if (body !== undefined) scanCode(body, scanner, loads); + } +}; + +/** Loads already read from bytes with a known SHA-256; see `DigestCache`. */ +const loadsByDigest = new DigestCache(512); + +/** + * Every load and loader reference of one JavaScript source, in source order. + * Synchronous and total: a source the scan cannot follow yields the loads it + * could read, never an error — syntax is another gate's concern. When the + * source's SHA-256 is known, a result remembered for those bytes is returned + * as is, and a fresh scan is remembered for the next pass over the same + * bytes; the result is frozen either way. + */ +export const scanModuleLoads = (source: string, options?: { readonly sha256?: string }): readonly ModuleLoad[] => { + const sha256 = options?.sha256; + if (sha256 !== undefined) { + const known = loadsByDigest.get(sha256); + if (known !== undefined) return known; + } + const { factories, loaders } = bindingNames(source); + const loads: ModuleLoad[] = []; + scanCode(source, loadScanner(loaders, factories), loads); + const frozen = Object.freeze(loads); + if (sha256 !== undefined) loadsByDigest.set(sha256, frozen); + return frozen; +}; diff --git a/packages/agent-bundle/src/build/pack-dependencies.ts b/packages/agent-bundle/src/build/pack-dependencies.ts index ebab0b70c..b7cd90bdc 100644 --- a/packages/agent-bundle/src/build/pack-dependencies.ts +++ b/packages/agent-bundle/src/build/pack-dependencies.ts @@ -9,23 +9,25 @@ import { sha256Hex } from '../core/digest.ts'; import { isErrno } from '../core/errors.ts'; import { isRecord } from '../core/strict-json.ts'; import { readModuleImports, type ModuleImport } from './module-imports.ts'; +import { decodeLiteral, quotedLiteral, scanModuleLoads } from './module-loads.ts'; /** * Evidence for the npm prepack dependency gate (`AB7014`/`AB7015`, emitted by * `pack-inventory.ts`): what `package.json` asks npm to install alongside the * package, and which packages the packed JavaScript and declaration files - * actually reference. JavaScript the framework compiled — the `dist` bundles - * and the host-pack modules — never carries a bare package `import`, static - * or dynamic, since `AB6005` fails the build first and `prepack` builds - * before it packs; the import evidence read here is therefore that of - * prebuilt payload modules and other packed scripts the framework copied - * rather than compiled. The `require`, `createRequire`, and - * `import.meta.resolve` evidence is different: those are calls the bundler - * leaves in place and `AB6005` does not walk, so they are read from every - * packed file, compiled bundles included — as are the `bin`-command, - * declaration, `imports`-map, and install-script evidence (a compiled bundle - * may run a dependency's command, `spawnSync("tsc")`, which is not an - * import). + * actually reference. The inventory lexes every packed JavaScript file + * (`importedPackageNames`, `dist/**` included). Because `AB6005` has already + * refused a bare load in every walked emitted module before the inventory + * runs (`prepack` builds first), the evidence that can still keep a + * dependency in a build that passed comes from a prebuilt payload module, + * packed JavaScript the `files` allowlist adds from outside the artifact + * and `dist`, a packed declaration reference, an install script, or a `bin` + * command. `AB6005` walks a compiled module's `import` records, static and + * dynamic, and its literal and computed `require`, `require.resolve`, + * `createRequire(…)` (direct or bound), `.resolve`, and `import.meta.resolve` + * loads alike (`module-loads.ts`, the scanner both gates share). A compiled + * bundle may still run a dependency's command (`spawnSync("tsc")`), which + * is not an import. */ /** @@ -290,154 +292,6 @@ export const packagedSourceInstallable = async ( return tarHoldsPackage(archive); }; -/** A single- or double-quoted string literal; the group after the opening quote is its body. */ -const quotedLiteral = String.raw`(["'])((?:(?!\1)[^\\\n]|\\.)+)\1`; - -const escapeSequence = /\\(?:x(?[0-9A-Fa-f]{2})|u\{(?[0-9A-Fa-f]+)\}|u(?[0-9A-Fa-f]{4})|(?.))/gsu; -const controlEscapes: Readonly> = { 0: '\0', b: '\b', f: '\f', n: '\n', r: '\r', t: '\t', v: '\v' }; - -/** - * The string a JavaScript literal's body denotes: `\x66oo` is `foo`, - * `foo\u002fsubpath` is `foo/subpath`, `\/` is `/`. Node resolves the value, - * not the source text, so a package name compared textually has to be - * decoded first. - */ -const decodeLiteral = (body: string): string => body.replace(escapeSequence, (...args) => { - const groups = args.at(-1) as Record; - if (groups.hex !== undefined) return String.fromCharCode(Number.parseInt(groups.hex, 16)); - if (groups.unit !== undefined) return String.fromCharCode(Number.parseInt(groups.unit, 16)); - if (groups.point !== undefined) { - const point = Number.parseInt(groups.point, 16); - // Beyond Unicode the literal is a syntax error; the file never loads anything. - return point > 0x10_ff_ff ? '' : String.fromCodePoint(point); - } - const other = groups.other ?? ''; - return controlEscapes[other] ?? other; -}); - -/** - * A `require("…")` call, or a resolution-only use — `require.resolve("…")`, - * `createRequire(…).resolve("…")`, `import.meta.resolve("…")` — with a - * literal argument. The ESM lexer reports `import` forms only; CommonJS - * payloads a consumer prebuilt reach the package through `require`, and a - * package located only to find an asset or executable is still a runtime - * dependency. Only these resolvers count: `path.resolve("foo")` or - * `Promise.resolve("foo")` never make an unused `foo` look reachable. A match - * inside a comment or string can only mark a dependency as imported, never as - * unused, so the pattern otherwise errs toward keeping a declaration. - */ -/** - * A parenthesised argument list with calls nested up to two deep — - * `(new URL("./entry.js", import.meta.url))`, `(join(dirname(x), "y"))` — the - * shapes a `createRequire` argument takes. - */ -const callArguments = (() => { - const flat = String.raw`[(][^()]*[)]`; - const nested = String.raw`[(](?:[^()]|${flat})*[)]`; - return String.raw`[(](?:[^()]|${nested})*[)]`; -})(); - -// Whitespace and comments, the trivia JavaScript allows around a call's parentheses: `require /* x */ ("y")`. -const trivia = String.raw`(?:\s|/\*[\s\S]*?\*/|//[^\n]*\n)*`; - -/** - * What may qualify a factory: nothing (`createRequire(…)` after a named - * import), a dotted namespace (`Module.createRequire(…)` after `import * as - * Module from "node:module"`, `module.createRequire(…)`), or a CommonJS load - * (`require("node:module").createRequire(…)`, `require('module')…`; any - * argument, since a same-named factory from elsewhere can only keep a - * declaration). No capture group: the literal patterns after it count theirs - * by number. - */ -const factoryQualifier = String.raw`(?:(?:[A-Za-z_$][\w$]*\s*\.\s*)*|\brequire\s*${callArguments}\s*\.\s*)`; - -/** A factory call producing a loader, qualified or not: `createRequire(import.meta.url)`, `Module.createRequire(…)`, `require("node:module").createRequire(…)`. */ -const factoryCall = (factories: readonly string[]): string => - String.raw`${factoryQualifier}\b(?:${factories.join('|')})${trivia}${callArguments}`; - -/** The resolvers a file loads packages through, each followed by its argument list. */ -const loadCall = (loaders: readonly string[], factories: readonly string[]): string => - String.raw`(?:\b(?:${loaders.join('|')})(?:\.resolve)?|\bimport\.meta\.resolve|${factoryCall(factories)}(?:\.resolve)?)${trivia}[(]${trivia}`; - -const literalLoad = (loaders: readonly string[], factories: readonly string[]): RegExp => new RegExp( - String.raw`${loadCall(loaders, factories)}${quotedLiteral}${trivia}[)]`, - 'gu', -); - -/** - * A CommonJS load or resolution whose argument is not a string literal — - * `require(x)`, `require.resolve(x)`, `import.meta.resolve(x)`, or a direct - * `createRequire(…)(x)` — selecting a package at runtime, which no literal can - * prove. An argument that merely starts with a literal, `require("driver/" + - * variant)`, is computed too. Bundler runtimes (`__webpack_require__(…)`) have - * no word boundary before `require` and never match; `path.resolve(x)` and - * `Promise.resolve(x)` are not resolution and never match. - */ -const computedLoad = (loaders: readonly string[], factories: readonly string[]): RegExp => new RegExp( - // A comment is trivia the call prefix already consumed, not the start of a computed argument. - String.raw`${loadCall(loaders, factories)}(?:(?!/[*/])[^"'\s)]|"[^"\n]*"${trivia}(?!/[*/])[^)\s]|'[^'\n]*'${trivia}(?!/[*/])[^)\s])`, - 'u', -); - -const escapeIdentifier = (name: string): string => name.replace(/\$/gu, String.raw`\$`); - -/** - * `createRequire` renamed on import or destructuring: `import { createRequire - * as makeRequire } from "node:module"` or `const { createRequire: makeRequire } - * = require("node:module")`. Each alias is a factory like `createRequire` itself. - */ -const createRequireAlias = /\bcreateRequire\s*(?:as|:)\s*([A-Za-z_$][\w$]*)/gu; - -const factoryNames = (source: string): readonly string[] => [ - 'createRequire', - ...Array.from(source.matchAll(createRequireAlias), (match) => escapeIdentifier(match[1] ?? '')), -]; - -/** - * `const load = (…)`, the factory qualified as `factoryQualifier` - * allows or not: the binding is a loader, called like `require` from then on. - */ -const loaderBinding = (factories: readonly string[]): RegExp => new RegExp( - String.raw`\b(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*=\s*${factoryQualifier}\b(?:${factories.join('|')})\s*[(]`, - 'gu', -); - -/** - * The identifiers a file loads packages through: `require` itself plus every - * name bound to a `createRequire(…)` result — under the factory's own name or - * an alias — so `const load = createRequire(import.meta.url); load("driver")` - * counts like `require("driver")`. - */ -const loaderNames = (source: string): readonly string[] => [ - 'require', - ...Array.from(source.matchAll(loaderBinding(factoryNames(source))), (match) => escapeIdentifier(match[1] ?? '')), -]; - -/** - * JavaScript comments and string literals, each replaced by a space: the - * text that is not code. Bundled docblocks are prose ("may fail, require - * Effect services"), and a scan for a bare identifier has to skip them. - * Regular-expression literals are not recognised; one containing a quote - * can misalign the strings after it on the same line, which at worst hides - * or invents a bare reference there. - */ -const codeOnly = (source: string): string => - source.replace(/\/\*[\s\S]*?\*\/|\/\/[^\n]*|"(?:[^"\\\n]|\\.)*"|'(?:[^'\\\n]|\\.)*'|`(?:[^`\\]|\\[\s\S])*`/gu, ' '); - -/** - * A loader passed on as a value rather than called — `const load = require`, - * `fn(require)`, `[require]`, `{ require }`, `module.exports = require`, - * `return require`, `x ? require : y` — after which packages may be loaded - * under a name this scan never sees, so the file's evidence is incomplete - * like a computed load's. A call (`require("x")`), a property access - * (`require.resolve`), and `typeof require` pass nothing on and never match. - * Run on `codeOnly` text, so a mention in a comment or string is not one. - */ -const loaderReference = (loaders: readonly string[]): RegExp => new RegExp( - String.raw`(?:=>|\breturn|[=(,[{:?|&])\s*\b(?:${loaders.join('|')})\b\s*(?=[;,)\]}:]|$)`, - 'mu', -); - /** * Every module specifier a declaration file resolves: `from "…"`, * `import("…")`, `import x = require("…")`, `declare module "…"` (an @@ -445,7 +299,9 @@ const loaderReference = (loaders: readonly string[]): RegExp => new RegExp( * and `/// `. * A consumer needs the package that provides these types even though the * bundled JavaScript has no runtime import. Declarations are not ES modules - * the lexer accepts, so this is a text scan with the same keep-only bias. + * the lexer accepts, so this is a plain text scan, comments included — a + * keep-only bias it can afford, since a match inside a doc comment at worst + * keeps a declaration and never reports one. */ const declarationSpecifier = new RegExp( String.raw`\b(?:from|import|require|declare\s+module)\s*\(?\s*${quotedLiteral}| text.replace(/[.*+?^${}()|[\]\\]/ const commandLiteral = (command: string): RegExp => new RegExp(String.raw`(["'\x60])${escapeRegExp(command)}(?:\s[^"'\x60\n]*)?\1`, 'u'); -/** - * The module specifiers packed JavaScript resolves: the lexer's static and - * dynamic literal imports — never a mention inside a comment or string, - * which bundled library docblocks are full of — plus literal `require` calls; - * and the dependencies it runs by one of their `bin` commands. - */ /** * The module specifiers JavaScript source loads — the lexer's static and - * dynamic literal imports, plus literal `require`/`createRequire` calls — - * and whether that is all of them: a computed `import(x)` or `require(x)`, - * or a loader passed on as a value, means it is not — and so does source the - * lexer rejects, whose `import()` calls it could not report (syntax itself is - * another gate's concern). Packed files and inline `node -e` programs are - * read alike. + * dynamic literal imports, then the literal `require`, `require.resolve`, + * `createRequire(…)` (direct or bound), `.resolve`, and `import.meta.resolve` + * loads `scanModuleLoads` reports, in source order — and whether that is all + * of them: a computed `import(x)`, a computed recognised load (non-literal + * argument; optional `?.(` and a trailing comma count the same), or a loader + * passed on as a value (argument, array element, object-literal value, + * ternary branch, return/arrow value, assignment right-hand side, export — + * not a binding position) means it is not — and so does source the lexer + * rejects, whose `import()` calls it could not report (syntax itself is + * another gate's concern). The scanner steps over comments, string literals, + * regex literals in operand position, and template static text; a load + * inside a template `${…}` substitution is scanned as code. Packed files + * and inline `node -e` programs are read alike. */ const moduleLoads = async (source: string, sha256?: string): Promise> => { let imports: readonly ModuleImport[]; @@ -530,16 +387,14 @@ const moduleLoads = async (source: string, sha256?: string): Promise record.kind !== 'dynamic' || record.specifier !== undefined) - && !computedLoad(loaders, factories).test(source) - && !loaderReference(loaders).test(codeOnly(source)), + && loads.every((load) => load.kind === 'literal'), specifiers: [ ...imports.flatMap((record) => (record.specifier === undefined ? [] : [record.specifier])), - ...Array.from(source.matchAll(literalLoad(loaders, factories)), (match) => decodeLiteral(match[2] ?? '')), + ...loads.flatMap((load) => (load.kind === 'literal' ? [load.specifier] : [])), ], }; }; diff --git a/packages/agent-bundle/src/build/pack-inventory.ts b/packages/agent-bundle/src/build/pack-inventory.ts index b358ac88a..ad48ce3ec 100644 --- a/packages/agent-bundle/src/build/pack-inventory.ts +++ b/packages/agent-bundle/src/build/pack-inventory.ts @@ -143,16 +143,21 @@ const perField = ( * host packs, so an installed-dependency entry no packed file references * only makes every consumer's `npm install` fetch a build-time package — and * fail outright when the specifier is one a consumer's npm cannot resolve - * (git, remote tarball, path, or an unrewritten workspace protocol). A - * compiled bundle cannot `import` a bare package at all: `AB6005` fails the - * build on any import specifier that is not a Node built-in, and `prepack` - * runs that build before this inventory, so the import evidence `AB7014` - * accepts comes only from modules the framework copied rather than compiled - * — prebuilt payload modules and other scripts the `files` allowlist packs — - * never from a `dist` bundle or a host-pack module. A `require`, - * `createRequire(…)(…)`, or `import.meta.resolve(…)` call is not an import - * and `AB6005` does not walk it, so that evidence is read from every packed - * file, compiled bundles included. + * (git, remote tarball, path, or an unrewritten workspace protocol). An + * emitted module cannot load a bare package at all: `AB6005` walks every + * `.js`/`.mjs` module of a host pack and of `dist` — compiled bundles and + * copied scripts alike, prebuilt payload modules excepted — and fails any + * `import`, `require`, `require.resolve`, `createRequire(…)` loader call + * (direct or bound), `.resolve`, or `import.meta.resolve` whose specifier + * is not a Node built-in or a listed file inside the tree, and `prepack` + * runs that build before this inventory. The inventory lexes every packed + * JavaScript file (`importedPackageNames`, `dist/**` included); because + * `AB6005` has already refused a bare load in every walked emitted module + * before the inventory runs, the evidence that can still keep a dependency + * in a build that passed comes from a prebuilt payload module, packed + * JavaScript the `files` allowlist adds from outside the artifact and + * `dist`, a packed declaration reference, an install script, or a `bin` + * command. One scanner (`module-loads.ts`) reads the loads for both gates. */ const unresolvableMessage = (field: InstalledDependencyField, own: readonly DeclaredDependency[]): string => `package.json ${field} names packages a consumer's npm cannot resolve through a registry (an invalid name or a non-registry specifier): ${own.map((dependency) => @@ -170,9 +175,12 @@ const dependencyDiagnostics = async (options: { }): Promise => { const declared = declaredDependencies(options.packageDocument); if (declared.length === 0) return []; - // `prepack` runs the build before this inventory, and `AB6005` there refuses every bare import in a compiled - // bundle, so any `import` evidence found here belongs to a packed module the framework did not compile; the - // `require`/`createRequire`/`import.meta.resolve` evidence is not an import and may come from any packed file. + // The inventory lexes every packed JavaScript file. `prepack` runs the build first, and `AB6005` there + // refuses every bare import, `require`, `require.resolve`, `createRequire(…)` load (direct or bound), + // `.resolve`, and `import.meta.resolve` in a walked emitted module, so the evidence that can still keep + // a dependency in a build that passed comes from a prebuilt payload module, packed JavaScript the + // `files` allowlist adds from outside the artifact and `dist`, a packed declaration reference, an + // install script, or a `bin` command. const imported = await importedPackageNames({ declared: declared.filter((dependency) => dependency.installed).map((dependency) => dependency.name), packageDocument: options.packageDocument, @@ -225,7 +233,7 @@ const dependencyDiagnostics = async (options: { : 'Every consumer installs them for nothing; the emitted outputs already inline what they use.'), field === 'peerDependencies' ? 'Keep a deliberate compatibility peer, mark it optional in peerDependenciesMeta so npm stops installing it, or move a build-only package to devDependencies.' - : 'Move build-only packages to devDependencies; compiled bundles inline their imports (AB6005), so keep a runtime dependency only for what a prebuilt payload or other uncompiled packed module imports, a packed file requires or resolves (createRequire, import.meta.resolve), a packed declaration file references, a #subpath import reaches through the imports map, or an install script or packed JavaScript runs; a computed import() or require() in packed code withholds this check.', + : 'Move build-only packages to devDependencies; the build inlines every dependency into emitted modules (AB6005), so keep a runtime dependency only for what a prebuilt payload module, packed JavaScript the files allowlist adds from outside the artifact and dist, a packed declaration file, a #subpath import through the imports map, an install script, or a bin command still references, loads, or runs; a non-literal import(), require(), require.resolve(), createRequire()() or .resolve(), or import.meta.resolve(), or a loader passed on as a value, makes a packed file\'s evidence incomplete and withholds this check.', field === 'peerDependencies' ? 'warning' : 'error', )), // npm skips an optional dependency it cannot fetch, so the install survives — but only once the specifier parsed diff --git a/packages/agent-bundle/src/build/package-build.ts b/packages/agent-bundle/src/build/package-build.ts index 6ee8ab68d..b36b942da 100644 --- a/packages/agent-bundle/src/build/package-build.ts +++ b/packages/agent-bundle/src/build/package-build.ts @@ -371,15 +371,16 @@ export const buildPackageOutputs = async (options: { // The npm form of the plugin is held to the same line as its host packs: // every emitted `dist` module is walked as an ES module, and a bare - // specifier that is not a Node built-in — an import the `tools` hatch - // kept external — fails the build (`AB6005`) before `dist` is published, - // so a `dist/bin` executable imports nothing from a consumer's - // `node_modules`. The walk reads import specifiers (static and literal - // dynamic); a `createRequire(…)(…)` or `import.meta.resolve(…)` call is - // not an import and is outside it, in `dist` as in a host pack — the - // prepack gate reads those as dependency evidence. Declarations are not - // modules and are not walked; they may still reference declared - // dependencies. + // specifier that is not a Node built-in — a dependency the `tools` hatch + // kept external, whether Rspack emitted it as an `import` or as the + // `require` shim of a `node-commonjs` external — fails the build + // (`AB6005`) before `dist` is published, so a `dist/bin` executable loads + // nothing from a consumer's `node_modules`. The walk reads import + // specifiers (static and literal dynamic) and the module's `require`, + // `createRequire(…)` loader, and `import.meta.resolve` calls, in `dist` + // as in a host pack (`module-loads.ts`, the scanner the prepack gate + // shares). Declarations are not modules and are not walked; they may + // still reference declared dependencies. const selfContainment = await validateJavaScriptModules({ artifactRoot: stageRoot, bundledPaths: new Set(files.filter((file) => file.kind === 'bundle').map((file) => file.path)), diff --git a/packages/agent-bundle/src/build/validate-artifact-modules.ts b/packages/agent-bundle/src/build/validate-artifact-modules.ts index 26ca956c9..99aabd125 100644 --- a/packages/agent-bundle/src/build/validate-artifact-modules.ts +++ b/packages/agent-bundle/src/build/validate-artifact-modules.ts @@ -9,6 +9,31 @@ import { readFileBytes, runWithPlatform } from '../effect/platform.ts'; import { artifactDiagnostic as diagnostic, artifactDiagnosticRecoveries } from './artifact-diagnostics.ts'; import type { ArtifactFile } from './emit.ts'; import { readModuleImports, type ModuleImport, type ModuleSyntaxCheck } from './module-imports.ts'; +import { scanModuleLoads, type ComputedModuleLoad, type LiteralModuleLoad } from './module-loads.ts'; + +/** + * `AB6005`: the JavaScript modules an artifact — or a package build's staged + * `dist` — ships resolve nothing from outside their tree but Node built-ins. + * Every listed `.js`/`.mjs` module is walked, and every `.js`/`.mjs` module + * a walked one reaches: its `import` records as the ES-module lexer reports + * them, static and dynamic, and its recognised loads as `scanModuleLoads` + * reports them — `require(…)`, `require.resolve(…)`, + * `createRequire(…)(…)`/`.resolve(…)` (factory written out, + * namespace-qualified, or aliased), a loader bound by + * `const|let|var name = createRequire(…)` then called, and + * `import.meta.resolve(…)`; optional `?.(` and a trailing comma count the + * same. Compiled bundles and the framework-generated modules parsed in full + * are read the same way; prebuilt payload modules are opaque consumer output + * and are not walked. A relative or `file:` specifier, imported or loaded, + * must name a listed regular `.js`/`.mjs` module inside the tree, which is + * walked in turn, or listed valid JSON (host packs only), which is accepted + * as a terminal and is not walked; a bare name that is not a built-in, a + * non-literal specifier, or a loader name used as a value (argument, array + * element, object-literal value, ternary branch, return/arrow value, + * assignment right-hand side, default initializer, export — not a binding + * position: parameter, `catch`, destructuring pattern, import specifier) is + * a finding. + */ const javaScriptModuleSuffix = /\.(?:m?js)$/u; const generatedJavaScriptRecovery = artifactDiagnosticRecoveries.AB6005; @@ -30,6 +55,36 @@ const graphDiagnostic = (importer: string, message: string): Diagnostic => diagn generatedJavaScriptRecovery, ); +/** + * How a load names itself in an `AB6005` message — the call as written, its + * argument the literal specifier or `…` when computed: `require("left-pad")`, + * `mk(…).resolve(…)`, `load("left-pad"), a createRequire(…) loader`, + * `import.meta.resolve(…)`. + */ +const loadCall = (load: LiteralModuleLoad | ComputedModuleLoad): string => { + const argument = load.kind === 'literal' ? JSON.stringify(load.specifier) : '…'; + switch (load.form) { + case 'require': return `${load.loader}(${argument})`; + case 'require.resolve': return `${load.loader}.resolve(${argument})`; + case 'createRequire': return `${load.loader}(…)(${argument})`; + case 'createRequire.resolve': return `${load.loader}(…).resolve(${argument})`; + case 'bound-loader': return `${load.loader}(${argument}), a createRequire(…) loader`; + case 'bound-loader.resolve': return `${load.loader}.resolve(${argument}), a createRequire(…) loader`; + case 'import.meta.resolve': return `${load.loader}.resolve(${argument})`; + default: { + const exhaustive: never = load.form; + return exhaustive; + } + } +}; + +/** + * Where one specifier of a walked module leads: nowhere to walk (a built-in + * or valid JSON), a listed module to walk next, or a diagnostic. An import's + * message names the specifier alone; a load's (`via`) names the call it is + * the argument of, so `is missing "./driver.cjs"` becomes `is missing + * "./driver.cjs" in require("./driver.cjs")`. + */ const resolveJavaScriptImport = async (options: { readonly artifactRoot: string; readonly files: ReadonlyMap; @@ -39,63 +94,67 @@ const resolveJavaScriptImport = async (options: { readonly reportedImporter: string; readonly specifier: string; readonly validJson: ReadonlySet; + /** The load call the specifier is the argument of (`loadCall`), when it is a load's rather than an import's. */ + readonly via?: string; }): Promise<{ readonly diagnostic?: Diagnostic; readonly module?: string }> => { const importer = options.reportedImporter; + const specifier = JSON.stringify(options.specifier); + const failure = (message: string): { readonly diagnostic: Diagnostic } => ({ + diagnostic: graphDiagnostic(importer, options.via === undefined ? `${message}.` : `${message} in ${options.via}.`), + }); if (isBuiltin(options.specifier)) return {}; if (!options.specifier.startsWith('.') && !options.specifier.startsWith('file:')) { - return { diagnostic: graphDiagnostic(importer, `uses unsupported specifier ${JSON.stringify(options.specifier)}.`) }; + return failure(`uses unsupported specifier ${specifier}`); } let url: URL; try { url = new URL(options.specifier, pathToFileURL(resolve(options.artifactRoot, options.importer))); } catch { - return { diagnostic: graphDiagnostic(importer, `uses invalid specifier ${JSON.stringify(options.specifier)}.`) }; + return failure(`uses invalid specifier ${specifier}`); } if (url.protocol !== 'file:' || url.search.length > 0 || url.hash.length > 0) { - return { diagnostic: graphDiagnostic(importer, `uses unsupported specifier ${JSON.stringify(options.specifier)}.`) }; + return failure(`uses unsupported specifier ${specifier}`); } let path: string; try { path = fileURLToPath(url); } catch { - return { diagnostic: graphDiagnostic(importer, `uses invalid file URL ${JSON.stringify(options.specifier)}.`) }; + return failure(`uses invalid file URL ${specifier}`); } if (artifactPathFor(options.artifactRoot, path) === undefined) { - return { diagnostic: graphDiagnostic(importer, `resolves outside the artifact root: ${JSON.stringify(options.specifier)}.`) }; + return failure(`resolves outside the artifact root: ${specifier}`); } let metadata: Awaited>; try { metadata = await lstat(path); } catch { - return { diagnostic: graphDiagnostic(importer, `is missing ${JSON.stringify(options.specifier)}.`) }; + return failure(`is missing ${specifier}`); } if (!metadata.isFile()) { - return { diagnostic: graphDiagnostic(importer, `does not resolve to a regular file: ${JSON.stringify(options.specifier)}.`) }; + return failure(`does not resolve to a regular file: ${specifier}`); } let canonicalPath: string; try { canonicalPath = await realpath(path); } catch { - return { diagnostic: graphDiagnostic(importer, `is missing ${JSON.stringify(options.specifier)}.`) }; + return failure(`is missing ${specifier}`); } const artifactPath = artifactPathFor(options.artifactRoot, canonicalPath); if (artifactPath === undefined) { - return { diagnostic: graphDiagnostic(importer, `resolves outside the artifact root: ${JSON.stringify(options.specifier)}.`) }; + return failure(`resolves outside the artifact root: ${specifier}`); } if (!options.files.has(artifactPath)) { - return { diagnostic: graphDiagnostic(importer, `is not listed in the artifact manifest: ${JSON.stringify(options.specifier)}.`) }; + return failure(`is not listed in the artifact manifest: ${specifier}`); } if (jsonModuleSuffix.test(artifactPath)) { - return options.validJson.has(artifactPath) - ? {} - : { diagnostic: graphDiagnostic(importer, `references invalid JSON ${JSON.stringify(options.specifier)}.`) }; + return options.validJson.has(artifactPath) ? {} : failure(`references invalid JSON ${specifier}`); } if (!javaScriptModuleSuffix.test(artifactPath)) { - return { diagnostic: graphDiagnostic(importer, `uses unsupported target ${JSON.stringify(options.specifier)}.`) }; + return failure(`uses unsupported target ${specifier}`); } return { module: artifactPath }; }; @@ -149,34 +208,64 @@ export const validateJavaScriptModules = async (options: { visited.add(path); return; } - // Keyed by the digest of the bytes just read — not the inspection's — so - // a module rewritten between the two is never answered from the cache, - // while the same bytes scanned earlier in this process are not lexed twice. + // Both readers are keyed by the digest of the bytes just read — not the + // inspection's — so a module rewritten between the two is never answered + // from a cache, while the same bytes read earlier in this process are + // neither lexed nor scanned twice. + const source = bytes.toString('utf8'); + const sha256 = sha256Hex(bytes); let imports: readonly ModuleImport[]; try { - imports = await readModuleImports(bytes.toString('utf8'), { check, sha256: sha256Hex(bytes) }); + imports = await readModuleImports(source, { check, sha256 }); } catch { diagnostics.push(graphDiagnostic(reported(path), 'has invalid syntax.')); visiting.delete(path); visited.add(path); return; } - for (const imported of imports) { - if (imported.kind === 'meta') continue; - if (imported.specifier === undefined) { - diagnostics.push(graphDiagnostic(reported(path), 'has a non-literal dynamic import.')); - continue; - } + const follow = async (specifier: string, via?: string): Promise => { const resolved = await resolveJavaScriptImport({ artifactRoot, files, importer: path, reportedImporter: reported(path), - specifier: imported.specifier, + specifier, validJson: options.validJson, + ...(via === undefined ? {} : { via }), }); if (resolved.diagnostic !== undefined) diagnostics.push(resolved.diagnostic); else if (resolved.module !== undefined) await validateModule(resolved.module); + }; + for (const imported of imports) { + if (imported.kind === 'meta') continue; + if (imported.specifier === undefined) { + diagnostics.push(graphDiagnostic(reported(path), 'has a non-literal dynamic import.')); + continue; + } + await follow(imported.specifier); + } + // The loads the lexer does not report, held to the import rules: a literal + // specifier resolves like an import's (a built-in passes, a relative one + // is walked), a computed one and a loader passed on as a value are findings. + for (const load of scanModuleLoads(source, { sha256 })) { + switch (load.kind) { + case 'literal': + await follow(load.specifier, loadCall(load)); + break; + case 'computed': + diagnostics.push(graphDiagnostic(reported(path), `loads a non-literal specifier through ${loadCall(load)}.`)); + break; + case 'reference': + diagnostics.push(graphDiagnostic( + reported(path), + `passes ${load.form === 'require' ? load.loader : `${load.loader}, a createRequire(…) loader,`} on as a value instead of calling it.`, + )); + break; + default: { + const exhaustive: never = load; + return exhaustive; + } + } } visiting.delete(path); visited.add(path); diff --git a/packages/agent-bundle/src/core/dependency-manifest.ts b/packages/agent-bundle/src/core/dependency-manifest.ts index 753f7ce46..24940aa4f 100644 --- a/packages/agent-bundle/src/core/dependency-manifest.ts +++ b/packages/agent-bundle/src/core/dependency-manifest.ts @@ -1,27 +1,38 @@ -import { createRequire } from 'node:module'; import { dirname, join } from 'node:path'; -import { isErrno } from './errors.ts'; import { exists } from './paths.ts'; /** - * The manifest of dependency `name` as Node resolves it from `packageRoot`, - * which honours hoisting: npm, Yarn, and pnpm with a hoist pattern place a - * workspace dependency in an ancestor `node_modules`, where Rspack finds it - * too. A package whose `exports` map hides `package.json` makes that lookup - * throw, so the same ancestor walk is then performed by hand. + * The manifest of dependency `name` as seen from `packageRoot`: the first + * `/node_modules//package.json` that exists, walking from + * `packageRoot` through its ancestors up to the filesystem root, or + * `undefined` when no ancestor has the package. That walk probes only + * `node_modules` up the ancestor chain, so it honours hoisting the same + * way: npm, Yarn, and pnpm with a hoist pattern place a workspace + * dependency in an ancestor `node_modules`, where Rspack finds it too. + * `createRequire(…).resolve(…)` also consulted `NODE_PATH`, Node's global + * folders (`$HOME/.node_modules`, `$HOME/.node_libraries`, + * `$PREFIX/lib/node`), and a Yarn Plug'n'Play runtime when one is loaded; + * this walk no longer consults any of those. A scoped `name` is one + * package (`@scope/pkg`); a subpath is not supported. * - * Plain Node, no framework imports: the build's dependency-root discovery and - * `agent-bundle/serve-app-command`, which is bundled into generated - * executables, locate packages the same way. + * By hand, not through `createRequire(…).resolve(…)`: this module is bundled + * into every generated executable that imports + * `agent-bundle/serve-app-command`, and `AB6005` refuses a non-literal + * `createRequire(…).resolve(…)` in compiled output (#591), so the resolver + * call would fail the artifact build of every consumer that uses + * `spawnServeApp`. The walk also no longer depends on the package exporting + * `./package.json` — an `exports` map that hid it made the resolver throw, + * and this same walk was the fallback. + * + * The result is the path through `node_modules`, so a pnpm symlink is not + * followed here: the build-time caller (`declaredDependencyRoots` in + * `build/rslib.ts`) realpaths it, and `locateFrameworkCli` resolves the bin + * beside it, which works through the link. Plain Node, no framework imports, + * so the build's dependency-root discovery and the bundled + * `serve-app-command` locate packages the same way. */ export const dependencyManifestPath = async (packageRoot: string, name: string): Promise => { - try { - return createRequire(join(packageRoot, 'package.json')).resolve(`${name}/package.json`); - } catch (error) { - if (isErrno(error, 'MODULE_NOT_FOUND')) return undefined; - if (!isErrno(error, 'ERR_PACKAGE_PATH_NOT_EXPORTED')) throw error; - } let directory = packageRoot; while (true) { const candidate = join(directory, 'node_modules', ...name.split('/'), 'package.json'); diff --git a/packages/agent-bundle/src/core/digest.ts b/packages/agent-bundle/src/core/digest.ts index 7b6a70870..449b8e0db 100644 --- a/packages/agent-bundle/src/core/digest.ts +++ b/packages/agent-bundle/src/core/digest.ts @@ -8,6 +8,38 @@ export const sha256Hex = (bytes: string | Uint8Array): string => export const sha256File = async (path: string): Promise => sha256Hex(await readFile(path)); +/** + * Values remembered by the digest of the bytes they were computed from, at + * most `limit` of them: when the cache is full, a new key evicts the oldest + * entry (insertion order; re-setting a known key neither grows the cache nor + * evicts). Within one process the same emitted bundle is read by several + * passes whose bytes never change between them — the post-compile + * self-containment check, then artifact validation before and after the + * manifest is written — so a scan of a multi-megabyte module runs once and + * the digest of the bytes just read, not of an earlier inspection, is what + * says whether the remembered value still applies. + */ +export class DigestCache { + readonly #entries = new Map(); + readonly #limit: number; + + constructor(limit: number) { + this.#limit = limit; + } + + get(key: string): T | undefined { + return this.#entries.get(key); + } + + set(key: string, value: T): void { + if (this.#entries.size >= this.#limit && !this.#entries.has(key)) { + const oldest = this.#entries.keys().next(); + if (!oldest.done) this.#entries.delete(oldest.value); + } + this.#entries.set(key, value); + } +} + const serializeJson = (value: unknown, key = ''): string | undefined => { if (value !== null && typeof value === 'object') { const toJson = (value as { toJSON?: unknown }).toJSON; diff --git a/packages/agent-bundle/src/serve-app-command.ts b/packages/agent-bundle/src/serve-app-command.ts index f306b9260..041f11edb 100644 --- a/packages/agent-bundle/src/serve-app-command.ts +++ b/packages/agent-bundle/src/serve-app-command.ts @@ -151,12 +151,19 @@ export class ServeAppCommandError extends CodedError { /** * The `agent-bundle` CLI entry (`bin/agent-bundle.js`) of the framework * installed for the project at `root`, resolved the way the framework itself - * finds a dependency: through Node's resolution from the project's - * `package.json` (which honours hoisting and pnpm's layout), then by the - * ancestor `node_modules` walk when the package's `exports` hide its - * manifest. `undefined` when the framework is not installed: the published - * plugin package and an installed host pack ship no runtime dependencies, so - * only a checkout (or a consumer that installed `agent-bundle`) can serve. + * finds a dependency: the first `node_modules/agent-bundle/package.json` in + * `root` or an ancestor directory — `node_modules` probing up the ancestor + * chain only (`core/dependency-manifest.ts`), which honours hoisting and + * pnpm's layout. `createRequire(…).resolve(…)` also consulted `NODE_PATH`, + * Node's global folders (`$HOME/.node_modules`, `$HOME/.node_libraries`, + * `$PREFIX/lib/node`), and a Yarn Plug'n'Play runtime when one is loaded; + * this walk no longer consults any of those, and never calls + * `createRequire(…).resolve(…)`, a load `AB6005` would refuse in the + * executable this module is bundled into. The bin path is `bin` from that + * manifest resolved beside it. `undefined` when the framework is not + * installed: the published plugin package and an installed host pack ship + * no runtime dependencies, so only a checkout (or a consumer that + * installed `agent-bundle`) can serve. */ export const locateFrameworkCli = async (root: string): Promise => { const manifestPath = await dependencyManifestPath(resolve(root), 'agent-bundle'); diff --git a/packages/agent-bundle/tests/dependency-manifest.test.ts b/packages/agent-bundle/tests/dependency-manifest.test.ts new file mode 100644 index 000000000..332ff35f3 --- /dev/null +++ b/packages/agent-bundle/tests/dependency-manifest.test.ts @@ -0,0 +1,67 @@ +import { mkdir, mkdtemp, realpath, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { afterEach, describe, expect, it } from '@rstest/core'; + +import { dependencyManifestPath } from '../src/core/dependency-manifest.ts'; + +const roots: string[] = []; + +/** A fresh temporary tree, realpath'd so the pnpm expectation below compares real paths like for like. */ +const temporaryRoot = async (): Promise => { + const root = await realpath(await mkdtemp(join(tmpdir(), 'agent-bundle-dependency-manifest-'))); + roots.push(root); + return root; +}; + +/** Writes `/package.json` for `name`, creating the directory, and returns the manifest path. */ +const writeManifest = async (packageDirectory: string, name: string): Promise => { + await mkdir(packageDirectory, { recursive: true }); + const manifest = join(packageDirectory, 'package.json'); + await writeFile(manifest, `${JSON.stringify({ name, version: '1.0.0' })}\n`); + return manifest; +}; + +afterEach(async () => { + await Promise.all(roots.splice(0).map((root) => rm(root, { force: true, recursive: true }))); +}); + +describe('dependencyManifestPath', () => { + it('finds the manifest in the node_modules of the package root itself', async () => { + const root = await temporaryRoot(); + const manifest = await writeManifest(join(root, 'node_modules', 'left-pad'), 'left-pad'); + + await expect(dependencyManifestPath(root, 'left-pad')).resolves.toBe(manifest); + }); + + it('walks up to the ancestor node_modules where hoisting placed a scoped package', async () => { + const parent = await temporaryRoot(); + const manifest = await writeManifest(join(parent, 'node_modules', '@scope', 'pkg'), '@scope/pkg'); + const app = join(parent, 'app'); + // A nearer node_modules without the package does not end the walk. + await mkdir(join(app, 'node_modules'), { recursive: true }); + + await expect(dependencyManifestPath(app, '@scope/pkg')).resolves.toBe(manifest); + }); + + it('returns undefined when no ancestor node_modules has the package', async () => { + const root = await temporaryRoot(); + await writeManifest(join(root, 'node_modules', 'other'), 'other'); + await mkdir(join(root, 'app'), { recursive: true }); + + await expect(dependencyManifestPath(join(root, 'app'), 'left-pad')).resolves.toBeUndefined(); + await expect(dependencyManifestPath(root, '@scope/pkg')).resolves.toBeUndefined(); + }); + + it('returns the path through a symlinked package directory (the pnpm layout) and leaves realpath to the caller', async () => { + const root = await temporaryRoot(); + const real = await writeManifest(join(root, 'node_modules', '.pnpm', 'pkg@1.0.0', 'node_modules', 'pkg'), 'pkg'); + await symlink(join('.pnpm', 'pkg@1.0.0', 'node_modules', 'pkg'), join(root, 'node_modules', 'pkg'), 'dir'); + + const located = await dependencyManifestPath(root, 'pkg'); + expect(located).toBe(join(root, 'node_modules', 'pkg', 'package.json')); + // The link is where Node would find the package too; `declaredDependencyRoots` realpaths it. + expect(located === undefined ? undefined : await realpath(located)).toBe(real); + }); +}); diff --git a/packages/agent-bundle/tests/generated-module-loads.test.ts b/packages/agent-bundle/tests/generated-module-loads.test.ts new file mode 100644 index 000000000..86fbfcd77 --- /dev/null +++ b/packages/agent-bundle/tests/generated-module-loads.test.ts @@ -0,0 +1,529 @@ +/** + * The check #591 demands of the framework's own output: no JavaScript the + * framework generates loads anything by a bare package specifier through a + * form the bundler cannot inline. Every generator that renders a module the + * plugin build compiles or emits verbatim is rendered here with the smallest + * arguments it accepts and scanned with the leaf scanner `AB6005` runs over + * compiled host-pack modules (`build/module-loads.ts`): a `computed` load + * (`require(name)`, `createRequire(…)(expr)`, `import.meta.resolve(expr)`), + * a loader passed on as a value, or a `literal` load whose specifier is + * neither a Node built-in nor relative (`./`, `../`) nor `file:` fails the + * suite and prints the load. + * + * Coverage notes: + * - `build/cli-bins.ts` produces no source of its own: `cliBinRslibEntries` + * delegates every entry to `generatedCliBinEntrySource` and + * `generatedRenderedRouteWorkerSource`, both rendered below. + * - The composite plugin root (#555 W1, PR #578) renders no JavaScript of + * its own: `build/compose.ts` merges the adapters' planned entries, its + * `bin/` entries come from `cliBinRslibEntries` + * (`generatedCliBinEntrySource`, `generatedRenderedRouteWorkerSource`) and + * its `mcp/` entries from `planMcpEntriesSurface` + * (`generatedRouteMcpEntrySource` with `allowedTargets`/`hosts`, + * `generatedStdioMcpEntrySource`, `generatedRouteFlightWorkerSource`), all + * rendered below. + * - The event-route wrapper source is module-private to + * `adapters/hook-contract.ts`; it is reached through `planHooks` with each + * adapter's real hook contract — and, for a composite root, the per-plan + * Cursor contract whose wrapper paths `hookWrapperPath` assigns — and every + * planned `virtualSource` is scanned. + */ +import { isBuiltin } from 'node:module'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { describe, expect, it } from '@rstest/core'; + +import { type BuiltInHost, builtInHostNames, hookWrapperPath } from '../src/adapters/composite-layout.ts'; +import { createCursorHookContract, cursorArtifactPaths } from '../src/adapters/cursor.ts'; +import { + cursorHookWrapperSource, + nativeHookWrapperSource, + planHooks, + type TargetHookContract, + type TargetHookWrapper, +} from '../src/adapters/hook-contract.ts'; +import { createDefaultRegistry } from '../src/adapters/registry.ts'; +import { planMcpEntriesSurface } from '../src/build/entries.ts'; +import { + generatedCliBinEntrySource, + generatedExecutableEntrySource, + generatedInstallBinEntrySource, + generatedRenderedRouteWorkerSource, + generatedRenderedScriptEntrySource, + generatedRouteFlightWorkerSource, + generatedRouteMcpEntrySource, + generatedStdioMcpEntrySource, + stdioPreludeModuleSource, +} from '../src/build/entry-shell.ts'; +import { operatorEnvLayerModuleSource } from '../src/build/launch-env-shell.ts'; +import { generatedMetaModuleSource } from '../src/build/meta.ts'; +import { type ModuleLoad, scanModuleLoads } from '../src/build/module-loads.ts'; +import type { + NormalizedHook, + NormalizedNoticeRetentionPolicy, + NormalizedPlugin, + NormalizedStateDefinition, + SourceProvenance, +} from '../src/core/types.ts'; +import { installSurfaceEntries } from '../src/install/surface.ts'; +import type { CompiledAgentRoute, CompiledCliCommand, CompiledLayout, CompiledProvider } from '../src/routes/types.ts'; + +/** A literal load the bundler inlines or Node serves itself: a built-in, a relative path, or a `file:` URL. */ +const allowedLiteral = (specifier: string): boolean => + isBuiltin(specifier) || specifier.startsWith('./') || specifier.startsWith('../') || specifier.startsWith('file:'); + +/** Every load of `source` that AB6005 would refuse in a compiled host-pack module. */ +const offendingLoads = (source: string): readonly ModuleLoad[] => + scanModuleLoads(source).filter((load) => load.kind !== 'literal' || !allowedLiteral(load.specifier)); + +const describeLoads = (loads: readonly ModuleLoad[]): string => loads + .map((load) => `${load.kind} ${load.form} via ${load.loader}${load.kind === 'literal' ? ` ${JSON.stringify(load.specifier)}` : ''}`) + .join('; '); + +const expectNoBareLoads = (label: string, source: string): void => { + expect(source.length, `${label} rendered nothing`).toBeGreaterThan(0); + const offending = offendingLoads(source); + expect(offending, `${label} loads by a bare package specifier: ${describeLoads(offending)}`).toEqual([]); +}; + +/** Narrows an optional adapter binding the fixture depends on, naming it when the adapter stops declaring it. */ +const declared = (value: T | undefined, label: string): T => { + if (value === undefined) throw new Error(`${label} is not declared`); + return value; +}; + +const configProvenance: SourceProvenance = { kind: 'config', sourcePath: '/project/agent-bundle.config.ts' }; + +const route = ( + id: string, + kind: CompiledAgentRoute['kind'], + source: string, + extra: Partial> = {}, +): CompiledAgentRoute => ({ + config: {}, + id, + kind, + provenance: { kind: 'conventional', relativePath: source.slice('/project/'.length) }, + source, + ...extra, +}); + +const cliRoute = route('cli:report', 'cli', '/project/src/cli/report.ts'); +const toolRoute = route('tool:curator/inspect', 'tool', '/project/src/mcp/curator/tools/inspect.tsx', { serverId: 'mcp:curator' }); +const resourceRoute = route('resource:curator/catalog', 'resource', '/project/src/mcp/curator/resources/catalog.tsx', { + config: { uri: 'catalog://books' }, + serverId: 'mcp:curator', +}); +const scriptRoute = route('script:rebuild', 'script', '/project/src/scripts/rebuild.tsx'); + +const providers: readonly CompiledProvider[] = [{ + id: 'provider:zeta', + name: 'zeta', + provenance: { kind: 'conventional', relativePath: 'src/providers/zeta.ts' }, + source: '/project/src/providers/zeta.ts', +}]; + +const layouts: readonly CompiledLayout[] = [ + { + id: 'layout:root', + provenance: { kind: 'conventional', relativePath: 'src/layout.tsx' }, + scope: 'root', + source: '/project/src/layout.tsx', + }, + { + id: 'layout:mcp:curator', + provenance: { kind: 'conventional', relativePath: 'src/mcp/curator/layout.tsx' }, + scope: 'server', + serverId: 'mcp:curator', + source: '/project/src/mcp/curator/layout.tsx', + }, +]; + +const durableState: NormalizedStateDefinition = { + id: 'project/tasks', + lifetime: 'workspace-durable', + provenance: { kind: 'conventional', sourcePath: '/project/src/state.ts' }, + source: '/project/src/state.ts', +}; +const volatileState: NormalizedStateDefinition = { ...durableState, lifetime: 'process' }; +const noticeRetention: NormalizedNoticeRetentionPolicy = { maxJournalBytes: 1024, maxTerminal: 3, terminalTtlMs: 60_000 }; +const registry = createDefaultRegistry(); +const noticeDelivery = declared(registry.noticeDelivery('claude'), 'the claude adapter notice delivery advertisement'); + +const plainCommand: CompiledCliCommand = { + aliases: [], + exitCode: 'zero', + options: [], + path: ['report'], + rendered: false, + routeId: 'cli:report', +}; +const renderedCommand: CompiledCliCommand = { + aliases: [], + exitCode: 'zero', + mcp: { confirm: false, server: 'curator', tool: 'inspect' }, + options: [], + path: ['curator', 'inspect'], + rendered: true, + routeId: 'tool:curator/inspect', +}; +const plugin = { name: 'fixture', version: '1.0.0' }; + +/** The hosts of a composite root that receive hooks; `portable` hosts none. */ +const hookTargets: readonly string[] = ['claude', 'codex', 'cursor']; + +/** A config-declared hook: the wrapper the native (Claude/Codex) and Cursor codecs render directly. */ +const configHook: NormalizedHook = { + event: 'sessionStart', + id: 'hook:sessionStart:probe', + name: 'probe', + provenance: configProvenance, + source: '/project/src/hooks/probe.ts', + targets: hookTargets, + tools: [], +}; + +type HookEventRoute = NonNullable; + +/** A filesystem event route: the wrapper only `planHooks` can render, in each runtime and fallback mode. */ +const eventRouteHook = (event: NormalizedHook['event'], eventRoute: HookEventRoute): NormalizedHook => { + const slug = `${eventRoute.event.replace('/', '-')}-${eventRoute.runtime}-${eventRoute.fallback}`; + return { + event, + eventRoute, + id: `hook:event-route:${slug}`, + name: `event-route-${slug}`, + provenance: { kind: 'conventional', sourcePath: `/project/src/events/${eventRoute.event}.tsx` }, + source: `/project/src/events/${eventRoute.event}.tsx`, + targets: hookTargets, + tools: [], + }; +}; + +const eventRouteHooks: readonly NormalizedHook[] = [ + eventRouteHook('afterTool', { event: 'tool/after', fallback: 'none', runtime: 'shared' }), + eventRouteHook('afterTool', { event: 'tool/after', fallback: 'standalone', runtime: 'shared' }), + eventRouteHook('sessionEnd', { event: 'session/end', fallback: 'none', runtime: 'standalone' }), +]; + +const model = (state?: NormalizedStateDefinition): NormalizedPlugin => ({ + extensions: {}, + hooks: [configHook, ...eventRouteHooks], + mcpServers: [], + metadata: { + description: 'Renders every generated module for the AB6005 audit.', + id: 'plugin:fixture', + name: 'fixture', + provenance: configProvenance, + version: '1.0.0', + }, + runtime: { node: '22.12.0' }, + scripts: [], + skills: [], + ...(state === undefined ? {} : { state }), + targets: builtInHostNames.map((name) => ({ + id: `target:${name}`, + name, + provenance: configProvenance, + })), +}); + +/** + * Every hook contract the framework binds a `wrapperSource` to, with the + * target name `planHooks` selects hooks by: each adapter's registered + * contract, plus the per-plan Cursor contract a composite root uses + * (`adapters/cursor.ts`), whose wrapper paths carry the host suffix + * `hookWrapperPath` assigns when a hook reaches several selected hosts (#555). + */ +const hookPlanners: ReadonlyArray<{ + readonly contract: TargetHookContract; + readonly label: string; + readonly target: string; +}> = [ + // The production hook list comes from the registry: Claude, Codex, and Cursor bind a contract; + // portable declares hooks unavailable. + ...registry.names().flatMap((target) => { + const contract = registry.hookContract(target); + return contract === undefined ? [] : [{ contract, label: target, target }]; + }), + { + contract: createCursorHookContract({ + manifestPath: cursorArtifactPaths.hooks, + wrapperPath: (hook) => hookWrapperPath('cursor', hook.name, hook.targets, builtInHostNames), + }), + label: 'cursor, composite-root wrapper paths', + target: 'cursor', + }, +]; + +describe('generated JavaScript loads nothing by a bare package specifier', () => { + it('can fail: a createRequire load of a bare package is one offending load', () => { + expect(() => { + expectNoBareLoads( + 'negative control', + 'export const x = createRequire(import.meta.url)("left-pad");', + ); + }).toThrow(/left-pad/u); + // The allowed literal forms stay silent, so a passing suite means no bare load rather than no load. + expect(offendingLoads([ + 'const fs = require("node:fs");', + 'const path = require("path");', + 'require("./driver.cjs");', + 'import.meta.resolve("../worker.mjs");', + 'import.meta.resolve("file:///opt/worker.mjs");', + ].join('\n'))).toEqual([]); + }); + + it('build/entry-shell: the stdio prelude and the stdio MCP entry', () => { + expectNoBareLoads('stdioPreludeModuleSource()', stdioPreludeModuleSource()); + expectNoBareLoads('stdioPreludeModuleSource(env)', stdioPreludeModuleSource({ API_URL: 'https://api.example' })); + expectNoBareLoads( + 'generatedStdioMcpEntrySource', + generatedStdioMcpEntrySource({ entrySource: '/project/src/mcp/curator.ts', serverName: 'curator' }), + ); + }); + + it('build/entry-shell: the executable and install bin envelopes', () => { + expectNoBareLoads( + 'generatedExecutableEntrySource(main, cli)', + generatedExecutableEntrySource({ entrySource: '/project/src/cli.ts', exportName: 'main', hostSurface: 'cli' }), + ); + expectNoBareLoads( + 'generatedExecutableEntrySource(default, script)', + generatedExecutableEntrySource({ entrySource: '/project/src/scripts/export.ts', exportName: 'default', hostSurface: 'script' }), + ); + expectNoBareLoads( + 'generatedExecutableEntrySource(default)', + generatedExecutableEntrySource({ entrySource: '/project/src/scripts/export.ts', exportName: 'default' }), + ); + expectNoBareLoads( + 'generatedInstallBinEntrySource', + generatedInstallBinEntrySource({ artifactRelativeUrl: '../../artifact/', hosts: ['claude', 'codex', 'cursor'], name: 'installer' }), + ); + }); + + it('build/entry-shell: the routed CLI bin, its render worker, and the rendered script entry', () => { + expectNoBareLoads( + 'generatedCliBinEntrySource(plain)', + generatedCliBinEntrySource({ commands: [plainCommand], plugin, routes: [cliRoute] }), + ); + expectNoBareLoads( + 'generatedCliBinEntrySource(providers, durable state, artifact fallback)', + generatedCliBinEntrySource({ + commands: [plainCommand], + plugin: { ...plugin, description: 'Routed fixture CLI.' }, + providers, + routes: [cliRoute], + state: durableState, + stateFallback: 'artifact', + }), + ); + expectNoBareLoads( + 'generatedCliBinEntrySource(npm bin: durable state, no stateFallback, rendered MCP command)', + generatedCliBinEntrySource({ + commands: [renderedCommand, plainCommand], + noticeRetention, + plugin, + routes: [cliRoute, toolRoute], + state: durableState, + workerFile: 'fixture-flight.mjs', + }), + ); + expectNoBareLoads( + 'generatedCliBinEntrySource(volatile state)', + generatedCliBinEntrySource({ commands: [plainCommand], plugin, routes: [cliRoute], state: volatileState }), + ); + expectNoBareLoads( + 'generatedRenderedRouteWorkerSource(plain)', + generatedRenderedRouteWorkerSource({ routes: [cliRoute] }), + ); + expectNoBareLoads( + 'generatedRenderedRouteWorkerSource(layouts, providers, durable state, artifact fallback)', + generatedRenderedRouteWorkerSource({ + layouts, + noticeRetention, + providers, + routes: [cliRoute, toolRoute, scriptRoute], + state: durableState, + stateFallback: 'artifact', + }), + ); + expectNoBareLoads( + 'generatedRenderedRouteWorkerSource(volatile state)', + generatedRenderedRouteWorkerSource({ routes: [scriptRoute], state: volatileState }), + ); + expectNoBareLoads( + 'generatedRenderedScriptEntrySource', + generatedRenderedScriptEntrySource({ name: 'rebuild', routeId: 'script:rebuild', workerFile: 'rebuild-flight.mjs' }), + ); + expectNoBareLoads( + 'generatedRenderedScriptEntrySource(durable state)', + generatedRenderedScriptEntrySource({ + name: 'rebuild', + noticeRetention, + routeId: 'script:rebuild', + state: durableState, + workerFile: 'rebuild-flight.mjs', + }), + ); + }); + + it('build/entry-shell: the generated MCP server entry and its Flight worker', () => { + expectNoBareLoads( + 'generatedRouteFlightWorkerSource(stateless)', + generatedRouteFlightWorkerSource({ artifactEpoch: 'fixture@1.0.0', routes: [toolRoute], serverName: 'curator' }), + ); + expectNoBareLoads( + 'generatedRouteFlightWorkerSource(event routes, layouts, providers, durable state, notices)', + generatedRouteFlightWorkerSource({ + artifactEpoch: 'fixture@1.0.0', + eventRoutes: eventRouteHooks, + layouts, + noticeDelivery, + noticeRetention, + providers, + routes: [toolRoute, resourceRoute], + serverName: 'curator', + state: durableState, + }), + ); + expectNoBareLoads( + 'generatedRouteFlightWorkerSource(volatile state)', + generatedRouteFlightWorkerSource({ + artifactEpoch: 'fixture@1.0.0', + noticeDelivery, + routes: [toolRoute], + serverName: 'curator', + state: volatileState, + }), + ); + expectNoBareLoads( + 'generatedRouteMcpEntrySource(stateless)', + generatedRouteMcpEntrySource({ plugin, routes: [toolRoute], serverName: 'curator', workerFile: 'mcp-curator-flight.mjs' }), + ); + expectNoBareLoads( + 'generatedRouteMcpEntrySource(event routes, durable state, notices, composite root hosting the event runtime)', + generatedRouteMcpEntrySource({ + allowedTargets: hookTargets, + artifactEpoch: 'fixture@1.0.0', + eventRoutes: eventRouteHooks, + hosts: builtInHostNames, + noticeDelivery, + noticeRetention, + plugin, + routes: [toolRoute, resourceRoute], + serverName: 'curator', + state: durableState, + workerFile: 'mcp-curator-flight.mjs', + }), + ); + expectNoBareLoads( + 'generatedRouteMcpEntrySource(volatile state)', + generatedRouteMcpEntrySource({ + noticeDelivery, + plugin, + routes: [toolRoute], + serverName: 'curator', + state: volatileState, + workerFile: 'mcp-curator-flight.mjs', + }), + ); + }); + + it('build/launch-env-shell: the operator env layer', () => { + expectNoBareLoads('operatorEnvLayerModuleSource()', operatorEnvLayerModuleSource()); + expectNoBareLoads( + 'operatorEnvLayerModuleSource(env)', + operatorEnvLayerModuleSource({ API_URL: 'https://api.example', LOG_LEVEL: 'info' }), + ); + }); + + it('build/meta and entries: the project identity and MCP Apps registry modules', async () => { + expectNoBareLoads( + 'generatedMetaModuleSource', + generatedMetaModuleSource({ name: 'fixture', packageName: '@fixture/plugin', packageVersion: '1.0.0', version: '1.0.0' }), + ); + + const surface = await planMcpEntriesSurface([{ + args: ['mcp/mcp-curator-12345678.mjs'], + id: 'mcp:curator', + name: 'curator', + provenance: configProvenance, + source: import.meta.filename, + targets: ['claude'], + transport: 'stdio', + }], { + artifactEpoch: 'fixture@1', + eventHooks: [], + outDir: join(tmpdir(), 'agent-bundle-generated-module-loads'), + plugin, + target: 'claude', + targets: ['claude'], + }); + const mcpApps = declared( + surface.entries + .flatMap((entry) => entry.virtualModules ?? []) + .find((module) => module.name === 'agent-bundle/mcp-apps'), + 'the generated agent-bundle/mcp-apps registry module', + ); + expectNoBareLoads('planMcpEntriesSurface agent-bundle/mcp-apps', mcpApps.source); + }); + + it('adapters/hook-contract: the native and Cursor wrapper codecs', () => { + const wrapper: TargetHookWrapper = { + event: 'sessionStart', + hook: configHook, + nativeEvent: 'SessionStart', + relativePath: 'hooks/probe.mjs', + target: 'claude', + }; + + expectNoBareLoads('nativeHookWrapperSource(Claude)', nativeHookWrapperSource(wrapper, 'Claude')); + expectNoBareLoads('nativeHookWrapperSource(Codex)', nativeHookWrapperSource({ ...wrapper, target: 'codex' }, 'Codex')); + expectNoBareLoads( + 'cursorHookWrapperSource', + cursorHookWrapperSource({ ...wrapper, nativeEvent: 'sessionStart', target: 'cursor' }), + ); + }); + + it('adapters/hook-contract: every wrapper planHooks renders through each adapter hook contract, including event routes', () => { + for (const [stateLabel, state] of [['stateless', undefined], ['durable state', durableState]] as const) { + for (const planner of hookPlanners) { + const label = `planHooks(${planner.label}; ${stateLabel})`; + const plan = planHooks(model(state), planner.target, planner.contract); + + expect(plan.diagnostics, `${label} diagnostics`).toEqual([]); + expect(plan.hookEntries.length, `${label} planned no wrapper`).toBeGreaterThan(0); + // The event-route wrapper is module-private; the plan is the only way to render it. + expect( + plan.hookEntries.some((entry) => entry.hook.eventRoute !== undefined), + `${label} planned no event-route wrapper`, + ).toBe(true); + for (const entry of plan.hookEntries) { + expectNoBareLoads(`${label} ${entry.relativePath}`, entry.virtualSource); + } + } + } + }); + + it('install/surface: the verbatim installer of every selection that emits one', () => { + // One install surface per composite root (#555): each built-in host alone, and all four together. + const selections: readonly (readonly BuiltInHost[])[] = [ + ...builtInHostNames.map((host): readonly BuiltInHost[] => [host]), + builtInHostNames, + ]; + const emitting: string[] = []; + for (const hosts of selections) { + const label = hosts.join('+'); + for (const entry of installSurfaceEntries(model(), hosts)) { + if (!/\.(?:mjs|js)$/u.test(entry.relativePath)) continue; + emitting.push(label); + expectNoBareLoads(`installSurfaceEntries(${label}) ${entry.relativePath}`, entry.content); + } + } + // install.mjs is emitted verbatim, never bundled, so it is scanned as written; it ships whenever + // Cursor or the portable format is among the selected hosts. + expect(emitting).toEqual(['cursor', 'portable', 'claude+codex+cursor+portable']); + }); +}); diff --git a/packages/agent-bundle/tests/module-loads.test.ts b/packages/agent-bundle/tests/module-loads.test.ts new file mode 100644 index 000000000..43588b597 --- /dev/null +++ b/packages/agent-bundle/tests/module-loads.test.ts @@ -0,0 +1,342 @@ +import { describe, expect, it } from '@rstest/core'; + +import { + decodeLiteral, + quotedLiteral, + scanModuleLoads, + type LoaderReference, + type ModuleLoad, + type ModuleLoadForm, +} from '../src/build/module-loads.ts'; +import { sha256Hex } from '../src/core/digest.ts'; + +const literal = (form: ModuleLoadForm, loader: string, specifier: string): ModuleLoad => ({ form, kind: 'literal', loader, specifier }); +const computed = (form: ModuleLoadForm, loader: string): ModuleLoad => ({ form, kind: 'computed', loader }); +const reference = (form: LoaderReference['form'], loader: string): ModuleLoad => ({ form, kind: 'reference', loader }); + +const rspackShim = (declaration: 'const' | 'let' | 'var'): string => [ + 'import { createRequire as __rspack_createRequire } from "node:module";', + `${declaration} __rspack_createRequire_require = __rspack_createRequire(import.meta.url);`, + 'const left_pad_namespaceObject = __rspack_createRequire_require("left-pad");', + '', +].join('\n'); + +describe('scanModuleLoads reports a literal load', () => { + it.each([ + ['require("…")', 'module.exports = require("node:path");', [literal('require', 'require', 'node:path')]], + ['require.resolve("…")', 'module.exports = require.resolve("node:path");', [literal('require.resolve', 'require', 'node:path')]], + ['import.meta.resolve("…")', 'export const tool = import.meta.resolve("tool-pkg/bin/tool");', [literal('import.meta.resolve', 'import.meta', 'tool-pkg/bin/tool')]], + ['an inline node -e program', "require('optional-driver')", [literal('require', 'require', 'optional-driver')]], + ['a statement-position call', 'require("./polyfill.js");\n', [literal('require', 'require', './polyfill.js')]], + ['a relative require and createRequire target', 'const helper = require("./helper.cjs");\nconst data = createRequire(import.meta.url)("./data.json");\n', [ + literal('require', 'require', './helper.cjs'), + literal('createRequire', 'createRequire', './data.json'), + ]], + // The factory called at once, bare or qualified through a namespace, a default import, or a CommonJS load of node:module. + ['an inline factory call', 'import { createRequire } from "node:module";\nexport const driver = createRequire(import.meta.url)("driver-package");\n', [literal('createRequire', 'createRequire', 'driver-package')]], + ['a nested factory argument', 'import { createRequire } from "node:module";\nexport const driver = createRequire(new URL("./entry.js", import.meta.url))("driver-package");\n', [literal('createRequire', 'createRequire', 'driver-package')]], + ['Module.createRequire(…)("…")', 'import * as Module from "node:module";\nexport const driver = Module.createRequire(import.meta.url)("driver-package");', [literal('createRequire', 'createRequire', 'driver-package')]], + ['module.createRequire(…)("…")', 'import module from "node:module";\nexport const driver = module.createRequire(import.meta.url)("driver-package");', [literal('createRequire', 'createRequire', 'driver-package')]], + ['require("node:module").createRequire(…)("…")', 'module.exports = require("node:module").createRequire(__filename)("driver-package");', [ + literal('require', 'require', 'node:module'), + literal('createRequire', 'createRequire', 'driver-package'), + ]], + ["require('module').createRequire(…)('…')", "module.exports = require('module').createRequire(__filename)('driver-package');", [ + literal('require', 'require', 'module'), + literal('createRequire', 'createRequire', 'driver-package'), + ]], + ['require("node:module").createRequire(…).resolve("…")', 'module.exports = require("node:module").createRequire(__filename).resolve("driver-package");', [ + literal('require', 'require', 'node:module'), + literal('createRequire.resolve', 'createRequire', 'driver-package'), + ]], + ['an async node-commonjs external', 'const p = import("node:module").then(function(module) { return (module.createRequire(import.meta.url)("left-pad")) });\n', [literal('createRequire', 'createRequire', 'left-pad')]], + // A loader bound from the factory, under the factory's own name, an alias, or a qualifier. + ['an aliased factory bound to a loader', 'import { createRequire as makeRequire } from "node:module";\nconst load = makeRequire(import.meta.url);\nexport const driver = load("driver-package");\n', [literal('bound-loader', 'load', 'driver-package')]], + ['a loader bound from require("node:module").createRequire', 'const load = require("node:module").createRequire(__filename);\nmodule.exports = load("driver-package");\n', [ + literal('require', 'require', 'node:module'), + literal('bound-loader', 'load', 'driver-package'), + ]], + ['a loader bound from a two-level namespace', 'import * as ns from "node:module";\nconst load = ns.default.createRequire(import.meta.url);\nexport const driver = load("driver-package");', [literal('bound-loader', 'load', 'driver-package')]], + ['a bound loader\'s .resolve', 'const load = createRequire(import.meta.url);\nexport const where = load.resolve("driver-package");\n', [literal('bound-loader.resolve', 'load', 'driver-package')]], + ['the Rspack node-commonjs shim, const', rspackShim('const'), [literal('bound-loader', '__rspack_createRequire_require', 'left-pad')]], + ['the Rspack node-commonjs shim, let', rspackShim('let'), [literal('bound-loader', '__rspack_createRequire_require', 'left-pad')]], + ['the Rspack node-commonjs shim, var', rspackShim('var'), [literal('bound-loader', '__rspack_createRequire_require', 'left-pad')]], + ['the Rspack shim loading a built-in', 'import { createRequire as __rspack_createRequire } from "node:module";\nconst __rspack_createRequire_require = __rspack_createRequire(import.meta.url);\nmodule.exports = __rspack_createRequire_require("util");\n', [literal('bound-loader', '__rspack_createRequire_require', 'util')]], + ['a bound loader beside an optional-chained method of the same name', 'const load = createRequire(import.meta.url);\nregistry?.load(name);\nload("driver-package");\n', [literal('bound-loader', 'load', 'driver-package')]], + // Trivia and tokens around the call. + ['comment trivia inside the call', 'module.exports = require /* driver */ ( // which\n /* a */ "driver-package" /* b */ );\n', [literal('require', 'require', 'driver-package')]], + ['a hex escape in the specifier', String.raw`const hex = require("\x68ex-pkg");`, [literal('require', 'require', 'hex-pkg')]], + ['a unicode escape in the specifier', String.raw`const unicode = require('unicode-pkg\u002fsubpath');`, [literal('require', 'require', 'unicode-pkg/subpath')]], + ['a load after a nested template literal', 'const text = `outer ${flag ? `require("in-template")` : "x"} end`;\nrequire("after-template");\n', [literal('require', 'require', 'after-template')]], + ['a load after a regex literal holding a quote, on the same line', 'const quote = /["\']/u; require("left-pad");\n', [literal('require', 'require', 'left-pad')]], + // A template's substitutions are code: scanned with the source's names, reported where the template appears. + ['a load in a template substitution', 'const v = `pre ${require("x")} post`;\n', [literal('require', 'require', 'x')]], + ['a load in a nested template substitution', 'const v = `${`${require("x")}`}`;\n', [literal('require', 'require', 'x')]], + ['loads around and inside templates, in source order', 'require("1"); const v = `${require("2")} ${`${require("3")}`}`; require("4");\n', [ + literal('require', 'require', '1'), + literal('require', 'require', '2'), + literal('require', 'require', '3'), + literal('require', 'require', '4'), + ]], + ['a bound loader called in a template substitution', 'const load = createRequire(import.meta.url);\nconst v = `${load("driver-package")}`;\n', [literal('bound-loader', 'load', 'driver-package')]], + ['a loader bound and called inside a template substitution', 'const v = `${(() => { const load = createRequire(import.meta.url); return load("driver-package"); })()}`;\n', [literal('bound-loader', 'load', 'driver-package')]], + ['a factory aliased and called inside a template substitution', 'const v = `${(() => { const { createRequire: mk } = host; return mk(import.meta.url)("driver-package"); })()}`;\n', [literal('createRequire', 'mk', 'driver-package')]], + [ + 'a module bound from a factory call, then called: the load is the factory call, the binding is not a loader', + 'const pad = createRequire(import.meta.url)("driver-package");\nconst where = createRequire(import.meta.url).resolve("asset-pkg");\nexport const v = `${pad("", 2)}${where.length}`;\npad(name, 2);\n', + [literal('createRequire', 'createRequire', 'driver-package'), literal('createRequire.resolve', 'createRequire', 'asset-pkg')], + ], + ['a template beyond the nesting budget, whose text is scanned as code', 'const v = `${ {a:{b:{c: require("x")}}} }`;\n', [literal('require', 'require', 'x')]], + // A `/` after `++` or `--` is division, so the operand after it is code. + ['a load after a postfix increment and a division', 'count++ / require("left-pad") / divisor\n', [literal('require', 'require', 'left-pad')]], + ['a load after a postfix decrement and a division', 'count-- / require("left-pad") / divisor\n', [literal('require', 'require', 'left-pad')]], + ['a load after a prefix increment and a division', 'const x = ++i / 2; require("left-pad");\n', [literal('require', 'require', 'left-pad')]], + // Optional chaining before the argument list or `resolve`. + ['require?.("…")', 'module.exports = require?.("left-pad");', [literal('require', 'require', 'left-pad')]], + ['require.resolve?.("…")', 'module.exports = require.resolve?.("left-pad");', [literal('require.resolve', 'require', 'left-pad')]], + ['require?.resolve("…")', 'module.exports = require?.resolve("left-pad");', [literal('require.resolve', 'require', 'left-pad')]], + ['require ?. ("…") with spaces', 'module.exports = require ?. ("left-pad");', [literal('require', 'require', 'left-pad')]], + ['load?.("…") for a bound loader', 'const load = createRequire(import.meta.url);\nexport const driver = load?.("driver-package");\n', [literal('bound-loader', 'load', 'driver-package')]], + ['load?.resolve("…") for a bound loader', 'const load = createRequire(import.meta.url);\nexport const where = load?.resolve("driver-package");\n', [literal('bound-loader.resolve', 'load', 'driver-package')]], + ['import.meta.resolve?.("…")', 'export const tool = import.meta.resolve?.("tool-pkg");', [literal('import.meta.resolve', 'import.meta', 'tool-pkg')]], + ['createRequire(…)?.("…")', 'export const driver = createRequire(import.meta.url)?.("driver-package");', [literal('createRequire', 'createRequire', 'driver-package')]], + ['createRequire(…)?.resolve("…")', 'export const where = createRequire(import.meta.url)?.resolve("driver-package");', [literal('createRequire.resolve', 'createRequire', 'driver-package')]], + // A trailing comma after the one literal argument. + ['require("…",)', 'module.exports = require("left-pad",);', [literal('require', 'require', 'left-pad')]], + ['require.resolve("…", )', 'module.exports = require.resolve("left-pad", );', [literal('require.resolve', 'require', 'left-pad')]], + ['require("…", /* comment */)', 'module.exports = require("left-pad", /* was: "right-pad" */);', [literal('require', 'require', 'left-pad')]], + ['import.meta.resolve("…",)', 'export const tool = import.meta.resolve("tool-pkg",);', [literal('import.meta.resolve', 'import.meta', 'tool-pkg')]], + ['createRequire(…)("…",)', 'export const driver = createRequire(import.meta.url)("driver-package",);', [literal('createRequire', 'createRequire', 'driver-package')]], + ['load("…",) and load.resolve("…",) for a bound loader', 'const load = createRequire(import.meta.url);\nload("driver-package",);\nload.resolve("asset-pkg",);\n', [ + literal('bound-loader', 'load', 'driver-package'), + literal('bound-loader.resolve', 'load', 'asset-pkg'), + ]], + ['require rebound from createRequire, with every resolver', [ + 'const { createRequire } = await import("node:module");', + 'const require = createRequire(import.meta.url);', + 'const required = require("@scope/required/subpath");', + 'const asset = require.resolve("asset-pkg/package.json");', + 'const tool = import.meta.resolve("tool-pkg/bin/tool");', + String.raw`const hex = require("\x68ex-pkg");`, + String.raw`const unicode = require('unicode-pkg\u002fsubpath');`, + '// import { Function } from "effect" -- a comment never counts.', + ].join('\n'), [ + literal('require', 'require', '@scope/required/subpath'), + literal('require.resolve', 'require', 'asset-pkg/package.json'), + literal('import.meta.resolve', 'import.meta', 'tool-pkg/bin/tool'), + literal('require', 'require', 'hex-pkg'), + literal('require', 'require', 'unicode-pkg/subpath'), + ]], + ])('for %s', (_form, source, loads) => { + expect(scanModuleLoads(source)).toEqual(loads); + }); +}); + +describe('scanModuleLoads reports a computed load', () => { + it.each([ + ['require(x)', 'module.exports = (name) => require(name);', [computed('require', 'require')]], + ['require.resolve(x)', 'module.exports = (name) => require.resolve(name);', [computed('require.resolve', 'require')]], + ['import.meta.resolve(x)', 'export const where = (name) => import.meta.resolve(name);', [computed('import.meta.resolve', 'import.meta')]], + ['createRequire(…)(x)', 'import { createRequire } from "node:module";\nexport const load = (name) => createRequire(import.meta.url)(name);', [computed('createRequire', 'createRequire')]], + ['createRequire(…)(x) with a nested factory argument', 'import { createRequire } from "node:module";\nexport const any = (name) => createRequire(new URL("./entry.js", import.meta.url))(name);\n', [computed('createRequire', 'createRequire')]], + ['Module.createRequire(…)(x)', 'import * as Module from "node:module";\nexport const load = (name) => Module.createRequire(import.meta.url)(name);', [computed('createRequire', 'createRequire')]], + ['require("node:module").createRequire(…)(x)', 'module.exports = (name) => require("node:module").createRequire(__filename)(name);', [ + literal('require', 'require', 'node:module'), + computed('createRequire', 'createRequire'), + ]], + ['a bound loader called with an identifier', 'import * as Module from "node:module";\nconst load = Module.createRequire(import.meta.url);\nexport const any = (name) => load(name);\n', [computed('bound-loader', 'load')]], + ['a bound loader\'s .resolve called with an identifier', 'const load = createRequire(import.meta.url);\nexport const any = (name) => load.resolve(name);\n', [computed('bound-loader.resolve', 'load')]], + ['a literal-prefixed expression', 'module.exports = (variant) => require("chosen-at-runtime/" + variant);', [computed('require', 'require')]], + ['a template literal argument', 'module.exports = (variant) => require.resolve(`chosen-at-runtime/${variant}`);', [computed('require.resolve', 'require')]], + ['comment trivia before the argument', 'module.exports = (name) => require /* any */ (/* of */ name);\n', [computed('require', 'require')]], + ['a statement-position call after a literal one', 'require("./polyfill.js");\nrequire(pathOf(x));\n', [ + literal('require', 'require', './polyfill.js'), + computed('require', 'require'), + ]], + ['require?.(x)', 'module.exports = (name) => require?.(name);', [computed('require', 'require')]], + ['load?.resolve(x) for a bound loader', 'const load = createRequire(import.meta.url);\nexport const any = (name) => load?.resolve(name);\n', [computed('bound-loader.resolve', 'load')]], + ['two literal arguments', 'module.exports = require("left-pad", "right-pad");', [computed('require', 'require')]], + ['a parenthesised literal', 'module.exports = require(("left-pad"));', [computed('require', 'require')]], + // A template argument is a token like any template: its substitutions are code, its closing backtick closes it. + ['a static template literal argument', 'module.exports = require(`left-pad`);', [computed('require', 'require')]], + ['a template literal argument holding a load', 'module.exports = require(`${require("inner")}`);', [computed('require', 'require'), literal('require', 'require', 'inner')]], + ['a template literal argument before another template', 'require(`x`); require("after"); const t = `z`; require("last");\n', [ + computed('require', 'require'), + literal('require', 'require', 'after'), + literal('require', 'require', 'last'), + ]], + ])('for %s', (_form, source, loads) => { + expect(scanModuleLoads(source)).toEqual(loads); + }); +}); + +describe('scanModuleLoads reports a loader passed on as a value', () => { + it.each([ + ['const load = require;', 'const load = require;\nmodule.exports = load("chosen-at-runtime");', [reference('require', 'require')]], + ['fn(require)', 'module.exports = (fn) => fn(require);', [reference('require', 'require')]], + ['module.exports = require', 'module.exports = require', [reference('require', 'require')]], + ['[require]', 'module.exports = [require];', [reference('require', 'require')]], + ['{ require }', 'module.exports = { require };\nexport const pair = { other, require };\n', [reference('require', 'require'), reference('require', 'require')]], + ['{ key: require }', 'module.exports = { load: require };\n', [reference('require', 'require')]], + ['x ? require : y', 'module.exports = typeof require === "function" ? require : null;', [reference('require', 'require')]], + ['x ? y : require', 'module.exports = typeof require === "function" ? null : require;\n', [reference('require', 'require')]], + ['return require', 'function loader() {\n return require;\n}', [reference('require', 'require')]], + ['=> require', 'export const loader = () => require;', [reference('require', 'require')]], + ['fn(load) for a bound loader', 'import { createRequire } from "node:module";\nconst load = createRequire(import.meta.url);\nexport const use = (fn) => fn(load);', [reference('bound-loader', 'load')]], + ['return load for a bound loader', 'const load = createRequire(import.meta.url);\nfunction loader() { return load }\n', [reference('bound-loader', 'load')]], + ['=> load for a bound loader', 'const load = createRequire(import.meta.url);\nexport const loader = () => load;\n', [reference('bound-loader', 'load')]], + // Values that share a shape with a binding position, and are not one. + ['fn(a, require) and fn(require, b)', 'fn(a, require);\nfn(require, b);\n', [reference('require', 'require'), reference('require', 'require')]], + ['fn(require, callback) with a block body', 'register(require, function (id) { return id; });\nregister(require, () => { run(); });\n', [reference('require', 'require'), reference('require', 'require')]], + ['fn(require) as an if condition', 'if (accepts(require)) { run(); }\n', [reference('require', 'require')]], + ['use({ require }), an object literal argument', 'use({ require });\nuse({ other, require });\n', [reference('require', 'require'), reference('require', 'require')]], + ['const x = { require }, an object literal initialiser', 'const context = { require };\n', [reference('require', 'require')]], + ['{ require } returned before a later indexed assignment', 'function context() {\n return { require }\n}\ncache[0] = 1;\n', [reference('require', 'require')]], + ['[require] before a later array pattern', 'const list = [require];\nconst [first] = list;\n', [reference('require', 'require')]], + ['[require].map(…)', 'const names = [require].map((fn) => fn.name);\n', [reference('require', 'require')]], + ['a later declarator initialised with the loader', 'const a = b, load = require;\n', [reference('require', 'require')]], + ['x && require, x || require', 'const l = a && require;\nconst m = a || require;\n', [reference('require', 'require'), reference('require', 'require')]], + ['export { require }', 'const require = createRequire(import.meta.url);\nexport { require };\n', [reference('require', 'require')]], + ['fn(require) in a template substitution', 'const text = `${describe(require)}`;\n', [reference('require', 'require')]], + ['x ? require : y in a template substitution', 'const text = `${flag ? require : fallback}`;\n', [reference('require', 'require')]], + // A default initializer is a value the binding beside it receives, not a binding position. + ['function f(x = require) {', 'function f(x = require) { return x; }\n', [reference('require', 'require')]], + ['(x = load) => x for a bound loader', 'const load = createRequire(import.meta.url);\nconst g = (x = load) => x;\n', [reference('bound-loader', 'load')]], + ['const { x = require } = host', 'const { x = require } = host;\n', [reference('require', 'require')]], + ])('for %s', (_form, source, loads) => { + expect(scanModuleLoads(source)).toEqual(loads); + }); + + // The list walks that exclude a binding position read about a thousand characters past each loader name, so a list + // that names a loader thousands of times is scanned in time linear in its length, and past the bound every name is + // reported as a value: a 50 KB argument list, pattern, and declarator list each finish well inside a second. + it.each([ + ['an argument list', (names: string) => `fn(${names});\n`, 6000], + ['a destructuring pattern past the bound', (names: string) => `const { ${names} } = host;\n`, undefined], + ['a declarator list past the bound', (names: string) => `let a, ${names};\n`, undefined], + ])('scans %s naming require 6000 times in linear time', (_form, wrap, exact) => { + const source = wrap(Array.from({ length: 6000 }, () => 'require').join(', ')); + const started = performance.now(); + const loads = scanModuleLoads(source); + expect(performance.now() - started).toBeLessThan(1000); + expect(loads.every((load) => load.kind === 'reference')).toBe(true); + if (exact !== undefined) expect(loads).toHaveLength(exact); + else expect(loads.length).toBeGreaterThan(0); + }); +}); + +describe('scanModuleLoads reports nothing', () => { + it.each([ + ['typeof require', 'module.exports = typeof require;'], + ['the string "require"', 'module.exports = "require";'], + ['prose in comments', '/**\n * Use when a getter may fail, require\n * services, or run asynchronously.\n */\n// factory(module, require)\nmodule.exports = 1;'], + ['an express docblock', '/**\n * Module dependencies.\n * @private\n */\n\nvar debug = createDebug("express:router");\n// var depd = require("depd"); -- once\n'], + ['a bundler runtime named like require', 'const load = __webpack_require__;\nmodule.exports = load;'], + ['a longer identifier', 'var uri = require_fast_uri();\nvar u = __webpack_require__("./x");\n'], + ['path and Promise resolution', 'import path, { resolve } from "node:path";\nexport const f = (a, b) => [resolve(a, b), Promise.resolve(a), path.resolve("never-loaded"), Promise.resolve("never-loaded")];'], + ['ajv code in a template literal', 'code: (0, codegen_1._)`require("ajv/dist/runtime/validation_error").default`\n'], + ['ajv code in a string with escaped quotes', 'equal.code = "require(\\"ajv/dist/runtime/equal\\").default";\n'], + ['ajv code in a single-quoted string', "equal.code = 'require(\"ajv/dist/runtime/equal\").default';\n"], + ['code in a template whose substitution holds an object literal', 'const text = `${JSON.stringify({ a })} then require("ajv/dist/x") ${xs.map((x) => { return x; })}`;\n'], + ['a private #require method', 'class Store { #require(taskId) { return this.#records.get(taskId); }\n get(taskId) { const record = this.#require(taskId); return record; } }\n'], + ['a method named require on another object', 'const result = host.require(resolvedPath, pluginConfigEntry.name);\nsys.require = (baseDir, moduleName) => ({});\n'], + ['an object key named require', 'const sys = {\n base64encode: (input) => input,\n require: (baseDir, moduleName) => ({ baseDir, moduleName }),\n};\nconst conditions = { import: true, require: false };\nconst hooks = { require: fn };\n'], + ['a method and a function named require', 'class Host {\n require(id) { return this.modules.get(id); }\n}\nfunction require(id, parent) {\n return id;\n}\n'], + ['the Rspack missing-module stub', '!(function webpackMissingModule() { var e = new Error("Cannot find module \'left-pad\'"); e.code = \'MODULE_NOT_FOUND\'; throw e; }())\n'], + ['a regex literal holding a quote before a string with require', 'const quote = /["\']/u;\nconst example = "require(\'left-pad\')";\n'], + ['iconv prose in a comment and an error string', '// > iconv.enableStreamingAPI(require(\'stream\'));\nthrow new Error("Use iconv.enableStreamingAPI(require(\'stream\'))");\n'], + ['import.meta.url', 'const here = import.meta.url;\nconst dir = new URL(".", import.meta.url);\n'], + ['a dynamic import, which the lexer reports', 'const p = import("left-pad");\nconst q = import(name);\n'], + // Binding positions introduce a name; they pass no loader on. + ['a function parameter', 'function wrapper(module, exports, require) {}\nfunction first(require, module) {}\n'], + ['a function expression parameter', 'const wrapper = function (require) {};\nconst star = function* (module, require) {};\n'], + ['an arrow parameter list', 'const one = (require) => 1;\nconst two = (module, require) => 2;\nconst three = async (require) => {};\n'], + ['a bare arrow parameter', 'const one = require => 1;\n'], + ['a method parameter', 'const host = { load(require) { return 1; } };\nclass Host { run(require) {} }\n'], + ['a catch parameter', 'try { run(); } catch (require) {}\n'], + ['a bound loader as a parameter', 'const load = createRequire(import.meta.url);\nfunction wrap(load) {}\nconst arrow = (load) => 1;\n'], + ['an if, while, or switch head', 'if (require) {}\nwhile (require) {}\nswitch (require) {}\n'], + ['a declared object pattern', 'const { require } = host;\nlet { a, require: r } = host;\nvar { other, require } = host;\n'], + ['a declared object pattern, nested or renaming', 'const { a: { require } } = host;\nconst { a: require } = host;\n'], + ['a declared array pattern', 'const [require] = host;\nlet [a, require] = xs;\n'], + ['an assigned pattern', '({ require } = host);\n[require] = host;\n'], + ['a pattern parameter', 'function wrap({ require }) {}\nconst arrow = ({ require }) => 1;\nfor (const { require } of hosts) {}\n'], + ['a later declarator', 'let a, require;\nvar b = 1, require;\n'], + ['a bound loader in a pattern', 'const load = createRequire(import.meta.url);\nconst { load } = host;\n'], + ['an import specifier', 'import { require } from "./helper.js";\nimport { a, require } from "./helper.js";\nimport { require, a }\n from "./helper.js";\n'], + ['a re-export specifier', 'export { require } from "./helper.js";\n'], + // Binding names are read from code, never from a comment, a string, or a template. + ['a loader bound in a block comment', '/* const load = createRequire(import.meta.url); */\nfunction load(x) { return x; }\nload("left-pad");\n'], + ['a factory alias in a block comment', '/* { createRequire: mk } */\nmk(import.meta.url)("left-pad");\n'], + ['a factory alias in a line comment', '// import { createRequire as mk } from "node:module"\nmk(import.meta.url)("left-pad");\n'], + ['a loader bound in a string', 'const shim = "const load = createRequire(import.meta.url);";\nfunction load(x) { return x; }\nload("left-pad");\n'], + ['a loader bound in a template', 'const shim = `const load = createRequire(import.meta.url);`;\nload("left-pad");\n'], + ['a bound loader called in a comment', 'const load = createRequire(import.meta.url);\n/* load("left-pad") */\n'], + // Template quasis are text; an escaped `\${` is text; a whole-substitution loader name converts to a string. + ['a template whose quasis say require', 'const v = `require("x")`;\nconst w = `require("no") ${ok} require("no")`;\n'], + ['an escaped substitution', 'const v = `\\${require("x")}`;\n'], + ['a loader name as a whole substitution', 'const v = `${require}`;\n'], + ['an empty specifier', 'module.exports = require("");\n'], + ])('for %s', (_form, source) => { + expect(scanModuleLoads(source)).toEqual([]); + }); +}); + +/* + * A `/` directly after `)` is division, since `(a + b) / 2` is common in emitted code and `if (x) /re/` is not, so a + * regex written there is scanned as code: a load inside it is reported, and a quote inside it opens a string token + * that runs to the next quote on the line, hiding the code between. Both are pinned here as the approximation the + * module's header states. + */ +describe('scanModuleLoads reads a regex literal directly after `)` as code', () => { + it.each([ + ['a load written inside the regex is reported', 'if (enabled) /require("left-pad")/.test(text)\n', [literal('require', 'require', 'left-pad')]], + ['a quote inside the regex hides the rest of its line', 'if (x) /"/.test(y); require("hidden");\nrequire("next-line");\n', [literal('require', 'require', 'next-line')]], + ['division after `)` is code as it should be', 'const half = (a + b) / 2; require("left-pad");\n', [literal('require', 'require', 'left-pad')]], + ])('so %s', (_case, source, loads) => { + expect(scanModuleLoads(source)).toEqual(loads); + }); +}); + +describe('decodeLiteral', () => { + it.each([ + [String.raw`\x68ex-pkg`, 'hex-pkg'], + [String.raw`unicode-pkg\u002fsubpath`, 'unicode-pkg/subpath'], + [String.raw`\u{1F600}`, '\u{1F600}'], + [String.raw`a\nb\tc\0`, 'a\nb\tc\0'], + [String.raw`\/\'\"\\`, '/\'"\\'], + ['plain', 'plain'], + // Beyond Unicode the literal is a syntax error; the escape denotes nothing. + [String.raw`\u{110000}x`, 'x'], + ])('decodes %j to %j', (body, value) => { + expect(decodeLiteral(body)).toBe(value); + }); +}); + +it('quotedLiteral names its groups and numbers them 1 and 2 when it opens the expression', () => { + const match = new RegExp(quotedLiteral, 'u').exec(String.raw`x = "a\"b" + 'c'`); + expect(match?.groups).toEqual({ body: String.raw`a\"b`, quote: '"' }); + expect([match?.[1], match?.[2]]).toEqual(['"', String.raw`a\"b`]); + expect(new RegExp(quotedLiteral, 'u').exec("f('single')")?.groups?.body).toBe('single'); + // Nothing spans a newline or an empty body. + expect(new RegExp(quotedLiteral, 'u').exec('"a\nb"')).toBeNull(); + expect(new RegExp(quotedLiteral, 'u').exec('""')).toBeNull(); +}); + +it('remembers loads by digest so the same bytes are scanned once per process', () => { + const bytes = 'const load = createRequire(import.meta.url);\nload("driver-package");\n'; + const sha256 = sha256Hex(bytes); + const loads = scanModuleLoads(bytes, { sha256 }); + expect(loads).toEqual([literal('bound-loader', 'load', 'driver-package')]); + expect(Object.isFrozen(loads)).toBe(true); + expect(loads.every((load) => Object.isFrozen(load))).toBe(true); + expect(scanModuleLoads(bytes, { sha256 })).toBe(loads); + // The remembered value is the digest's, even from a different source string. + expect(scanModuleLoads('/* replaced */', { sha256 })).toBe(loads); + // Without a digest nothing is remembered. + const first = scanModuleLoads(bytes); + expect(first).toEqual(loads); + expect(first).not.toBe(loads); + expect(scanModuleLoads(bytes)).not.toBe(first); + expect(Object.isFrozen(first)).toBe(true); +}); diff --git a/packages/agent-bundle/tests/package-build.test.ts b/packages/agent-bundle/tests/package-build.test.ts index f6c12a24a..ee6bb1b1f 100644 --- a/packages/agent-bundle/tests/package-build.test.ts +++ b/packages/agent-bundle/tests/package-build.test.ts @@ -289,6 +289,100 @@ describe('framework-owned package build', () => { expect((await readdir(root)).filter((entry) => entry.startsWith('.dist.stage-'))).toEqual([]); }, 120_000); + it('fails the package build with AB6005 when node-commonjs externals reach dist through the createRequire shim', async () => { + // Under `externalsType: 'node-commonjs'` Rspack reaches an external not + // through an `import` but through the loader shim it emits into ESM output + // (`const __rspack_createRequire_require = __rspack_createRequire(import.meta.url)`), + // so no import record names `left-pad`: the load scan is what holds the line. + const root = await fixtureRoot({ + ...conventionFixture(), + 'agent-bundle.config.ts': [ + 'export default {', + ' lib: false,', + " plugin: { name: 'package-build-fixture', version: '1.0.0' },", + " targets: ['portable'],", + " tools: { rsbuild: { output: { externals: ['left-pad'] } }, rspack: { externalsType: 'node-commonjs' } },", + '};', + '', + ].join('\n'), + 'src/cli.ts': [ + "import leftPad from 'left-pad';", + '', + 'export const main = async (argv: readonly string[]): Promise => {', + " process.stdout.write(`${leftPad(argv.join(','), 8)}\\n`);", + ' return 0;', + '};', + '', + ].join('\n'), + }); + + const failure = await build({ output: 'artifact', packageOutputs: true, root }).then( + () => undefined, + (error: unknown) => error, + ); + expect(failure).toBeInstanceOf(DiagnosticError); + expect((failure as DiagnosticError).diagnostics).toEqual([{ + code: 'AB6005', + generatedPath: 'dist/bin/package-build-fixture.js', + message: 'Generated JavaScript import from "dist/bin/package-build-fixture.js" uses unsupported specifier "left-pad"' + + ' in __rspack_createRequire_require("left-pad"), a createRequire(…) loader.', + recovery: 'Bundle every JavaScript dependency into the artifact, then rebuild it.', + severity: 'error', + }]); + await expect(stat(join(root, 'dist'))).rejects.toMatchObject({ code: 'ENOENT' }); + expect((await readdir(root)).filter((entry) => entry.startsWith('.dist.stage-'))).toEqual([]); + }, 120_000); + + it('fails the package build with AB6005 when source loads a package through createRequire(), literal or computed', async () => { + // Neither call is an import, so the bundler never resolves `left-pad` (it + // is not installed) and both reach the emitted bin verbatim; the walk + // reports the literal one by specifier and the computed one as such, in + // source order. + const root = await fixtureRoot({ + ...conventionFixture(), + 'agent-bundle.config.ts': [ + 'export default {', + ' lib: false,', + " plugin: { name: 'package-build-fixture', version: '1.0.0' },", + " targets: ['portable'],", + '};', + '', + ].join('\n'), + 'src/cli.ts': [ + "import { createRequire } from 'node:module';", + '', + 'type Pad = (value: string, size: number) => string;', + '', + 'export const main = async (argv: readonly string[]): Promise => {', + " const literal = createRequire(import.meta.url)('left-pad') as Pad;", + " const chosen = createRequire(import.meta.url)(argv[0] ?? 'left-pad') as Pad;", + " process.stdout.write(`${literal('', 2)}${chosen('', 2)}\\n`);", + ' return 0;', + '};', + '', + ].join('\n'), + }); + + const failure = await build({ output: 'artifact', packageOutputs: true, root }).then( + () => undefined, + (error: unknown) => error, + ); + expect(failure).toBeInstanceOf(DiagnosticError); + const bin = 'dist/bin/package-build-fixture.js'; + const load = (detail: string) => ({ + code: 'AB6005', + generatedPath: bin, + message: `Generated JavaScript import from ${JSON.stringify(bin)} ${detail}`, + recovery: 'Bundle every JavaScript dependency into the artifact, then rebuild it.', + severity: 'error', + }); + expect((failure as DiagnosticError).diagnostics).toEqual([ + load('uses unsupported specifier "left-pad" in createRequire(…)("left-pad").'), + load('loads a non-literal specifier through createRequire(…)(…).'), + ]); + await expect(stat(join(root, 'dist'))).rejects.toMatchObject({ code: 'ENOENT' }); + }, 120_000); + it('accepts Node built-ins under node: and bare specifiers in dist bundles', async () => { const root = await fixtureRoot({ ...conventionFixture(), @@ -308,7 +402,10 @@ describe('framework-owned package build', () => { '', 'export const main = async (): Promise => {', ' const requireFromHere = createRequire(import.meta.url);', - " process.stdout.write(`${join('built', 'ins')}:${typeof readFileSync}:${typeof requireFromHere.resolve}\\n`);", + " // Prose naming require('left-pad') is a comment, never a load.", + " const os = requireFromHere('node:os') as { platform(): string };", + " process.stdout.write(`${join('built', 'ins')}:${typeof readFileSync}:${typeof requireFromHere.resolve}" + + ":${typeof os.platform}:${requireFromHere.resolve('fs')}:${import.meta.resolve('node:path')}\\n`);", ' return 0;', '};', '', @@ -318,7 +415,7 @@ describe('framework-owned package build', () => { expect(result.packageBuild?.files.map((file) => file.path)).toContain('bin/package-build-fixture.js'); const binPath = join(root, 'dist', 'bin', 'package-build-fixture.js'); - await expect(execFile(binPath, [])).resolves.toMatchObject({ stdout: 'built/ins:function:function\n' }); + await expect(execFile(binPath, [])).resolves.toMatchObject({ stdout: 'built/ins:function:function:function:fs:node:path\n' }); // Rspack keeps Node built-ins external, so the emitted module still // imports them by specifier — under both spellings — and the walker // accepts those imports (and `import.meta.url`) as it does relative ones. @@ -327,6 +424,14 @@ describe('framework-owned package build', () => { expect(binSource).toMatch(/from\s*["']node:module["']/u); expect(binSource).toMatch(/from\s*["'](?:node:)?path["']/u); expect(binSource).toContain('import.meta.url'); + // The built-in loads reach the emitted bin as written — a bound + // `createRequire()` loader, its `resolve`, and `import.meta.resolve` — + // and the walk accepts every one under either spelling, while the comment + // that names a package is stepped over rather than read as a load. + expect(binSource).toMatch(/requireFromHere\(["']node:os["']\)/u); + expect(binSource).toMatch(/requireFromHere\.resolve\(["']fs["']\)/u); + expect(binSource).toMatch(/import\.meta\.resolve\(["']node:path["']\)/u); + expect(binSource).toContain("require('left-pad')"); }, 120_000); it('keeps colocated tests out of the declaration program and the package output', async () => { diff --git a/packages/agent-bundle/tests/prepack.test.ts b/packages/agent-bundle/tests/prepack.test.ts index 57a7ec8c7..6a412ed41 100644 --- a/packages/agent-bundle/tests/prepack.test.ts +++ b/packages/agent-bundle/tests/prepack.test.ts @@ -884,10 +884,10 @@ const createSiblingProject = async ( return root; }; -it('accepts a dependency that only a prebuilt payload module imports: prepack passes, AB6005 does not walk copied files', async () => { +it('accepts a dependency that only a prebuilt payload module imports or requires: prepack passes, AB6005 does not walk prebuilt payloads', async () => { const root = await createSiblingProject('prebuilt-project', { bin: { 'prebuilt-fixture': './dist/bin/prebuilt-fixture.js' }, - dependencies: { express: '^5.0.0' }, + dependencies: { 'body-parser': '^2.0.0', cors: '^2.8.5', express: '^5.0.0' }, files: ['dist', 'host-packs', 'README.md'], name: 'prebuilt-fixture', type: 'module', @@ -903,9 +903,18 @@ it('accepts a dependency that only a prebuilt payload module imports: prepack pa " targets: ['cursor'],", '};', ], { - // A bare import in a module the framework copies rather than compiles: AB6005 never walks it, and the - // import is the usage evidence that keeps `express` out of AB7014. - 'built/runtime/mcp/server.js': 'import express from "express";\nexport default express;\n', + // A bare import, a `require()`, and a `require.resolve()` in a module the framework copies rather than + // compiles: AB6005 never walks it — the prebuilt payload stays opaque to the load scan as to the import + // walk — while the shared scanner still reads all three as the usage evidence that keeps `express`, + // `body-parser`, and `cors` out of AB7014. + 'built/runtime/mcp/server.js': [ + 'import express from "express";', + 'const body = require("body-parser");', + 'const where = require.resolve("cors");', + 'export default express;', + 'export { body, where };', + '', + ].join('\n'), 'src/index.ts': 'export const value = 1;\n', }); const packed = await prepack({ root }); diff --git a/packages/agent-bundle/tests/serve-app-command.test.ts b/packages/agent-bundle/tests/serve-app-command.test.ts index c15215187..6f2374b91 100644 --- a/packages/agent-bundle/tests/serve-app-command.test.ts +++ b/packages/agent-bundle/tests/serve-app-command.test.ts @@ -368,7 +368,7 @@ describe('locateFrameworkCli', () => { .resolves.toBe(join(root, 'node_modules/agent-bundle/bin/agent-bundle.js')); }); - it('walks the ancestor node_modules by hand when the package exports hide its manifest', async () => { + it('finds the manifest whether or not the package exports it: the walk never asks the resolver', async () => { const root = await temporaryDirectory(); await writeManifest(root, { bin: { 'agent-bundle': './bin/agent-bundle.js' }, diff --git a/packages/agent-bundle/tests/validate-artifact-modules.test.ts b/packages/agent-bundle/tests/validate-artifact-modules.test.ts new file mode 100644 index 000000000..63cf70dec --- /dev/null +++ b/packages/agent-bundle/tests/validate-artifact-modules.test.ts @@ -0,0 +1,386 @@ +import { mkdir, mkdtemp, realpath, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; + +import { afterEach, describe, expect, it } from '@rstest/core'; + +import { listArtifactFiles, type ArtifactFile } from '../src/build/emit.ts'; +import { validateJavaScriptModules } from '../src/build/validate-artifact-modules.ts'; +import type { Diagnostic } from '../src/core/diagnostics.ts'; + +const roots: string[] = []; + +afterEach(async () => { + await Promise.all(roots.splice(0).map((root) => rm(root, { force: true, recursive: true }))); +}); + +interface StagedTree { + readonly artifactRoot: string; + readonly files: readonly ArtifactFile[]; +} + +/** Writes `files` under a fresh root and lists them the way the artifact and package builds list a staged tree. */ +const stage = async (files: Readonly>): Promise => { + const artifactRoot = await realpath(await mkdtemp(join(tmpdir(), 'agent-bundle-validate-modules-'))); + roots.push(artifactRoot); + for (const [path, contents] of Object.entries(files)) { + const destination = join(artifactRoot, path); + await mkdir(dirname(destination), { recursive: true }); + await writeFile(destination, contents); + } + return { artifactRoot, files: await listArtifactFiles(artifactRoot) }; +}; + +const recovery = 'Bundle every JavaScript dependency into the artifact, then rebuild it.'; + +/** The one diagnostic the walk raises: AB6005 against `generatedPath`, with `detail` after the shared prefix. */ +const finding = (generatedPath: string, detail: string): Diagnostic => ({ + code: 'AB6005', + generatedPath, + message: `Generated JavaScript import from ${JSON.stringify(generatedPath)} ${detail}`, + recovery, + severity: 'error', +}); + +const probe = 'scripts/probe.mjs'; +const compiled: ReadonlySet = new Set([probe]); +const noJson: ReadonlySet = new Set(); + +/** Validates `source` staged as the compiled `scripts/probe.mjs`, alone or beside `siblings`. */ +const validateProbe = async ( + source: string, + siblings: Readonly> = {}, +): Promise => validateJavaScriptModules({ + ...await stage({ [probe]: source, ...siblings }), + bundledPaths: compiled, + validJson: noJson, +}); + +const factoryImport = 'import { createRequire } from "node:module";\n'; +const boundLoader = (declaration: 'const' | 'let' | 'var'): string => + `${factoryImport}${declaration} load = createRequire(import.meta.url);\n`; + +/** The loader shim Rspack emits into ESM output for a `node-commonjs` external or a bundled CommonJS `require()`. */ +const rspackShim = (specifier: string): string => [ + 'import { createRequire as __rspack_createRequire } from "node:module";', + 'const __rspack_createRequire_require = __rspack_createRequire(import.meta.url);', + `export const dep = __rspack_createRequire_require(${JSON.stringify(specifier)});`, + '', +].join('\n'); + +const unsupported = (call: string): string => `uses unsupported specifier "left-pad" in ${call}.`; +const computed = (call: string): string => `loads a non-literal specifier through ${call}.`; +const requireReference = 'passes require on as a value instead of calling it.'; +const boundReference = 'passes load, a createRequire(…) loader, on as a value instead of calling it.'; + +describe('validateJavaScriptModules', () => { + it.each([ + ['require()', 'export const pad = require("left-pad");\n', unsupported('require("left-pad")')], + [ + 'require() in a template substitution', + 'export const banner = `v${require("left-pad")}`;\n', + unsupported('require("left-pad")'), + ], + [ + 'require() after postfix increment and division', + 'const n = count++ / require("left-pad") / 2;\nexport { n };\n', + unsupported('require("left-pad")'), + ], + ['an optional require() call', 'export const pad = require?.("left-pad");\n', unsupported('require("left-pad")')], + ['require() with a trailing comma', 'export const pad = require("left-pad",);\n', unsupported('require("left-pad")')], + ['require.resolve()', 'export const where = require.resolve("left-pad");\n', unsupported('require.resolve("left-pad")')], + [ + 'a direct createRequire()() call', + `${factoryImport}export const pad = createRequire(import.meta.url)("left-pad");\n`, + unsupported('createRequire(…)("left-pad")'), + ], + [ + 'createRequire().resolve()', + `${factoryImport}export const where = createRequire(import.meta.url).resolve("left-pad");\n`, + unsupported('createRequire(…).resolve("left-pad")'), + ], + ['a const-bound loader', `${boundLoader('const')}export const pad = load("left-pad");\n`, unsupported('load("left-pad"), a createRequire(…) loader')], + ['a let-bound loader', `${boundLoader('let')}export const pad = load("left-pad");\n`, unsupported('load("left-pad"), a createRequire(…) loader')], + [ + "a var-bound loader's resolve()", + `${boundLoader('var')}export const where = load.resolve("left-pad");\n`, + unsupported('load.resolve("left-pad"), a createRequire(…) loader'), + ], + [ + 'an aliased factory', + 'import { createRequire as mk } from "node:module";\nexport const pad = mk(import.meta.url)("left-pad");\n', + unsupported('mk(…)("left-pad")'), + ], + [ + 'a namespace-qualified factory', + 'import * as Module from "node:module";\nexport const pad = Module.createRequire(import.meta.url)("left-pad");\n', + unsupported('createRequire(…)("left-pad")'), + ], + ["Rspack's createRequire shim", rspackShim('left-pad'), unsupported('__rspack_createRequire_require("left-pad"), a createRequire(…) loader')], + ['import.meta.resolve()', 'export const where = import.meta.resolve("left-pad");\n', unsupported('import.meta.resolve("left-pad")')], + // The specifier is decoded before it is judged and named: `\x6c` is `l`. + ['a hex-escaped require() literal', `${String.raw`export const pad = require("\x6ceft-pad");`}\n`, unsupported('require("left-pad")')], + ])('rejects a compiled module that loads a bare package through %s', async (_form, source, detail) => { + await expect(validateProbe(source)).resolves.toEqual([finding(probe, detail)]); + }); + + it.each([ + ['require()', 'export const load = (name) => require(name);\n', computed('require(…)')], + ['require.resolve()', 'export const where = (name) => require.resolve(name);\n', computed('require.resolve(…)')], + ['import.meta.resolve()', 'export const where = (name) => import.meta.resolve(name);\n', computed('import.meta.resolve(…)')], + [ + 'a direct createRequire()() call', + `${factoryImport}export const load = (name) => createRequire(import.meta.url)(name);\n`, + computed('createRequire(…)(…)'), + ], + [ + 'createRequire().resolve()', + `${factoryImport}export const where = (name) => createRequire(import.meta.url).resolve(name);\n`, + computed('createRequire(…).resolve(…)'), + ], + ['a bound loader', `${boundLoader('const')}export const any = (name) => load(name);\n`, computed('load(…), a createRequire(…) loader')], + [ + "a bound loader's resolve()", + `${boundLoader('const')}export const where = (name) => load.resolve(name);\n`, + computed('load.resolve(…), a createRequire(…) loader'), + ], + ['require() of a literal-prefixed expression', 'export const driver = (variant) => require("driver/" + variant);\n', computed('require(…)')], + ['require.resolve() of a template literal', 'export const driver = (variant) => require.resolve(`driver/${variant}`);\n', computed('require.resolve(…)')], + ])('rejects a compiled module whose %s argument is computed', async (_form, source, detail) => { + await expect(validateProbe(source)).resolves.toEqual([finding(probe, detail)]); + }); + + it.each([ + // `l` is no loader the scan knows, so its call raises nothing further. + ['const l = require;', 'const l = require;\nexport const pad = l("left-pad");\n', requireReference], + ['fn(load) with a bound loader', `${boundLoader('const')}export const use = (fn) => fn(load);\n`, boundReference], + ['[require]', 'export const loaders = [require];\n', requireReference], + ['{ key: require }', 'export const host = { key: require };\n', requireReference], + ['{ require }', 'export const host = { require };\n', requireReference], + ['x ? load : y', `${boundLoader('const')}export const pick = (x, y) => x ? load : y;\n`, boundReference], + ['return load', `${boundLoader('const')}export function loader() {\n return load;\n}\n`, boundReference], + ['=> load', `${boundLoader('const')}export const loader = () => load;\n`, boundReference], + ['a default initializer, function f(x = require) {', 'export function f(x = require) { return x; }\n', requireReference], + ['a default initializer in a pattern, const { x = load } = host', `${boundLoader('const')}const { x = load } = host;\nexport { x };\n`, boundReference], + ])('rejects a compiled module that passes %s on as a value', async (_form, source, detail) => { + await expect(validateProbe(source)).resolves.toEqual([finding(probe, detail)]); + }); + + it('names dist paths through reportedRoot the way the package build does', async () => { + await expect(validateJavaScriptModules({ + ...await stage({ 'bin/tool.js': 'export const pad = require("left-pad");\n' }), + bundledPaths: new Set(['bin/tool.js']), + reportedRoot: 'dist', + validJson: noJson, + })).resolves.toEqual([finding('dist/bin/tool.js', unsupported('require("left-pad")'))]); + }); + + it('accepts Node built-ins loaded through every resolver, under both spellings', async () => { + await expect(validateProbe([ + boundLoader('const').trimEnd(), + 'export const loaded = [', + ' require("node:fs"),', + ' require("fs"),', + ' require.resolve("path"),', + ' createRequire(import.meta.url)("node:util"),', + ' createRequire(import.meta.url).resolve("util"),', + ' load("node:os"),', + ' load.resolve("crypto"),', + ' import.meta.resolve("node:path"),', + String.raw` require("node:f\x73"),`, + '];', + '', + ].join('\n'), { 'scripts/shim.mjs': rspackShim('perf_hooks') })).resolves.toEqual([]); + }); + + it('resolves a relative literal load inside the tree and walks the target', async () => { + const helper = { 'scripts/helper.js': 'import "left-pad";\nexport const helper = true;\n' }; + await expect(validateProbe( + `${factoryImport}export const helper = createRequire(import.meta.url)("./helper.js");\n`, + helper, + )).resolves.toEqual([finding('scripts/helper.js', 'uses unsupported specifier "left-pad".')]); + + // The target is walked from the load, not merely as a root of its own: + // the entry sorts first, and its walk reports the helper's import before + // the entry's own later load. + await expect(validateJavaScriptModules({ + ...await stage({ + 'scripts/entry.mjs': `${boundLoader('const')}export const helper = load("./helper.js");\nexport const pad = load("right-pad");\n`, + ...helper, + }), + bundledPaths: new Set(['scripts/entry.mjs']), + validJson: noJson, + })).resolves.toEqual([ + finding('scripts/helper.js', 'uses unsupported specifier "left-pad".'), + finding('scripts/entry.mjs', 'uses unsupported specifier "right-pad" in load("right-pad"), a createRequire(…) loader.'), + ]); + }); + + it.each([ + ['import.meta.resolve()', 'export const where = import.meta.resolve("./helper.js");\n'], + ['require.resolve()', 'export const where = require.resolve("./helper.js");\n'], + ['hex-escaped require()', `${String.raw`export const helper = require("./hel\x70er.js");`}\n`], + ])('accepts a relative %s target that exists in the tree', async (_form, source) => { + // No `bundledPaths`: both modules are parsed in full, as copied ones are. + await expect(validateJavaScriptModules({ + ...await stage({ [probe]: source, 'scripts/helper.js': 'export const helper = true;\n' }), + validJson: noJson, + })).resolves.toEqual([]); + }); + + it('reports a relative load whose target is missing, or a JSON target not listed as valid', async () => { + const staged = await stage({ + [probe]: [ + 'export const missing = require("./missing.js");', + 'export const data = require("./data.json");', + 'export const outside = import.meta.resolve("../../outside.js");', + '', + ].join('\n'), + 'scripts/data.json': '{"ok":true}\n', + }); + await expect(validateJavaScriptModules({ ...staged, bundledPaths: compiled, validJson: noJson })).resolves.toEqual([ + finding(probe, 'is missing "./missing.js" in require("./missing.js").'), + finding(probe, 'references invalid JSON "./data.json" in require("./data.json").'), + finding(probe, 'resolves outside the artifact root: "../../outside.js" in import.meta.resolve("../../outside.js").'), + ]); + await expect(validateJavaScriptModules({ + ...staged, + bundledPaths: compiled, + validJson: new Set(['scripts/data.json']), + })).resolves.toEqual([ + finding(probe, 'is missing "./missing.js" in require("./missing.js").'), + finding(probe, 'resolves outside the artifact root: "../../outside.js" in import.meta.resolve("../../outside.js").'), + ]); + }); + + it('never scans a prebuilt payload module, even one a compiled module loads', async () => { + await expect(validateJavaScriptModules({ + ...await stage({ + [probe]: `${factoryImport}export const server = createRequire(import.meta.url)("./runtime/server.js");\n`, + 'scripts/runtime/server.js': [ + 'import express from "express";', + 'const parser = require(process.env.PARSER);', + 'export default express;', + 'export { parser };', + '', + ].join('\n'), + }), + bundledPaths: compiled, + prebuiltPaths: new Set(['scripts/runtime/server.js']), + validJson: noJson, + })).resolves.toEqual([]); + }); + + it.each>]>([ + ['a line comment', '// require("probe-dep") is prose\nexport const ok = true;\n'], + ['a bundled docblock', '/**\n * Use require("probe-dep") when the host lacks it.\n */\nexport const ok = true;\n'], + ['a string literal', 'export const text = \'require("probe-dep")\';\n'], + ['a template literal', 'export const code = `require("ajv/dist/runtime/equal").default`;\n'], + ['an escaped-quote string', `${String.raw`export const code = "require(\"ajv/dist/runtime/equal\").default";`}\n`], + ['a regex literal holding a quote before such a string', 'export const quote = /["\']/u;\nexport const example = "require(\'probe-dep\')";\n'], + [ + 'a bundler runtime named like require', + 'const __webpack_require__ = (id) => id;\nexport const mod = __webpack_require__("./node_modules/probe-dep/index.js");\n', + ], + [ + "Rspack's missing-module stub", + 'export const missing = Object(function webpackMissingModule() { var e = new Error("Cannot find module \'probe-dep\'"); e.code = \'MODULE_NOT_FOUND\'; throw e; }());\n', + ], + ['typeof require', 'export const cjs = typeof require === "function";\n'], + ['path and Promise resolution', 'import path from "node:path";\nexport const where = [path.resolve("probe-dep"), Promise.resolve("probe-dep")];\n'], + ['a loader bound but never called', `${boundLoader('const')}export const canResolve = typeof load.resolve === "function";\n`], + [ + 'a private #require method', + 'export class Store {\n #records = new Map();\n #require(id) { return this.#records.get(id); }\n get(id) { return this.#require(id); }\n}\n', + ], + ['a require method on another object', 'export const load = (host) => host.require("probe-dep");\n'], + ['an object key named require', 'export const conditions = { import: true, require: false };\nexport const sys = { require: (base, name) => ({ base, name }) };\n'], + [ + 'a method and a function definition named require', + 'export class Host {\n require(id) { return this.modules.get(id); }\n}\nfunction require(id, parent) {\n return id;\n}\n', + ], + [ + 'require binding positions', + [ + 'import { require } from "./helper.js";', + 'export function wrapper(module, exports, require) { return 1; }', + 'try { x(); } catch (require) {}', + '{ const { require } = host; }', + '', + ].join('\n'), + { 'scripts/helper.js': 'export const require = 1;\n' }, + ], + [ + 'a createRequire name inside a comment', + '/* const load = createRequire(import.meta.url); */\nfunction load(x) { return x; }\nload("left-pad");\n', + ], + ['a longer identifier', 'const require_fast_uri = () => "fast-uri";\nexport const uri = require_fast_uri();\n'], + ])('does not fail a compiled module for %s', async (_form, source, siblings = {}) => { + await expect(validateProbe(source, siblings)).resolves.toEqual([]); + }); + + it('finds the same loads whether a module is lexed as a bundle or parsed in full', async () => { + const staged = await stage({ + [probe]: [ + boundLoader('const').trimEnd(), + 'export const pad = load("left-pad");', + 'export const any = (name) => require(name);', + 'export const where = import.meta.resolve("right-pad");', + '', + ].join('\n'), + }); + const expected = [ + finding(probe, unsupported('load("left-pad"), a createRequire(…) loader')), + finding(probe, computed('require(…)')), + finding(probe, 'uses unsupported specifier "right-pad" in import.meta.resolve("right-pad").'), + ]; + await expect(validateJavaScriptModules({ ...staged, bundledPaths: compiled, validJson: noJson })).resolves.toEqual(expected); + await expect(validateJavaScriptModules({ ...staged, validJson: noJson })).resolves.toEqual(expected); + }); + + it("reports a module's import findings before its load findings, each in source order", async () => { + await expect(validateProbe([ + 'import "left-pad";', + 'import "right-pad";', + 'export const top = require("top-pad");', + 'export const bottom = require.resolve("bottom-pad");', + '', + ].join('\n'))).resolves.toEqual([ + finding(probe, 'uses unsupported specifier "left-pad".'), + finding(probe, 'uses unsupported specifier "right-pad".'), + finding(probe, 'uses unsupported specifier "top-pad" in require("top-pad").'), + finding(probe, 'uses unsupported specifier "bottom-pad" in require.resolve("bottom-pad").'), + ]); + }); +}); + +// The host-pack shapes `validateArtifact` hands the walk: a compiled +// (`bundle`) script under a target directory, a prebuilt payload module, and +// a copied one. `validateArtifact` maps the manifest kinds onto +// `bundledPaths` and `prebuiltPaths`; these cases hold the walk itself to the +// same findings over that layout. +describe('validateJavaScriptModules over a host-pack layout', () => { + const loader = 'custom/scripts/loader.mjs'; + + it.each([ + ['require()', 'export const pad = require("left-pad");\n', unsupported('require("left-pad")')], + ["Rspack's createRequire shim", rspackShim('left-pad'), unsupported('__rspack_createRequire_require("left-pad"), a createRequire(…) loader')], + ['import.meta.resolve()', 'export const where = import.meta.resolve("left-pad");\n', unsupported('import.meta.resolve("left-pad")')], + ['a computed require()', 'export const load = (name) => require(name);\n', computed('require(…)')], + ])('rejects a host-pack script that loads a package through %s', async (_form, source, detail) => { + await expect(validateJavaScriptModules({ + ...await stage({ 'custom/document.json': '{"kind":"custom"}\n', [loader]: source }), + bundledPaths: new Set([loader]), + validJson: new Set(['custom/document.json']), + })).resolves.toEqual([finding(loader, detail)]); + }); + + it('leaves a prebuilt payload module opaque to the load scan while a copied module is parsed', async () => { + const source = 'const express = require("express");\nexport default express;\n'; + await expect(validateJavaScriptModules({ + ...await stage({ 'custom/scripts/copied.mjs': source, 'custom/scripts/runtime/server.mjs': source }), + prebuiltPaths: new Set(['custom/scripts/runtime/server.mjs']), + validJson: noJson, + })).resolves.toEqual([finding('custom/scripts/copied.mjs', 'uses unsupported specifier "express" in require("express").')]); + }); +}); diff --git a/website/docs/en/guide/distribution/validation.mdx b/website/docs/en/guide/distribution/validation.mdx index 1453fb40b..ef9a22c8b 100644 --- a/website/docs/en/guide/distribution/validation.mdx +++ b/website/docs/en/guide/distribution/validation.mdx @@ -26,12 +26,34 @@ hand-edited generated file fails rather than passing because the path still exis files are checked too — a manifest-declared `logo` that is missing from the artifact or escapes the deploy tree reports `AB6025`. -Every emitted JavaScript module is walked as an ES module (`AB6005`) — the host-pack modules of +Every emitted `.js`/`.mjs` module is walked as an ES module (`AB6005`) — the host-pack modules of every artifact build, and the package build's `dist` bundles (`dist/bin/*.js`, a rendered route's -Flight worker `.mjs`, the `lib` entry) before `dist` is published: each import must be a literal -specifier that either names a Node built-in (`node:fs`, `fs`) or resolves — as a relative or -`file:` specifier — to a regular file inside the emitted tree (for a host pack, one the manifest -lists). No non-literal dynamic imports, no bare package names, nothing outside the tree; a `dist` +Flight worker `.mjs`, the `lib` entry) before `dist` is published. The scanner reads imports and +the recognised CommonJS-style loads: `require(…)`, `require.resolve(…)`, +`createRequire(…)(…)` and `.resolve(…)` with the factory written inline, namespace-qualified, or +imported/destructured under an alias, a loader declared with `const`/`let`/`var` from +`createRequire(…)` and then called directly or through `.resolve(…)`, and +`import.meta.resolve(…)`. Optional calls and a trailing comma after the literal count the same. +Examples of a named loader are `const load = createRequire(import.meta.url); load("…")` and the +shim Rspack emits for a `node-commonjs` external and for a bundled dependency's `require`. +Each specifier must be literal and either name a Node built-in (`node:fs`, `fs`) or resolve, as a +relative or `file:` specifier, to a listed regular `.js`/`.mjs` file inside the emitted tree; that +JavaScript target is walked in turn. In a host pack only, a relative target to listed valid JSON +is accepted as a terminal and is not walked. No non-literal dynamic imports or load arguments, no +bare package names, nothing outside the tree, and no `require` or loader used as a value instead +of called; parameters, `catch` bindings, destructuring patterns, and import specifiers are binding +positions rather than references, while a default initializer (`x = require`) is a reference. Comments, strings, and template text are stepped over, as are +regular-expression literals after an operator, `(`, `,`, `;`, `{`, `}`, `[`, `:`, `?`, `!`, or +a keyword such as `return`, `typeof`, or `case`; `${…}` template substitutions are code and are +scanned. Binding and alias names are read from code only, never from comments or strings. After +`)`, `]`, an identifier, a number, `++`, or `--`, `/` is treated as division and following text +is scanned as code, failing closed. Assignment-bound and second-hop loader aliases, +`.call`/`.apply`, `globalThis.require`, `module.require`, +`import.meta["resolve"]`, `require("")`, and a regex written directly after `)` or an identifier +are outside the recognised emitted forms. A +load finding names the call — `uses unsupported specifier "left-pad" in require("left-pad").`, +`loads a non-literal specifier through import.meta.resolve(…).`, +`passes load, a createRequire(…) loader, on as a value instead of calling it.` — and a `dist` finding names its file as `dist/`. How thoroughly a module's syntax is checked follows who produced its bytes. A module the framework compiled (manifest kind `bundle`) is the bundler's own output, so only the ESM lexer runs over it, which rejects unterminated strings, templates, @@ -194,13 +216,15 @@ exact package and artifact inventory, manifest hashes, package bin targets, rele agreement, and the installed-dependency fields of `package.json`. The build inlines every dependency into `dist/bin` and the host packs, so a published plugin should install nothing: declare the framework, `@agent-bundle/runtime`, `react`, `zod`, and the rest of the stack under -`devDependencies`. A compiled bundle — a host-pack module or a `dist` bundle alike — cannot import -a bare package at all: `AB6005` fails the build before `prepack` reaches the inventory. So -`dependencies` is only for what the packed files demonstrably need by other means — a prebuilt -payload module's import, a `require`, `createRequire`, or `import.meta.resolve` call in a packed -file (a call is not an import, so `AB6005` does not walk it), a packed declaration's reference, a -consumer install script, or a `bin` command a packed file runs — since only JavaScript the -framework did not compile can still import one. `--output` is an artifact path relative to `--root` that overrides the configured +`devDependencies`. An emitted module — a host-pack module, a copied artifact script, or a `dist` +bundle alike — cannot import, require, or resolve a bare package at all: `AB6005` fails the build +before `prepack` reaches the inventory. `AB7014` then lexes every packed `.js`/`.mjs`/`.cjs` +file, including `dist` and artifact files. Because `AB6005` has already refused a bare load in +every walked emitted module, the evidence that can still keep a dependency in a build that +passed comes from a prebuilt payload module, packed JavaScript the `files` allowlist adds from +outside the artifact and `dist`, a packed declaration reference, an install script, or a `bin` +command. +`--output` is an artifact path relative to `--root` that overrides the configured `output.distPath`, defaulting to `artifact`. Use it as an npm `prepack` script; `--ignore-scripts` prevents recursion, and no npm lifecycle ever performs a host install. @@ -211,19 +235,21 @@ and `splitChunks: false`, and the framework adds no `externals` of its own; Rsli leaves only Node built-ins (`node:fs`, `path`, plus Yarn PnP's `pnpapi`) external. MCP App views inline every script and style into one HTML file. Artifact validation holds the compiled host-pack bytes to the same line, and the package build holds its `dist` bundles to it before publishing -them: a bare import specifier that is not a Node built-in is `AB6005`, so a generated executable — in a -host pack or in `dist` — imports nothing but built-ins from outside its tree. An import kept -external through the [`tools` hatch](../../reference/configuration.mdx#tools) is not a way around -that: it fails the build in either output and never reaches the prepack inventory. A -`dependencies` entry is therefore only for what the packed files demonstrably need from outside a -compiled bundle — a package a prebuilt `.js`/`.mjs`/`.cjs` payload module imports (prebuilt files -are opaque to `AB6005`, and `AB7014` scans only those extensions, so an extensionless prebuilt -module counts for nothing), one a packed file loads through a `require`, `createRequire`, or -`import.meta.resolve` call (a call is not an import, so `AB6005` does not walk it in either -output), one a consumer-side install script runs, one whose `bin` a packed file executes, or one a -packed declaration file references (`.d.ts` outputs are not walked by `AB6005`) — and `AB7014` -reports a declared dependency with none of that evidence, while `AB7015` reports one a consumer's -npm cannot install. +them: a bare specifier that is not a Node built-in is `AB6005` whether it is imported, `require`d, +loaded through a `createRequire(…)` loader, or passed to `import.meta.resolve`, so a generated +executable — in a host pack or in `dist` — loads nothing but built-ins from outside its tree. A +dependency kept external through the [`tools` hatch](../../reference/configuration.mdx#tools) is +not a way around that in any emitted form: an ES `import` external and the `node-commonjs` +`createRequire` shim both fail the build in either output; a direct `require("pkg")` call, +however emitted, is rejected the same way. A `dependencies` entry is therefore only for evidence +that remains after that gate — a package a prebuilt `.js`/`.mjs`/`.cjs` payload module imports, +requires, or resolves (prebuilt files are opaque to `AB6005`, and `AB7014` scans only those extensions, so +an extensionless prebuilt module counts for nothing), one that JavaScript packed from outside the +artifact and `dist` imports or loads through a `require`, `createRequire`, or `import.meta.resolve` +call, one a consumer-side install script runs or loads from an inline `node -e` program, one whose +`bin` a packed file executes, or one a packed declaration file references (`.d.ts` outputs are not +walked by `AB6005`) — and `AB7014` reports a declared dependency with none of that evidence, while +`AB7015` reports one a consumer's npm cannot install. | Code | Meaning | | --- | --- | @@ -231,13 +257,15 @@ npm cannot install. | `AB7011` | An on-disk artifact file no longer matches its manifest SHA-256. Rebuild, and do not modify generated host packs. | | `AB7012` | A `package.json` bin points outside the packed `dist` output (including `src/`) or names a file npm omitted. Point it at the generated `dist/bin` file. | | `AB7013` | `package.json`, normalized plugin metadata, a host manifest, or artifact provenance reports a different release version. Make every release identity agree. | -| `AB7014` | A `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses — no packed JavaScript imports, requires, or resolves them (compiled bundles inline their imports and `AB6005` fails one they kept external, so `import` evidence comes from prebuilt payload modules and other scripts the framework did not compile, while `require`, `createRequire`, and `import.meta.resolve` calls count from any packed file), no packed declaration references them, no `imports` mapping or consumer install script reaches them (one diagnostic per field; optional peers are skipped here but their specifier is still checked by `AB7015`, and an `optionalDependencies` entry supersedes the same name under `dependencies`). Every consumer would install them for nothing. Move build-only packages to `devDependencies`. For `peerDependencies` this is a warning, since a required peer nothing imports may be a deliberate host-compatibility contract; mark it optional in `peerDependenciesMeta` if npm should stop installing it. | +| `AB7014` | A `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses. Every packed `.js`/`.mjs`/`.cjs` file is lexed, including `dist` and artifact files, but `AB6005` has already refused bare loads in walked emitted modules. In a passing build, remaining evidence comes from a prebuilt payload module, JavaScript packed from outside the artifact and `dist`, a packed declaration reference, an `imports` mapping, an install script, or a `bin` command. Optional peers are skipped here but their specifier is still checked by `AB7015`, and an `optionalDependencies` entry supersedes the same name under `dependencies`. Move build-only packages to `devDependencies`. For `peerDependencies` this is a warning, since a required peer nothing imports may be a deliberate host-compatibility contract; mark it optional in `peerDependenciesMeta` if npm should stop installing it. | | `AB7015` | A `dependencies`, `optionalDependencies`, or `peerDependencies` entry a consumer's npm cannot resolve through a registry. Name and specifier are read with `npm-package-arg`, npm's own parser, and come out as one of three kinds: registry (a version, range, dist-tag, or `npm:` alias of one), fetched (a git, GitHub-shorthand, remote-tarball, or path source — npm 12 refuses git and remote fetches by default (`allow-git`, `allow-remote`), and a path never exists on the consumer's disk), or unparseable (a name npm rejects, a scheme it lacks such as `link:`, `portal:`, or a typo, a selector that is neither a range nor a URL-safe dist-tag, an alias of a non-registry target, or an invalid URL — the manifest read itself fails, so this is reported even on an optional peer). A fetched `optionalDependencies` entry warns instead of failing, since npm continues without it; it stays an error when unparseable, or when a consumer install script needs the skipped package (runs its command — not merely mentions it — loads it from an inline `node -e` program — scanned for `require`, `createRequire`, and `import()` like a packed file — preloads it with `node -r`/`--require`/`--import`/`--loader`, or runs a packed file that imports it, `node .` running the root `main` included; each command after `&&`, `;`, or a newline is read on its own, shell quotes and backslash escapes resolved, and `node`'s options belong to `node` alone and end at the program — `node install.js --require x` preloads nothing, while a `NODE_OPTIONS=--require=x` assignment on the same `node` command does; every script word naming a packed JavaScript file counts as run, deliberately, so that runners the gate does not model — `tsx`, `zx`, `bun`, `deno run` — still have their file's dependencies traced, at the cost of a rare escalation for a word such as `echo install.js`, which the diagnostic makes visible by naming the file). A peer that `dependencies` or `optionalDependencies` also names is judged by that concrete entry; npm never reads the duplicate peer's selector. `workspace:`/`catalog:` are reported too unless the `prepack` lifecycle runs under pnpm, Yarn, or Bun, which rewrite them at pack time — `npm publish` does not, so under npm `AB7015` fires; entries the tarball itself carries — `bundleDependencies` npm actually packed (never peers), and a `file:` path inside the package whose packed source is installable (a directory with a parseable `package.json`, or a well-formed tarball whose `package.json` parses) — are not reported. Depend on a published version, or bundle the package and declare it under `devDependencies`. | The dependency evidence is read from the packed bytes: every `.js`/`.mjs`/`.cjs` file npm would publish is lexed for `import` specifiers and scanned for literal `require("…")`, `require.resolve("…")`, and -`import.meta.resolve("…")` calls (a `createRequire(…)` binding counts as `require`, even with the factory -renamed on import, as do direct `Module.createRequire(…)("…")` and `require("node:module").createRequire(…)("…")` calls; `path.resolve("…")` does not count), and every +`import.meta.resolve("…")` calls (a call through a `createRequire(…)` binding counts as +`require`, even with the factory renamed on import, as do direct +`Module.createRequire(…)("…")` and `require("node:module").createRequire(…)("…")` calls; +`path.resolve("…")` does not count), and every packed `.d.ts` is scanned for the modules its types reference (a `/// ` directive counts for the package and its `@types/*` twin); specifiers are reduced to package names, string escapes decoded first, with Node built-ins ignored. A dependency packed code runs as an executable — a string literal @@ -252,11 +280,16 @@ string-form `bin` is named after the installed manifest, read as npm reads it wins — and the unscoped name stands in when the dependency is not installed locally or its manifest is not JSON). A computed `import(expression)` or `require(expression)` (likewise `require.resolve`, `import.meta.resolve`, -a direct `createRequire(…)(…)`, or a `createRequire` binding) in packed code could load any declared -package, so it withholds `AB7014` altogether; so does packed source the ESM lexer rejects, whose `import()` +a direct `createRequire(…)(…)`, or a call through a loader bound from `createRequire(…)`) in packed code +`AB6005` never walked — an +emitted module with one has already failed the build — could load any declared package, so it withholds +`AB7014` altogether; so does packed source the ESM lexer rejects, whose `import()` calls it cannot report, and so does `require` passed on as a value (`const load = require`, -`fn(require)`) rather than called, since packages may then be loaded under a name the scan never sees. A mention inside a comment can only keep a dependency, never -report one; `devDependencies` are never inspected, and an `npm:` alias counts as a registry specifier when its target does. +`fn(require)`) rather than called, since packages may then be loaded under a name the scan never sees. +A `require` or import mention inside a comment, string, template text, or regular-expression +literal is neither evidence nor a load; only install-script command text and `bin` command strings +are text evidence. `devDependencies` are never inspected, and an `npm:` alias counts as a registry +specifier when its target does. A release build also refuses a project with **no** release version at all (`AB4013`), so a published artifact never carries the `0.0.0-dev.` development fallback. A diff --git a/website/docs/zh/guide/distribution/validation.mdx b/website/docs/zh/guide/distribution/validation.mdx index efc59e4f8..402472723 100644 --- a/website/docs/zh/guide/distribution/validation.mdx +++ b/website/docs/zh/guide/distribution/validation.mdx @@ -22,15 +22,33 @@ npx agent-bundle validate --artifact artifact --strict # 已构建字节， 把真实字节与这些摘要比对，因此被手工改过的生成文件会失败，而不会因为路径还在就通过。被引用的文件同样 会被检查——清单声明的 `logo` 若在产物中缺失或逃逸出部署树，会报告 `AB6025`。 -每个输出的 JavaScript 模块都会被当作 ES 模块遍历（`AB6005`）——既包括每次产物构建的宿主包模块，也包括 +每个输出的 `.js`/`.mjs` 模块都会被当作 ES 模块遍历（`AB6005`）——既包括每次产物构建的宿主包模块，也包括 包构建在发布 `dist` 之前的各个 `dist` bundle（`dist/bin/*.js`、渲染式路由的 Flight worker `.mjs`、`lib` -入口）：每个 import 必须是字面量说明符，要么指向 Node 内建模块（`node:fs`、`fs`），要么以相对或 `file:` -说明符解析到输出树内的常规文件（对宿主包而言，即清单中列出的文件）。不允许非字面量的动态 import，不允许 -裸包名，不允许指向树外；`dist` 中的发现以 `dist/` 点名其文件。模块语法检查的深度取决于它的字节由谁 +入口）。扫描器读取 import 与受识别的 CommonJS 式加载：`require(…)`、`require.resolve(…)`， +以及 `createRequire(…)(…)` 与 `.resolve(…)`；其中工厂可以内联写出、以命名空间限定，或通过 import/ +解构取别名。由 `const`/`let`/`var` 把 `createRequire(…)` 声明为加载器后，可以直接调用它或调用其 +`.resolve(…)`；`import.meta.resolve(…)` 也受识别。可选调用与字面量后的尾随逗号等效。命名加载器的示例包括 +`const load = createRequire(import.meta.url); load("…")`，以及 Rspack 为 `node-commonjs` 外部依赖和 +被打包依赖内部的 `require` 所生成的垫片。每个说明符都必须是字面量，并且要么指向 Node 内建模块 +（`node:fs`、`fs`），要么以相对或 `file:` 说明符解析到输出树内 +已列出的常规 `.js`/`.mjs` 文件；该 JavaScript 目标随后也会被遍历。仅在宿主包中，指向已列出合法 JSON +的相对目标会作为终点被接受，不会被遍历。不允许非字面量的动态 import 或加载参数，不允许裸包名，不允许指向树外， +也不允许把 `require` 或加载器当作值使用而不调用；参数、`catch` 绑定、解构模式与 import 说明符属于绑定 +位置而非引用，而默认值初始化（`x = require`）则算作引用。扫描器会跳过注释、字符串与模板静态文本，也会跳过位于运算符、`(`、`,`、`;`、`{`、`}`、 +`[`、`:`、`?`、`!` 或 `return`、`typeof`、`case` 等关键字之后的正则字面量；`${…}` 模板替换是代码， +仍会被扫描。绑定名与别名只从代码中读取，绝不从注释或字符串中读取。在 `)`、`]`、标识符、数字、`++` +或 `--` 之后，`/` 被当作除法，其后文本按代码扫描并以闭合失败处理。赋值绑定或第二跳加载器别名、 +`.call`/`.apply`、`globalThis.require`、 +`module.require`、`import.meta["resolve"]`、`require("")`，以及直接写在 `)` 或标识符之后的正则不在 +受识别的输出形式范围内。加载类的发现会点名该调用—— +`uses unsupported specifier "left-pad" in require("left-pad").`、 +`loads a non-literal specifier through import.meta.resolve(…).`、 +`passes load, a createRequire(…) loader, on as a value instead of calling it.`——`dist` 中的发现以 +`dist/` 点名其文件。模块语法检查的深度取决于它的字节由谁 产出。框架编译的模块（清单 kind 为 `bundle`）是打包器自己的输出，因此只由 ESM 词法分析器扫描，它会拒绝未 终止的字符串、模板、注释与正则以及不配对的花括号。框架没有编译的模块——被复制的消费者脚本、生成的安装器—— -则会被完整解析；若一次构建的 [`tools` 逃生口](../../reference/configuration.mdx#tools)有可能改写了输出资源， -该构建的每个 bundle 也会被完整解析，`dist` 与宿主包同理。预构建载荷（`kind: 'prebuilt'`）保持不透明，只做 +则会被完整解析；若一次构建的 [`tools` 逃生舱](../../reference/configuration.mdx#tools)有可能改写了输出资源， +该构建的每个 bundle 也会被完整解析，`dist` 与宿主包同理。预构建 payload（`kind: 'prebuilt'`）保持不透明，只做 哈希锁定；声明文件（`.d.ts`）不会被遍历。路由图会在打包器运行之前守住同一份自包含性： 路由模块、布局或 provider——或它们之一通过相对导入触达的模块——若值导入了携带编译器的框架入口 （`agent-bundle`、`agent-bundle/api`、`agent-bundle/config`、`agent-bundle/eval`、`agent-bundle/rstest`、 @@ -167,11 +185,13 @@ npx agent-bundle prepack --root . --output artifact --json `prepack` 运行发布构建与 `npm pack --dry-run --json --ignore-scripts`，随后对精确的包与产物清单、清单 哈希、包 bin 目标、发布版本一致性以及 `package.json` 中的安装期依赖字段把关。构建会把每个依赖内联进 `dist/bin` 与各宿主包，因此已发布的插件不应安装任何东西：请把框架、`@agent-bundle/runtime`、`react`、 -`zod` 以及其余技术栈都声明在 `devDependencies` 下。已编译的 bundle——无论是宿主包模块还是 `dist` bundle—— -根本不能导入裸包：`AB6005` 会在 `prepack` 触及清单之前就让构建失败。因此 `dependencies` 只留给打包后的 -文件以其他方式有据可证地需要的内容——预构建 payload 模块的导入、打包后文件中的 `require`、`createRequire` -或 `import.meta.resolve` 调用（调用不是导入，`AB6005` 不会遍历它）、打包后声明文件的引用、消费者侧安装脚本， -或打包后的文件运行的 `bin` 命令——因为只有框架没有编译的 JavaScript 才仍然可能导入它。`--output` 是相对 +`zod` 以及其余技术栈都声明在 `devDependencies` 下。输出的模块——无论是宿主包模块、被复制的产物脚本还是 +`dist` bundle——根本不能导入、require 或解析裸包：`AB6005` 会在 `prepack` 触及清单之前就让构建失败。 +随后 `AB7014` 会对每个打包后的 `.js`/`.mjs`/`.cjs` 文件做词法分析，包括 `dist` 与产物文件。由于 +`AB6005` 已在清单检查运行之前拒绝了每个已遍历输出模块中的裸加载，一个通过构建中仍能保留依赖的证据来自 +预构建 payload 模块、`files` 允许列表从产物与 `dist` 之外加入的打包后 JavaScript、打包后的声明引用、 +安装脚本或 `bin` 命令。 +`--output` 是相对 `--root` 的产物路径，会覆盖配置中的 `output.distPath`，默认值为 `artifact`。把它用作 npm 的 `prepack` 脚本；`--ignore-scripts` 可防止递归，而且任何 npm 生命周期都绝不会执行宿主安装。 @@ -180,15 +200,17 @@ CLI、MCP 入口、钩子包装层以及包构建的 JavaScript bundle——都 `splitChunks: false` 的配置编译，框架自身也不添加任何 `externals`；Rslib 的 `node` target 只把 Node 内建模块（`node:fs`、`path`， 以及 Yarn PnP 的 `pnpapi`）保持外部化。MCP App 视图则把每个脚本与样式都内联进同一个 HTML 文件。产物校验对 编译出的宿主包字节坚持同一条界线，包构建也在发布 `dist` bundle 之前对它们坚持同一条界线：任何不是 Node 内建 -模块的裸导入说明符即为 `AB6005`，因此生成的可执行文件——无论位于宿主包还是 `dist`——从自己的输出树之外导入的只有 -内建模块。通过 [`tools` 逃生舱](../../reference/configuration.mdx#tools)保持外部化的导入并不能绕过这条界线： -它在两种输出中都会让构建失败，永远到不了 prepack 的清单检查。因此，`dependencies` 条目只留给打包后的文件 -有据可证地需要从已编译 bundle 之外获取的内容——预构建的 `.js`/`.mjs`/`.cjs` payload 模块导入的包（预构建 -文件对 `AB6005` 不透明，而 `AB7014` 只扫描这些扩展名，因此无扩展名的预构建模块不算任何证据），打包后的 -文件通过 `require`、`createRequire` 或 `import.meta.resolve` 调用加载的包（调用不是导入，所以 `AB6005` 在两种 -输出中都不会遍历它），消费者侧安装脚本运行的包，打包后的文件执行其 `bin` 的包，或打包后的声明文件引用的包 -（`.d.ts` 输出不会被 `AB6005` 遍历）——`AB7014` 会报告不具备上述任何一种证据的已声明依赖，而 `AB7015` 会报告 -消费者的 npm 无法安装的依赖。 +模块的裸说明符即为 `AB6005`，无论它是被导入、被 `require`、经 `createRequire(…)` 加载器加载，还是传给 +`import.meta.resolve`，因此生成的可执行文件——无论位于宿主包还是 `dist`——从自己的输出树之外加载的只有 +内建模块。通过 [`tools` 逃生舱](../../reference/configuration.mdx#tools)保持外部化的依赖，无论以何种输出形式 +都不能绕过这条界线：ES `import` 形式的外部依赖与 `node-commonjs` `createRequire` 垫片在两种输出中都会 +让构建失败；无论如何输出，直接的 `require("pkg")` 调用也会以同样方式被拒绝。因此，`dependencies` 条目 +只留给通过该门禁后仍存在的证据——预构建的 `.js`/`.mjs`/`.cjs` payload 模块导入、require 或解析的包 +（预构建文件对 `AB6005` 不透明，而 `AB7014` 只扫描这些扩展名，因此无扩展名的预构建模块不算任何证据）， +从产物与 `dist` 之外打包进来的 JavaScript 导入或通过 `require`、`createRequire`、`import.meta.resolve` +调用加载的包，消费者侧安装脚本运行或在内联 `node -e` 程序中加载的包，打包后的文件执行其 `bin` 的包，或 +打包后的声明文件引用的包（`.d.ts` 输出不会被 `AB6005` 遍历）——`AB7014` 会报告不具备上述任何一种证据的 +已声明依赖，而 `AB7015` 会报告消费者的 npm 无法安装的依赖。 | 代码 | 含义 | | --- | --- | @@ -196,14 +218,23 @@ CLI、MCP 入口、钩子包装层以及包构建的 JavaScript bundle——都 | `AB7011` | 磁盘上的某个产物文件与其清单 SHA-256 不再匹配。请重新构建，且不要修改生成的宿主包。 | | `AB7012` | 某个 `package.json` bin 指向了打包后的 `dist` 输出之外（包括 `src/`），或指名了一个被 npm 忽略的文件。请把它指向生成的 `dist/bin` 文件。 | | `AB7013` | `package.json`、规范化后的插件元数据、某份宿主清单或产物 provenance 报告了不同的发布版本。请让每处发布标识一致。 | -| `AB7014` | `dependencies`、`optionalDependencies` 或 `peerDependencies` 字段里列出的包在整个包里无人使用——没有打包后的 JavaScript 导入、require 或解析它（已编译的 bundle 会内联自己的导入，被保持外部化的导入会被 `AB6005` 判为失败，因此 `import` 证据只来自预构建 payload 模块以及其他框架没有编译的脚本，而 `require`、`createRequire` 与 `import.meta.resolve` 调用则可来自任何打包后的文件），没有打包后的声明文件引用它，也没有 `imports` 映射或消费者侧安装脚本触及它（每个字段一条诊断；可选 peer 在此被跳过，但其说明符仍由 `AB7015` 检查，`optionalDependencies` 中的同名项优先于 `dependencies`）。每位消费者都会白白安装它们。请把仅构建期需要的包移到 `devDependencies`。对 `peerDependencies` 而言这是警告：无人导入的必需 peer 可能是有意的宿主兼容性约束；若希望 npm 不再安装它，请在 `peerDependenciesMeta` 中标记为可选。 | +| `AB7014` | `dependencies`、`optionalDependencies` 或 `peerDependencies` 字段里列出的包在整个包里无人使用。每个打包后的 `.js`/`.mjs`/`.cjs` 文件都会被词法分析，包括 `dist` 与产物文件；但 `AB6005` 已拒绝已遍历输出模块中的裸加载。在通过的构建中，剩余证据来自预构建 payload 模块、从产物与 `dist` 之外打包进来的 JavaScript、打包后的声明引用、`imports` 映射、安装脚本或 `bin` 命令。可选 peer 在此被跳过，但其说明符仍由 `AB7015` 检查，`optionalDependencies` 中的同名项优先于 `dependencies`。请把仅构建期需要的包移到 `devDependencies`。对 `peerDependencies` 而言这是警告：无人导入的必需 peer 可能是有意的宿主兼容性约束；若希望 npm 不再安装它，请在 `peerDependenciesMeta` 中标记为可选。 | | `AB7015` | `dependencies`、`optionalDependencies` 或 `peerDependencies` 中的某一项无法被消费者的 npm 经由注册表解析。包名与说明符一并交给 npm 自己的解析器 `npm-package-arg` 读取，结果分为三类：注册表类（版本、范围、dist-tag，或指向它们的 `npm:` 别名）、抓取类（git、GitHub 简写、远程 tarball 或路径来源——npm 12 默认拒绝 git 与远程抓取（`allow-git`、`allow-remote`），而路径在消费者磁盘上并不存在），以及不可解析类（npm 不接受的包名、它不支持的协议如 `link:`、`portal:` 或拼写错误、既非范围也非 URL 安全 dist-tag 的选择器、指向非注册表目标的别名，或非法 URL——清单本身就读不下去，因此即使出现在可选 peer 上也会被报告）。`optionalDependencies` 中的抓取类项只发出警告而不会失败，因为 npm 会继续安装；不可解析的项，或消费者安装脚本需要的项（运行其命令——而非仅仅提及它——在内联的 `node -e` 程序中加载它——该程序像已打包文件一样被扫描 `require`、`createRequire` 与 `import()`——用 `node -r`/`--require`/`--import`/`--loader` 预加载它，或运行某个导入它的已打包文件，包括通过根 `main` 运行的 `node .`；`&&`、`;` 或换行之后的每条命令单独判读，shell 引号与反斜杠转义均已解析，`node` 的选项只属于 `node` 且到程序为止——`node install.js --require x` 不会预加载任何东西，而同一条 `node` 命令上的 `NODE_OPTIONS=--require=x` 赋值会；脚本中每个点名已打包 JavaScript 文件的词都视为被运行，这是有意为之，使本闸门未建模的运行器——`tsx`、`zx`、`bun`、`deno run`——所运行文件的依赖仍被追踪，代价是像 `echo install.js` 这样的词偶有误升级，诊断会点名该文件使之可见），仍是错误。若 `dependencies` 或 `optionalDependencies` 也声明了同名 peer，则只按该具体条目判断；npm 从不读取重复 peer 的选择器。`workspace:`/`catalog:` 同样会被报告，除非 `prepack` 生命周期运行在 pnpm、Yarn 或 Bun 之下——它们会在打包时重写这些协议；`npm publish` 不会重写，因此在 npm 下 `AB7015` 会触发；tarball 自身携带的项——`bundleDependencies` 中 npm 实际打包进去的项（peer 除外），以及已打包来源可供安装的包内 `file:` 路径（带有可解析 `package.json` 的目录，或格式正确且其 `package.json` 可解析的 tarball）——不会被报告。请依赖已发布的版本，或把该包打包进产物并声明在 `devDependencies` 下。 | 依赖证据直接读取自打包后的字节：npm 将发布的每个 `.js`/`.mjs`/`.cjs` 文件都会被词法分析出 `import` -说明符，并扫描字面量 `require("…")`、`require.resolve("…")` 与 `import.meta.resolve("…")` 调用（`createRequire(…)` 的绑定视同 `require`，即使导入时重命名了该工厂，直接调用的 `Module.createRequire(…)("…")` 与 `require("node:module").createRequire(…)("…")` 亦然；`path.resolve("…")` 不计入）；每个打包后的 `.d.ts` 会被扫描其类型所引用的模块（`/// ` 指令同时计入该包及其 +说明符，并扫描字面量 `require("…")`、`require.resolve("…")` 与 `import.meta.resolve("…")` 调用 +（通过 `createRequire(…)` 绑定进行的调用视同 `require`，即使导入时重命名了该工厂，直接调用的 +`Module.createRequire(…)("…")` 与 `require("node:module").createRequire(…)("…")` 亦然； +`path.resolve("…")` 不计入）；每个打包后的 `.d.ts` 会被扫描其类型所引用的模块（`/// ` 指令同时计入该包及其 `@types/*` 对应包）。说明符先解码字符串转义再归约为包名，忽略 Node 内建模块。打包代码作为可执行文件运行的依赖——字符串字面量正是其已安装清单所声明的某个 `bin` 命令，如 `spawnSync("tsc", ["--version"])`——视为已使用；`#子路径` 导入计入 `imports` 映射所指向的每个包；消费者侧 `preinstall`/`install`/`postinstall` 脚本（不含 `prepare`——npm 从不为已发布的 tarball 运行它）或其通过 `npm run`（取 `run` 之后的第一个位置参数；`run` 前后的选项，带值与否均被跳过，其后的词如 `npm run setup -- dormant` 中的 `dormant` 是该脚本的参数）或直接的 `npm test`/`start`/`stop`/`restart`（没有 `restart` 脚本时，`npm restart` 依次运行 `stop` 与 `start`）委托的任何脚本点名（或运行其 `bin` 命令；字符串形式的 `bin` 以已安装清单的名字命名——清单按 npm 的方式读取，重复键以最后一个为准——依赖未在本地安装或其清单不是 JSON 时以去掉作用域的包名代替）的依赖 -同样视为已使用。打包代码中的计算型 `import(表达式)` 或 `require(表达式)`（同样包括 `require.resolve`、`import.meta.resolve`、直接的 `createRequire(…)(…)` 或 `createRequire` 绑定）可能加载任何已声明的包，因此会整体撤回 `AB7014`；被 ESM 词法分析器拒绝的打包源码亦然，因为其中的 `import()` 调用无法被报告；把 `require` 当作值传递而非调用（`const load = require`、`fn(require)`）亦然，因为此后包可能以扫描看不到的名字被加载。注释中的提及只会保留某个依赖，绝不会报告它；`devDependencies` 永不检查， -`npm:` 别名在其目标是注册表说明符时才视为注册表说明符。 +同样视为已使用。`AB6005` 从未遍历的打包代码中的计算型 `import(表达式)` 或 `require(表达式)`（同样包括 +`require.resolve`、`import.meta.resolve`、直接的 `createRequire(…)(…)`，或通过由 +`createRequire(…)` 绑定的加载器进行的调用；带有这类调用的输出模块早已让构建失败）可能加载任何已声明的包， +因此会整体撤回 `AB7014`；被 ESM 词法分析器拒绝的打包源码亦然，因为其中的 `import()` 调用无法被报告； +把 `require` 当作值传递而非调用（`const load = require`、`fn(require)`）亦然，因为此后包可能以扫描看不到的 +名字被加载。注释、字符串、模板静态文本或正则字面量中的 `require` 或 import 提及既不是证据，也不是加载； +只有安装脚本命令文本与 `bin` 命令字符串属于文本证据。`devDependencies` 永不检查，`npm:` 别名在其目标是 +注册表说明符时才视为注册表说明符。 发布构建同样会拒绝**完全没有**发布版本的项目（`AB4013`），因此已发布的产物绝不会携带 `0.0.0-dev.` 这个开发期回退值。声明的 `plugin.version` 与 `package.json` 不一致时会