diff --git a/.changeset/619-define-prebuilt.md b/.changeset/619-define-prebuilt.md new file mode 100644 index 000000000..860ed43a9 --- /dev/null +++ b/.changeset/619-define-prebuilt.md @@ -0,0 +1,11 @@ +--- +"agent-bundle": minor +--- + +Add `definePrebuilt` to `agent-bundle` and `agent-bundle/config`, with a +`runtimeDependencies` field for the bare package names a prebuilt payload +loads. Report a malformed list as `AB4740`, and as `AB4751` a name npm does +not read as a bare package name or one `package.json` does not install for a +consumer (`dependencies`, `optionalDependencies`, or a peer not marked +optional); count declared runtime dependencies as used for `AB7014` and expose +them as `NormalizedPayload.runtimeDependencies`. (#630) diff --git a/docs/diagnostics.md b/docs/diagnostics.md index 8beadb9cb..6becc7620 100644 --- a/docs/diagnostics.md +++ b/docs/diagnostics.md @@ -25,7 +25,7 @@ even when no error diagnostic was reported. | `AB471x` | Package build `lib` configuration (`AB4710`–`AB4715`) and declaration generation (`AB4716`); see below. | | `AB472x` | The `tools.rsbuild` / `tools.rspack` escape hatch (`AB4720`–`AB4723`: shape; `AB4724`: a framework-owned Rsbuild plugin re-added through `tools.rsbuild.plugins`; `AB4725`: `tools` externalizes a non-built-in; see below). | | `AB473x` | Migration nudges (informational; see below). | -| `AB474x`/`AB4750` | Prebuilt payloads and prebuilt entries (see below). | +| `AB4740`–`AB4751` | Prebuilt payloads and prebuilt entries (see below). | | `AB4760` | The published `agent-bundle/meta` identity module evaluated outside every compiled surface and outside the Rstest presets (see below). | | `AB4765`–`AB4766` | Artifact-hosted routed CLI: a target without the `cli` capability omits `bin/.mjs`; a host-emitted file collides with it (see below). | | `AB477x` | MCP App view compilation (`AB4770`: compile error with file, line, column and the bundler message; `AB4771`: compile warning; `AB4772`: emitted-size advisory; see below). | @@ -426,7 +426,7 @@ Validation happens at three moments, all fail-closed: | `AB7011` | An on-disk artifact file no longer matches its manifest SHA-256. Rebuild and do not modify generated host packs. | | `AB7012` | A `package.json` bin points outside the packed `dist` output (including `src/`) or names a file npm omitted. Point it at the generated `dist/bin` file. | | `AB7013` | `package.json`, normalized plugin metadata, a host manifest, or artifact provenance reports a different release version. Make every release identity agree. | -| `AB7014` | A `package.json` `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses: no packed JavaScript imports, requires, or resolves them, or runs one of their `bin` commands, no packed declaration file references them, no `#subpath` import reaches them through the manifest's `imports` map, and no consumer-side install script (or script it delegates to) runs them (one diagnostic per field; the full evidence list follows this table). Peers `peerDependenciesMeta` marks optional are never installed and are not inspected here (their specifier is still checked by `AB7015`), and a name under both `dependencies` and `optionalDependencies` is judged by its optional entry, which npm lets override. The build inlines every dependency into `dist` and the host packs, and `AB6005` fails a compiled bundle that imports a bare specifier other than a Node built-in (`prepack` runs that build before this inventory), so a compiled bundle can never supply `import` evidence here and such an entry only makes every consumer's `npm install` fetch build-time packages; the packed-JavaScript `import` evidence class is for modules the framework copied rather than compiled — prebuilt payload modules and other packed scripts — while `require`, `createRequire`, and `import.meta.resolve` calls, which `AB6005` does not walk, count from any packed file. Move build-only packages to `devDependencies`; keep a runtime dependency only for what a prebuilt payload module imports, a packed file requires or resolves, a packed declaration references, a `#subpath` import reaches, or an install script or packed file runs. For `peerDependencies` the diagnostic is a warning: a required peer nothing imports may be a deliberate compatibility contract with the host that loads the package, though npm 7+ still installs it for every consumer — keep it, mark it optional in `peerDependenciesMeta`, or move a build-only package to `devDependencies`. | +| `AB7014` | A `package.json` `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses: no packed JavaScript imports, requires, or resolves them, or runs one of their `bin` commands, no packed declaration file references them, no `#subpath` import reaches them through the manifest's `imports` map, and no consumer-side install script (or script it delegates to) runs them, and no prebuilt payload names them in `runtimeDependencies` (one diagnostic per field; the full evidence list follows this table). Peers `peerDependenciesMeta` marks optional are never installed and are not inspected here (their specifier is still checked by `AB7015`), and a name under both `dependencies` and `optionalDependencies` is judged by its optional entry, which npm lets override. The build inlines every dependency into `dist` and the host packs, and `AB6005` fails a compiled bundle that imports a bare specifier other than a Node built-in (`prepack` runs that build before this inventory), so a compiled bundle can never supply `import` evidence here and such an entry only makes every consumer's `npm install` fetch build-time packages; the packed-JavaScript `import` evidence class is for modules the framework copied rather than compiled — prebuilt payload modules and other packed scripts — while `require`, `createRequire`, and `import.meta.resolve` calls, which `AB6005` does not walk, count from any packed file. Move build-only packages to `devDependencies`; keep a runtime dependency only for what a prebuilt payload module imports, a packed file requires or resolves, a packed declaration references, a `#subpath` import reaches, an install script or packed file runs, or a prebuilt payload names in `runtimeDependencies` (`definePrebuilt`). For `peerDependencies` the diagnostic is a warning: a required peer nothing imports may be a deliberate compatibility contract with the host that loads the package, though npm 7+ still installs it for every consumer — keep it, mark it optional in `peerDependenciesMeta`, or move a build-only package to `devDependencies`. | | `AB7015` | A `package.json` `dependencies`, `optionalDependencies`, or `peerDependencies` entry that a consumer's npm cannot resolve through a registry. Each entry — name and specifier together, the value exactly as written (a leading space makes `" npm:bar@1"` an invalid dist-tag, not an alias) — is read with `npm-package-arg`, the parser npm, Arborist, and pacote share, so the verdict is npm's own rather than an imitation of its grammar: **registry** (a version, range, or dist-tag, or an `npm:` alias of one — the only kind a published package can rely on), **fetched** (parseable, but a `git`/`github:`/`gitlab:`/`bitbucket:`/`gist:` source or `owner/repo` shorthand, an `http(s):` tarball, or a `file:`/relative/bare path or tarball filename — npm 12 refuses git and remote fetches by default (`allow-git=none`, `allow-remote=none`) and a path never exists on the consumer's disk), or **unparseable** (npm rejects the manifest before fetching anything: `EINVALIDPACKAGENAME` for a name such as `bad name`, `.hidden`, or `node_modules`; `EUNSUPPORTEDPROTOCOL` for `link:`, `portal:`, `jsr:`, a `git+` transport npm lacks, or a typo; `EINVALIDTAGNAME` for a selector that is neither a range nor a URL-safe dist-tag, such as `"not a valid spec"`; an `npm:` alias without a name or with a non-registry target, since aliases only work for registry dependencies; or an invalid URL such as `http:%zz`). A fetched specifier is reported on installed entries only; an unparseable one is reported on every entry, even an optional peer npm would never install, because the manifest read itself fails. A peer that `dependencies` or `optionalDependencies` also names is judged by that concrete entry alone: npm resolves the concrete declaration and never reads the duplicate peer's selector. For an `optionalDependencies` entry that is fetched, the diagnostic is a warning, not an error (`agent-bundle prepack` prints it and exits 0): npm continues an install without such a dependency, but every consumer still tries and fails to fetch it. It stays an error when the entry is unparseable, or when a consumer-side install script needs the skipped package — runs one of its `bin` commands in command position (`setup-tool --init`, `npx setup-tool`, `cross-env CI=1 setup-tool`, `./node_modules/.bin/setup-tool`; a mention elsewhere, `echo setup-tool`, proves nothing), runs one of its files (`node node_modules/setup-tool/install.js`), loads it from an inline program (`node -e "require('setup-tool')"`, `node --input-type=module -e "await import('setup-tool')"`, also `-p`, `-pe`, `--eval=…`, `--print=…`; the program is read as a packed file is — `require`, `createRequire`, and `import()` — and a computed load there, or a program the lexer rejects, may need any declared package), preloads it (`node -r setup-tool/register install.js`; `-r`/`--require`, `--import`, `--loader`/`--experimental-loader`, with a space or `=` before the module — read as Node does, `node [options] script [arguments]`: options end at the first positional (the script, or an argument when `-e`/`-p` supply the program) or a `--`, valued options such as `--conditions x` or `--env-file x` taking their word with them, so `node install.js --require x` passes `--require x` to `install.js` and preloads nothing; a `NODE_OPTIONS` assignment on the same command — `NODE_OPTIONS=--require=setup-tool/register node install.js`, `cross-env NODE_OPTIONS="-r setup-tool/register" node .` — supplies options Node applies before the command line's, while one `export`ed by an earlier command is not read), or runs a packed file (`node install.cjs`, `node scripts/install` resolving `scripts/install.js`, `node "scripts/my install.cjs"`, `node install.js&&echo done`, `node .` or `node ./` running the root `main`, `node --import ./setup.mjs .` running a packed preload) that imports it — every word of the script that names a packed JavaScript file counts as run, deliberately, so that runners this gate does not model (`tsx`, `ts-node`, `zx`, `bun`, `deno run`, `npx `) still have the dependencies their file loads traced; the cost is a rare escalation for a word that names a packed file without running it (`echo install.js`), which the diagnostic makes visible by naming the file — directly, through relative imports inside the tarball (`require("./lib")` following `lib/package.json`'s `main` before `lib/index.js`, as Node does), or through the `imports` map resolved as Node does (`"#setup": "./setup.js"`; `#setup/foo` through `"#setup/*": "./scripts/*.js"`, a preloaded `#setup` included): npm continues past the failed fetch, then the script fails on the missing command or module. Each command of a script is read on its own: after a shell operator (`&&`, `;`, and the rest) or a newline — the second line of a script, and each lifecycle script after the first, starts a new command — and Node's options belong to `node` alone (`rm -r dist` preloads nothing, `npm --prefix . run setup` runs no `main`). Depend on a published registry version, or bundle the package and declare it under `devDependencies`. Entries the tarball itself carries are never reported, since a consumer does not fetch them: `bundleDependencies` (by name or `true`; never a peer, which npm cannot bundle; only when the pack inventory contains `node_modules//package.json`, since npm silently packs nothing for a bundled name absent from `node_modules`), and a `file:` or bare path inside the package (`file:vendor/foo`, `file:vendor/foo.tgz`) whose packed source npm can install from — a directory whose packed `package.json` parses to an object, or a packed tarball (gzipped or plain tar, ustar headers with valid checksums and payloads inside the archive) whose `/package.json` entry parses to an object — since npm installs it from the consumer's own copy. A path that escapes the package (`file:../sibling`), whose source is not packed, or whose packed source is not installable (a `.tgz` that is not an archive, or is malformed or truncated, fails the consumer's install with `TAR_BAD_ARCHIVE`; a manifest that does not parse, on disk or inside the archive, fails it with `EJSONPARSE`) is reported. `workspace:` and `catalog:` count as registry specifiers only when the `prepack` lifecycle runs under pnpm, Yarn, or Bun (`npm_config_user_agent`), which rewrite them in the tarball they pack; `npm publish` publishes them verbatim and consumers fail with `EUNSUPPORTEDPROTOCOL`, so under npm — or when `agent-bundle prepack` runs outside any package-manager lifecycle — they are reported. The `npm pack --dry-run` that `prepack` itself spawns is only the file inventory; the tarball consumers receive is the lifecycle's packer's, which is what the user agent identifies. | Compiled bundles reach this gate without bare imports: `prepack` builds before it packs, and the compiler @@ -435,7 +435,9 @@ reports `AB6005` when a `dist` bundle or host-pack module keeps anything other t at all. The emitted-module walk remains behind that compile-time check, so the lexed `import` evidence below describes prebuilt payload modules and other packed scripts the framework did not compile. A `require`, `createRequire(…)(…)`, or `import.meta.resolve(…)` call the compiler does not resolve is not a module dependency; its evidence is read from every packed file, compiled bundles -included. The dependency evidence is read from the packed bytes themselves: every `.js`/`.mjs`/`.cjs` file +included. Because the compiler never opens a payload file, its +`runtimeDependencies` declaration also counts as dependency evidence. The +dependency evidence is read from the packed bytes themselves: every `.js`/`.mjs`/`.cjs` file `npm pack --dry-run` lists is lexed for static and dynamic `import` specifiers and scanned for literal `require("…")` and `.resolve("…")` calls (`require.resolve`, `createRequire(…).resolve`, `import.meta.resolve`: a package located only to find an asset is still a runtime dependency; a @@ -763,7 +765,7 @@ make the module self-executing (drop the default export and run at top level); or prefix a path segment with `_` (`src/scripts/_name.ts`) to keep the module out of script discovery and bin-only. -## Prebuilt payloads (`AB4740`–`AB4750`) +## Prebuilt payloads (`AB4740`–`AB4751`) The `payload` block and `{ prebuilt: ... }` entries (see `docs/entry-conventions.md`) package files the framework did not compile. @@ -774,7 +776,7 @@ simply not been built yet is a validation **warning** that only | Code | Severity | Trigger | | --- | --- | --- | -| `AB4740` | error | The `payload` block, one entry, or its `targets` list is malformed, or a payload selects an unknown target. | +| `AB4740` | error | The `payload` block, one entry, or its `targets` list is malformed, or a payload selects an unknown target. A `runtimeDependencies` list that is not an array of nonempty strings is also `AB4740`. | | `AB4741` | error | A payload destination is not a safe directory name, or shadows a compiler-owned artifact namespace (`assets`, `hooks`, `mcp`, `mcp-apps`, `scripts`, `skills`, root documents). | | `AB4742` | error | A payload source escapes the project root, is not a directory, or contains another payload's source. | | `AB4743` | warning | A declared payload directory does not exist yet or contains no files. Run the project's own build first. | @@ -785,6 +787,7 @@ simply not been built yet is a validation **warning** that only | `AB4748` | error (build) | `agent-bundle build` refuses a prebuilt entry file absent from its payload. | | `AB4749` | error (build) | A payload directory overlaps the artifact `--output` root. | | `AB4750` | info | A payload is older than the newest project source file and may be stale; rerun the project's own build if so. | +| `AB4751` | error | A `runtimeDependencies` entry is not a bare package name as npm reads one (a subpath such as `sharp/lib`, a selector such as `sharp@1`, a path, or a URL scheme), or `package.json` does not declare it as a dependency a consumer installs — `dependencies`, `optionalDependencies`, or a peer not marked optional in `peerDependenciesMeta` (the set `AB7014` judges). The declaration check is skipped when `package.json` is missing (silent) or unparsable or outside the root (`AB4011`). | ## Build-time identity outside the compiler (`AB4760`) diff --git a/docs/entry-conventions.md b/docs/entry-conventions.md index bb7b74d77..715da7dae 100644 --- a/docs/entry-conventions.md +++ b/docs/entry-conventions.md @@ -1139,11 +1139,17 @@ declares already-built directory trees the build packages **as-is**, and the inside them: ```ts +import { defineConfig, definePrebuilt } from 'agent-bundle'; + export default defineConfig({ payload: { // key = artifact-root destination directory, value = the built tree app: './dist/app', - runtime: { source: './dist/runtime', targets: ['claude', 'codex'] }, + runtime: definePrebuilt({ + source: './dist/runtime', + targets: ['claude', 'codex'], + runtimeDependencies: ['sharp'], + }), }, mcp: { servers: { @@ -1191,6 +1197,14 @@ export default defineConfig({ simulatable hook index. MCP Apps declared on a prebuilt server stay a development surface (the Workbench compiles them live); the build assumes the payload already serves the resource. +- **Declare what the payload loads.** Because payload trees are opaque, + `runtimeDependencies` on a `definePrebuilt` entry lists the bare package + names its files load. A name npm would not read as a bare package name, + or one `package.json` does not install for a consumer (`dependencies`, + `optionalDependencies`, or a peer not marked optional), is `AB4751`; a + malformed list is `AB4740`. The declaration check is skipped when + `package.json` is missing (silent), unparsable, or outside the root + (`AB4011`), and declared names count as used for `AB7014`. - **Ordering.** Run your own build before `agent-bundle build`: a missing or empty payload is a validation warning (`AB4743`/`AB4745`) so `dev` works from a clean checkout, but `agent-bundle build` refuses it diff --git a/packages/agent-bundle/src/build/pack-dependencies.ts b/packages/agent-bundle/src/build/pack-dependencies.ts index ebab0b70c..3a9d07205 100644 --- a/packages/agent-bundle/src/build/pack-dependencies.ts +++ b/packages/agent-bundle/src/build/pack-dependencies.ts @@ -28,95 +28,6 @@ import { readModuleImports, type ModuleImport } from './module-imports.ts'; * import). */ -/** - * The `package.json` fields npm installs alongside the published package. - * `peerDependencies` counts because npm 7+ installs peers automatically; - * `devDependencies` never reach a consumer and are not inspected. - */ -export const installedDependencyFields = Object.freeze([ - 'dependencies', - 'optionalDependencies', - 'peerDependencies', -] as const); - -export type InstalledDependencyField = (typeof installedDependencyFields)[number]; - -export interface DeclaredDependency { - readonly field: InstalledDependencyField; - readonly name: string; - readonly specifier: string; - /** Embedded in the tarball by npm (`bundleDependencies`), so a consumer never fetches its specifier. */ - readonly bundled: boolean; - /** - * Fetched for a consumer. `false` for a peer `peerDependenciesMeta` marks - * optional: npm parses its specifier — an unsupported protocol still fails - * the install — but never installs it, so no packed file has to use it. - */ - readonly installed: boolean; -} - -/** Peers `peerDependenciesMeta` marks optional: npm parses but never installs them. */ -const optionalPeers = (packageDocument: Readonly>): ReadonlySet => { - const meta = packageDocument.peerDependenciesMeta; - return new Set(isRecord(meta) - ? Object.entries(meta).filter(([, entry]) => isRecord(entry) && entry.optional === true).map(([name]) => name) - : []); -}; - -/** - * Which dependencies npm embeds under `node_modules` in the published tarball: - * `bundleDependencies` (or the `bundledDependencies` spelling) as a name list, - * or `true` for every entry of `dependencies`. Peers are never bundled, whatever - * the list says: npm packs no `node_modules` entry for a peer-only name, so a - * consumer still resolves the peer's own specifier. - */ -const bundledDependencies = ( - packageDocument: Readonly>, -): ((field: InstalledDependencyField, name: string) => boolean) => { - const value = packageDocument.bundleDependencies ?? packageDocument.bundledDependencies; - const names = new Set(Array.isArray(value) ? value.filter((entry): entry is string => typeof entry === 'string') : []); - return (field, name) => field !== 'peerDependencies' && (value === true ? field === 'dependencies' : names.has(name)); -}; - -/** - * The entries a consumer's npm reads, one per name and field. A name in both - * `dependencies` and `optionalDependencies` is the optional entry (npm lets - * the optional declaration override), and a peer that `dependencies` or - * `optionalDependencies` also names is that concrete entry — npm resolves - * the concrete declaration and never reads the duplicate peer's selector; an - * optional peer is kept, not `installed`. - */ -export const declaredDependencies = (packageDocument: Readonly>): readonly DeclaredDependency[] => { - const skippedPeers = optionalPeers(packageDocument); - const bundled = bundledDependencies(packageDocument); - const entries = (field: InstalledDependencyField): readonly (readonly [string, string])[] => { - const value = packageDocument[field]; - return isRecord(value) ? Object.entries(value).filter((entry): entry is [string, string] => typeof entry[1] === 'string') : []; - }; - const optional = new Set(entries('optionalDependencies').map(([name]) => name)); - const concrete = new Set([...entries('dependencies').map(([name]) => name), ...optional]); - const shadowed = (field: InstalledDependencyField, name: string): boolean => { - switch (field) { - case 'dependencies': return optional.has(name); - case 'peerDependencies': return concrete.has(name); - case 'optionalDependencies': return false; - default: { - const exhaustive: never = field; - return exhaustive; - } - } - }; - return installedDependencyFields.flatMap((field) => entries(field) - .filter(([name]) => !shadowed(field, name)) - .map(([name, specifier]) => ({ - field, - name, - specifier, - bundled: bundled(field, name), - installed: !(field === 'peerDependencies' && skippedPeers.has(name)), - }))); -}; - /** Relative, package-imports (`#`), absolute, and URL-scheme specifiers (`node:`, `data:`, `file:`, `C:\`) name no package. */ const nonPackageSpecifier = /^(?:[.#/]|[a-z][a-z0-9+.-]*:)/iu; /** `@scope/name` or `name`; a subpath after it is dropped. */ diff --git a/packages/agent-bundle/src/build/pack-inventory.ts b/packages/agent-bundle/src/build/pack-inventory.ts index b358ac88a..b535222a9 100644 --- a/packages/agent-bundle/src/build/pack-inventory.ts +++ b/packages/agent-bundle/src/build/pack-inventory.ts @@ -10,16 +10,14 @@ import { isRecord } from '../core/strict-json.ts'; import { readFileBytes, readFileString, runWithPlatform } from '../effect/platform.ts'; import { artifactManifestName } from './emit.ts'; import { parseArtifactManifest } from './manifest.ts'; +import { declaredDependencies, type DeclaredDependency, type InstalledDependencyField } from '../core/package-dependencies.ts'; import { classifyDependency, - declaredDependencies, importedPackageNames, isWorkspaceProtocol, packagedSourceInstallable, packagedSourcePath, - type DeclaredDependency, type DependencyKind, - type InstalledDependencyField, } from './pack-dependencies.ts'; import type { PackageBuildResult } from './package-build.ts'; @@ -152,7 +150,9 @@ const perField = ( * never from a `dist` bundle or a host-pack module. A `require`, * `createRequire(…)(…)`, or `import.meta.resolve(…)` call is not an import * and `AB6005` does not walk it, so that evidence is read from every packed - * file, compiled bundles included. + * file, compiled bundles included. A prebuilt payload's `runtimeDependencies` + * declaration is evidence of the same standing: the compiler never opens a + * payload file, so the author states what it loads. */ const unresolvableMessage = (field: InstalledDependencyField, own: readonly DeclaredDependency[]): string => `package.json ${field} names packages a consumer's npm cannot resolve through a registry (an invalid name or a non-registry specifier): ${own.map((dependency) => @@ -163,6 +163,7 @@ const unresolvableRecovery = 'Depend on a published registry version, or bundle + 'which only pnpm, Yarn, or Bun rewrite while packing.'; const dependencyDiagnostics = async (options: { + readonly declaredRuntimeDependencies: ReadonlySet; readonly packageDocument: Readonly>; readonly packedPaths: readonly string[]; readonly packerRewritesWorkspaceProtocols: boolean; @@ -212,20 +213,22 @@ const dependencyDiagnostics = async (options: { && !imported.installScripts.has(dependency.name); // A computed import() may load any declared package; nothing can then be called unused. const unused = imported.complete - ? declared.filter((dependency) => dependency.installed && !imported.names.has(dependency.name)) + ? declared.filter((dependency) => dependency.installed + && !imported.names.has(dependency.name) + && !options.declaredRuntimeDependencies.has(dependency.name)) : []; return [ // A peer nothing imports may be a deliberate compatibility contract with the host that loads the package; // npm 7+ still installs it for every consumer, so it is worth a look, not a refusal. ...perField(unused, (field, own) => diagnostic( 'AB7014', - `package.json ${field} names packages no packed JavaScript or declaration file references, runs, or install script needs: ${quoteAll(own.map((dependency) => dependency.name))}. ` + `package.json ${field} names packages no packed JavaScript or declaration file references, runs, or install script needs, and no prebuilt payload declares: ${quoteAll(own.map((dependency) => dependency.name))}. ` + (field === 'peerDependencies' ? 'If they only constrain the host version, that is a compatibility contract; npm 7+ still installs them for every consumer.' : 'Every consumer installs them for nothing; the emitted outputs already inline what they use.'), field === 'peerDependencies' ? 'Keep a deliberate compatibility peer, mark it optional in peerDependenciesMeta so npm stops installing it, or move a build-only package to devDependencies.' - : 'Move build-only packages to devDependencies; compiled bundles inline their imports (AB6005), so keep a runtime dependency only for what a prebuilt payload or other uncompiled packed module imports, a packed file requires or resolves (createRequire, import.meta.resolve), a packed declaration file references, a #subpath import reaches through the imports map, or an install script or packed JavaScript runs; a computed import() or require() in packed code withholds this check.', + : 'Move build-only packages to devDependencies; compiled bundles inline their imports (AB6005), so keep a runtime dependency only for what a prebuilt payload or other uncompiled packed module imports, a packed file requires or resolves (createRequire, import.meta.resolve), a packed declaration file references, a #subpath import reaches through the imports map, an install script or packed JavaScript runs, or a prebuilt payload names in runtimeDependencies (definePrebuilt); a computed import() or require() in packed code withholds this check.', field === 'peerDependencies' ? 'warning' : 'error', )), // npm skips an optional dependency it cannot fetch, so the install survives — but only once the specifier parsed @@ -336,6 +339,7 @@ export const packInventoryDiagnostics = async (options: { } diagnostics.push(...await dependencyDiagnostics({ + declaredRuntimeDependencies: new Set((options.model.payloads ?? []).flatMap((payload) => payload.runtimeDependencies)), packageDocument, packedPaths: [...packed], packerRewritesWorkspaceProtocols: options.packerRewritesWorkspaceProtocols, diff --git a/packages/agent-bundle/src/config/index.ts b/packages/agent-bundle/src/config/index.ts index 68c0575ed..45537d140 100644 --- a/packages/agent-bundle/src/config/index.ts +++ b/packages/agent-bundle/src/config/index.ts @@ -5,7 +5,7 @@ import type { PortableConfigExtension } from '../adapters/portable.ts'; import type { AgentBundleConfig as CoreAgentBundleConfig } from '../core/types.ts'; export { discoverProject } from './discover.ts'; -export { defineConfig } from '../core/types.ts'; +export { defineConfig, definePrebuilt } from '../core/types.ts'; export type { AgentProviderContext, AgentProviderFactory, diff --git a/packages/agent-bundle/src/config/normalize.ts b/packages/agent-bundle/src/config/normalize.ts index 23368c08b..4461eda25 100644 --- a/packages/agent-bundle/src/config/normalize.ts +++ b/packages/agent-bundle/src/config/normalize.ts @@ -372,13 +372,15 @@ const normalizePayloads = ( for (const [name, declaration] of Object.entries(configured).sort(([left], [right]) => left.localeCompare(right))) { const source = payloadDeclarationSource(loaded.context.projectRoot, declaration); if (source === undefined) continue; + const entry = typeof declaration === 'string' ? undefined : declaration; payloads.push({ files: (discoveredByName.get(name)?.files ?? []).map((file) => ({ ...file })), id: `payload:${name}`, name, provenance: { kind: 'prebuilt', sourcePath: loaded.configPath }, + runtimeDependencies: sortedUnique(entry?.runtimeDependencies ?? []), source, - targets: sortedUnique(typeof declaration === 'string' ? targetNames : (declaration.targets ?? targetNames)), + targets: sortedUnique(entry?.targets ?? targetNames), }); } return payloads; diff --git a/packages/agent-bundle/src/config/validate.ts b/packages/agent-bundle/src/config/validate.ts index da8927ca6..f441c094e 100644 --- a/packages/agent-bundle/src/config/validate.ts +++ b/packages/agent-bundle/src/config/validate.ts @@ -12,8 +12,10 @@ import { isPlainRecord, isRecord } from '../core/strict-json.ts'; import type { Diagnostic } from '../core/diagnostics.ts'; import { stableJson } from '../core/digest.ts'; import { unsupportedMcpTransportDiagnostic } from '../core/mcp-transport.ts'; +import { declaredDependencies, isBarePackageName } from '../core/package-dependencies.ts'; import { developmentFallbackVersion, + readPackageDocument, snapshotPackageIdentity, type PackageIdentityIssueKind, } from '../core/project-context.ts'; @@ -1658,11 +1660,66 @@ const payloadTargetDiagnostics = ( }; /** - * AB4740-AB4743 and the AB4750 freshness nudge: shape, destination-name, - * source-path, and existence checks for the prebuilt `payload` block. - * Missing or empty payloads warn here (development flows never require the - * consumer's own build to have run); `agent-bundle build` refuses them with - * AB4747/AB4748. + * The names a consumer's npm installs — `dependencies`, `optionalDependencies`, + * and peers not marked optional — the same set `AB7014` judges; undefined when + * the package document is absent or is one AB4011 already reports + * (unparsable, outside the root). + */ +const installedDependencyNames = (projectRoot: string): ReadonlySet | undefined => { + const read = readPackageDocument(projectRoot); + if (read.kind !== 'document') return undefined; + return new Set(declaredDependencies(read.document) + .filter((dependency) => dependency.installed) + .map((dependency) => dependency.name)); +}; + +/** + * AB4740/AB4751: one payload declaration's optional `runtimeDependencies`. + * The compiler never opens a payload file, so this list is the payload's + * only dependency evidence (`AB7014`); every name must be a bare package + * name (npm's own grammar) a consumer's npm installs. + */ +const payloadRuntimeDependencyDiagnostics = ( + name: string, + runtimeDependencies: unknown, + loaded: LoadedConfig, + installed: ReadonlySet | undefined, +): Diagnostic[] => { + if (runtimeDependencies === undefined) return []; + if (!Array.isArray(runtimeDependencies) || !runtimeDependencies.every(nonemptyString)) { + return [sourceDiagnostic( + 'AB4740', + `Payload ${JSON.stringify(name)} runtimeDependencies must be an array of package names.`, + loaded.configPath, + )]; + } + return runtimeDependencies.flatMap((dependency: string) => { + if (!isBarePackageName(dependency)) { + return [sourceDiagnostic( + 'AB4751', + `Payload ${JSON.stringify(name)} runtimeDependencies entry ${JSON.stringify(dependency)} is not a bare package name.`, + loaded.configPath, + 'Name the package (e.g. "sharp" or "@scope/name"), not a subpath or a specifier.', + )]; + } + if (installed !== undefined && !installed.has(dependency)) { + return [sourceDiagnostic( + 'AB4751', + `Payload ${JSON.stringify(name)} runtimeDependencies names ${JSON.stringify(dependency)}, which package.json does not declare as a dependency a consumer installs (dependencies, optionalDependencies, or a peer not marked optional).`, + loaded.configPath, + 'Declare the package under dependencies, optionalDependencies, or peerDependencies (not marked optional) so a consumer installs it, or remove it from runtimeDependencies.', + )]; + } + return []; + }); +}; + +/** + * AB4740-AB4743, AB4751, and the AB4750 freshness nudge: shape, + * destination-name, source-path, runtime-dependency, and existence checks + * for the prebuilt `payload` block. Missing or empty payloads warn here + * (development flows never require the consumer's own build to have run); + * `agent-bundle build` refuses them with AB4747/AB4748. */ const validatePayload = ( loaded: LoadedConfig, @@ -1676,6 +1733,7 @@ const validatePayload = ( } const diagnostics: Diagnostic[] = []; const sources: { name: string; source: string }[] = []; + const installed = installedDependencyNames(loaded.context.projectRoot); for (const [name, declaration] of Object.entries(configured)) { if (!isSafeOutputName(name) || reservedPayloadDestinations.has(name)) { const diagnostic = sourceDiagnostic( @@ -1700,7 +1758,10 @@ const validatePayload = ( continue; } if (typeof declaration !== 'string') { - diagnostics.push(...payloadTargetDiagnostics(name, declaration.targets, loaded, registry)); + diagnostics.push( + ...payloadTargetDiagnostics(name, declaration.targets, loaded, registry), + ...payloadRuntimeDependencyDiagnostics(name, declaration.runtimeDependencies, loaded, installed), + ); } const source = payloadDeclarationSource(loaded.context.projectRoot, declaration); if (source === undefined) { diff --git a/packages/agent-bundle/src/core/package-dependencies.ts b/packages/agent-bundle/src/core/package-dependencies.ts new file mode 100644 index 000000000..1d7ff8421 --- /dev/null +++ b/packages/agent-bundle/src/core/package-dependencies.ts @@ -0,0 +1,114 @@ +import npa from 'npm-package-arg'; + +import { isRecord } from './strict-json.ts'; + +/** + * The `package.json` dependency grammar as a consumer's npm reads it, shared by + * the prepack gate (`build/pack-dependencies.ts`, `AB7014`/`AB7015`) and + * config validation (`AB4751`). A leaf so `agent-bundle/config` never loads the + * packed-file scanners. + */ + +/** + * The `package.json` fields npm installs alongside the published package. + * `peerDependencies` counts because npm 7+ installs peers automatically; + * `devDependencies` never reach a consumer and are not inspected. + */ +const installedDependencyFields = Object.freeze([ + 'dependencies', + 'optionalDependencies', + 'peerDependencies', +] as const); + +export type InstalledDependencyField = (typeof installedDependencyFields)[number]; + +export interface DeclaredDependency { + readonly field: InstalledDependencyField; + readonly name: string; + readonly specifier: string; + /** Embedded in the tarball by npm (`bundleDependencies`), so a consumer never fetches its specifier. */ + readonly bundled: boolean; + /** + * Fetched for a consumer. `false` for a peer `peerDependenciesMeta` marks + * optional: npm parses its specifier — an unsupported protocol still fails + * the install — but never installs it, so no packed file has to use it. + */ + readonly installed: boolean; +} + +/** Peers `peerDependenciesMeta` marks optional: npm parses but never installs them. */ +const optionalPeers = (packageDocument: Readonly>): ReadonlySet => { + const meta = packageDocument.peerDependenciesMeta; + return new Set(isRecord(meta) + ? Object.entries(meta).filter(([, entry]) => isRecord(entry) && entry.optional === true).map(([name]) => name) + : []); +}; + +/** + * Which dependencies npm embeds under `node_modules` in the published tarball: + * `bundleDependencies` (or the `bundledDependencies` spelling) as a name list, + * or `true` for every entry of `dependencies`. Peers are never bundled, whatever + * the list says: npm packs no `node_modules` entry for a peer-only name, so a + * consumer still resolves the peer's own specifier. + */ +const bundledDependencies = ( + packageDocument: Readonly>, +): ((field: InstalledDependencyField, name: string) => boolean) => { + const value = packageDocument.bundleDependencies ?? packageDocument.bundledDependencies; + const names = new Set(Array.isArray(value) ? value.filter((entry): entry is string => typeof entry === 'string') : []); + return (field, name) => field !== 'peerDependencies' && (value === true ? field === 'dependencies' : names.has(name)); +}; + +/** + * The entries a consumer's npm reads, one per name and field. A name in both + * `dependencies` and `optionalDependencies` is the optional entry (npm lets + * the optional declaration override), and a peer that `dependencies` or + * `optionalDependencies` also names is that concrete entry — npm resolves + * the concrete declaration and never reads the duplicate peer's selector; an + * optional peer is kept, not `installed`. + */ +export const declaredDependencies = (packageDocument: Readonly>): readonly DeclaredDependency[] => { + const skippedPeers = optionalPeers(packageDocument); + const bundled = bundledDependencies(packageDocument); + const entries = (field: InstalledDependencyField): readonly (readonly [string, string])[] => { + const value = packageDocument[field]; + return isRecord(value) ? Object.entries(value).filter((entry): entry is [string, string] => typeof entry[1] === 'string') : []; + }; + const optional = new Set(entries('optionalDependencies').map(([name]) => name)); + const concrete = new Set([...entries('dependencies').map(([name]) => name), ...optional]); + const shadowed = (field: InstalledDependencyField, name: string): boolean => { + switch (field) { + case 'dependencies': return optional.has(name); + case 'peerDependencies': return concrete.has(name); + case 'optionalDependencies': return false; + default: { + const exhaustive: never = field; + return exhaustive; + } + } + }; + return installedDependencyFields.flatMap((field) => entries(field) + .filter(([name]) => !shadowed(field, name)) + .map(([name, specifier]) => ({ + field, + name, + specifier, + bundled: bundled(field, name), + installed: !(field === 'peerDependencies' && skippedPeers.has(name)), + }))); +}; + +/** + * True when npm reads the text as exactly one package name with no selector: + * `sharp`, `@scope/name`, and legacy-cased names such as `JSONStream`; not a + * subpath (`sharp/lib`), a selector (`sharp@1`), a path, or a URL scheme. + */ +export const isBarePackageName = (text: string): boolean => { + let parsed; + try { + parsed = npa(text); + } catch { + return false; + } + return parsed.type === 'range' && parsed.rawSpec === '*' && parsed.name === text; +}; diff --git a/packages/agent-bundle/src/core/project-context.ts b/packages/agent-bundle/src/core/project-context.ts index 22a8bbe85..b6a72781b 100644 --- a/packages/agent-bundle/src/core/project-context.ts +++ b/packages/agent-bundle/src/core/project-context.ts @@ -72,46 +72,74 @@ export const isValidPackageName = (value: string): boolean => /** True for a strict semver 2.0.0 version. */ export const isValidPackageVersion = (value: string): boolean => packageVersionPattern.test(value); +/** The outcome of reading `/package.json` for any project-identity purpose. */ +export type PackageDocumentRead = + /** No package.json, or one that cannot be read: a normal development state. */ + | { readonly kind: 'absent' } + | { readonly document: Readonly>; readonly kind: 'document' } + | { readonly issue: PackageIdentityIssue; readonly kind: 'issue' }; + /** - * Derives the release-identity axes from `/package.json`. A missing - * package.json (or missing name/version fields) is a normal development - * state: no identity and no issues. An invalid name or version becomes an - * issue for the caller to surface as a diagnostic, never a crash, and the - * invalid value is withheld from the derived identity. + * Reads `/package.json` the one way every identity-derived judgement + * shares. A package.json symlinked outside the project cannot join the + * identity: its bytes are invisible to the source snapshot, so anything + * derived from it could drift without a revision change. */ -export const snapshotPackageIdentity = (root: string): PackageIdentitySnapshot => { +export const readPackageDocument = (root: string): PackageDocumentRead => { let packageJsonPath: string; let canonicalRoot: string; try { canonicalRoot = realpathSync(resolve(root)); packageJsonPath = realpathSync(join(resolve(root), 'package.json')); } catch { - return deepFreeze({ issues: [] }); + return { kind: 'absent' }; } - // A package.json symlinked outside the project cannot join the identity: - // its bytes are invisible to the source snapshot, so deriving release - // identity from it would let identity drift without a revision change. if (!isInsideOrEqual(canonicalRoot, packageJsonPath)) { - return deepFreeze({ - issues: [{ kind: 'outside-root', message: 'package.json resolves outside the project root; package identity is ignored.' }], - }); + return { + issue: { kind: 'outside-root', message: 'package.json resolves outside the project root; package identity is ignored.' }, + kind: 'issue', + }; } let bytes: string; try { bytes = readFileSync(packageJsonPath, 'utf8'); } catch { - return deepFreeze({ issues: [] }); + return { kind: 'absent' }; } let parsed: unknown; try { parsed = JSON.parse(bytes); } catch { - return deepFreeze({ issues: [{ kind: 'unparsable', message: 'package.json is not valid JSON.' }] }); + return { issue: { kind: 'unparsable', message: 'package.json is not valid JSON.' }, kind: 'issue' }; } if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) { - return deepFreeze({ issues: [{ kind: 'unparsable', message: 'package.json must contain a JSON object.' }] }); + return { issue: { kind: 'unparsable', message: 'package.json must contain a JSON object.' }, kind: 'issue' }; + } + return { document: parsed as Readonly>, kind: 'document' }; +}; + +/** + * Derives the release-identity axes from `/package.json`. A missing + * package.json (or missing name/version fields) is a normal development + * state: no identity and no issues. An invalid name or version becomes an + * issue for the caller to surface as a diagnostic, never a crash, and the + * invalid value is withheld from the derived identity. + */ +export const snapshotPackageIdentity = (root: string): PackageIdentitySnapshot => { + const read = readPackageDocument(root); + switch (read.kind) { + case 'absent': + return deepFreeze({ issues: [] }); + case 'issue': + return deepFreeze({ issues: [read.issue] }); + case 'document': + break; + default: { + const exhaustive: never = read; + throw new TypeError(`Unknown package document read ${String(exhaustive)}.`); + } } - const record = parsed as Readonly>; + const record = read.document; const issues: PackageIdentityIssue[] = []; let packageName: string | undefined; if (record.name !== undefined) { diff --git a/packages/agent-bundle/src/core/types.ts b/packages/agent-bundle/src/core/types.ts index 359e3dfc1..b402492b6 100644 --- a/packages/agent-bundle/src/core/types.ts +++ b/packages/agent-bundle/src/core/types.ts @@ -204,6 +204,14 @@ export interface AgentBundleScriptEntry { /** One declared prebuilt payload directory with an optional target restriction. */ export interface AgentBundlePayloadEntry { + /** + * Bare package names the payload's files load at run time. The compiler + * never opens a payload file, so this declaration is the only evidence + * that a `package.json` dependency is used by the payload (`AB7014`); + * each name must be one a consumer's npm installs — `dependencies`, + * `optionalDependencies`, or a peer not marked optional (`AB4751`). + */ + runtimeDependencies?: readonly string[]; source: string; targets?: readonly string[]; } @@ -602,6 +610,8 @@ export interface NormalizedPayload { /** The artifact-root destination directory name. */ readonly name: string; readonly provenance: SourceProvenance; + /** Declared bare package names the payload loads at run time, sorted and unique. */ + readonly runtimeDependencies: readonly string[]; /** Absolute payload source directory. */ readonly source: string; readonly targets: readonly string[]; @@ -868,6 +878,14 @@ export const defineConfig = ( config: AgentBundleConfig | ConfigFactory, ): AgentBundleConfig | ConfigFactory => config; +/** + * Declares one prebuilt payload directory for the `payload` block: a tree the + * project compiled itself, packaged byte-for-byte, with the package names its + * files load at run time. The compiler treats the tree as opaque, so + * `runtimeDependencies` is what keeps those packages out of `AB7014`. + */ +export const definePrebuilt = (entry: AgentBundlePayloadEntry): AgentBundlePayloadEntry => entry; + export const pathTokens = Object.freeze({ pluginRoot: 'agent-bundle:path:plugin-root', pluginData: 'agent-bundle:path:plugin-data', diff --git a/packages/agent-bundle/src/index.ts b/packages/agent-bundle/src/index.ts index 9904f8cf1..9d619253d 100644 --- a/packages/agent-bundle/src/index.ts +++ b/packages/agent-bundle/src/index.ts @@ -4,7 +4,7 @@ import type { CursorConfigExtension } from './adapters/cursor.ts'; import type { PortableConfigExtension } from './adapters/portable.ts'; import type { AgentBundleConfig as CoreAgentBundleConfig } from './core/types.ts'; -export { defineConfig, pathTokens, pluginRootEnvAnchor } from './core/types.ts'; +export { defineConfig, definePrebuilt, pathTokens, pluginRootEnvAnchor } from './core/types.ts'; export { defineSkill, Skill } from './skills/define.ts'; export { classifySkillToken, diff --git a/packages/agent-bundle/tests/pack-dependencies.test.ts b/packages/agent-bundle/tests/pack-dependencies.test.ts index bb1966e82..f801c1871 100644 --- a/packages/agent-bundle/tests/pack-dependencies.test.ts +++ b/packages/agent-bundle/tests/pack-dependencies.test.ts @@ -2,13 +2,13 @@ import { expect, it } from '@rstest/core'; import { classifyDependency, - declaredDependencies, isWorkspaceProtocol, declarationSpecifiers, packageNameOf, rewritesWorkspaceProtocols, shellWords, } from '../src/build/pack-dependencies.ts'; +import { declaredDependencies, isBarePackageName } from '../src/core/package-dependencies.ts'; it.each([ // Registry: versions, ranges (npm's loose grammar included), and dist-tags. @@ -246,3 +246,8 @@ it('marks bundleDependencies entries, by name list or wholesale, as bundled', () }).map((d) => [d.name, d.bundled])).toEqual([['optional', true], ['peer', false]]); expect(declaredDependencies({ bundleDependencies: true, optionalDependencies: { optional: '^1' } })[0]?.bundled).toBe(false); }); + +it('reads a bare package name as npm does: a name with no selector, subpath, path, or scheme', () => { + expect(['sharp', '@scope/name', 'JSONStream'].filter(isBarePackageName)).toEqual(['sharp', '@scope/name', 'JSONStream']); + expect(['sharp/lib', '@scope/name/sub', 'sharp@1', './x', 'node:fs', 'npm:foo', 'bad name', '@scope/', ''].filter(isBarePackageName)).toEqual([]); +}); diff --git a/packages/agent-bundle/tests/prebuilt-payload.test.ts b/packages/agent-bundle/tests/prebuilt-payload.test.ts index cd16da82c..fa4707fa0 100644 --- a/packages/agent-bundle/tests/prebuilt-payload.test.ts +++ b/packages/agent-bundle/tests/prebuilt-payload.test.ts @@ -4,7 +4,9 @@ import { join } from 'node:path'; import { expect, it } from '@rstest/core'; import { build, validate } from '../src/api.ts'; +import { definePrebuilt as definePrebuiltFromConfig } from '../src/config/index.ts'; import { DiagnosticError } from '../src/core/diagnostics.ts'; +import { definePrebuilt as definePrebuiltFromIndex } from '../src/index.ts'; import { parseArtifactManifest } from '../src/build/manifest.ts'; import { createProjectFixture, removeProjectFixture } from './helpers/project-fixture.ts'; @@ -64,6 +66,117 @@ const createProject = async (options: { const readJson = async (path: string): Promise => JSON.parse(await readFile(path, 'utf8')) as Document; +it('exports definePrebuilt from the package and config entry points as an identity helper', () => { + const entry = { runtimeDependencies: ['sharp'], source: './built/runtime' } as const; + expect(definePrebuiltFromIndex(entry)).toBe(entry); + expect(definePrebuiltFromConfig(entry)).toBe(entry); +}); + +it('validates declared payload runtime dependencies and normalizes them sorted and unique', async () => { + const root = await createProject({ + files: { + 'package.json': `${JSON.stringify({ + dependencies: { sharp: '^0.33.0', zod: '^4.0.0' }, + name: 'prebuilt-fixture', + version: '1.0.0', + }, null, 2)}\n`, + }, + payload: " payload: { runtime: { source: './built/runtime', runtimeDependencies: ['zod', 'sharp', 'sharp'] } },", + }); + try { + const validated = await validate({ root }); + expect(validated.diagnostics.filter((diagnostic) => /^AB474\d$|^AB4751$/u.test(diagnostic.code))).toEqual([]); + + const built = await build({ output: join(root, 'out'), root }); + expect(built.model.payloads).toMatchObject([ + { name: 'runtime', runtimeDependencies: ['sharp', 'zod'] }, + ]); + } finally { + await removeProjectFixture(root); + } +}); + +it('reports a payload runtime dependency that is not a bare package name', async () => { + const root = await createProject({ + payload: " payload: { runtime: { source: './built/runtime', runtimeDependencies: ['sharp/lib'] } },", + }); + try { + expect((await validate({ root })).diagnostics).toContainEqual(expect.objectContaining({ + code: 'AB4751', + message: 'Payload "runtime" runtimeDependencies entry "sharp/lib" is not a bare package name.', + })); + } finally { + await removeProjectFixture(root); + } +}); + +const undeclaredMessage = 'Payload "runtime" runtimeDependencies names "sharp", which package.json does not declare as a dependency a consumer installs (dependencies, optionalDependencies, or a peer not marked optional).'; + +it.each([ + ['nothing', '{"name":"prebuilt-fixture","version":"1.0.0"}'], + // npm never installs an optional peer, so no packed file can rely on it. + ['an optional peer', '{"name":"prebuilt-fixture","version":"1.0.0","peerDependencies":{"sharp":"^0.33.0"},"peerDependenciesMeta":{"sharp":{"optional":true}}}'], +])('reports a payload runtime dependency package.json installs as %s', async (_declared, packageJson) => { + const root = await createProject({ + files: { 'package.json': `${packageJson}\n` }, + payload: " payload: { runtime: { source: './built/runtime', runtimeDependencies: ['sharp'] } },", + }); + try { + expect((await validate({ root })).diagnostics).toContainEqual(expect.objectContaining({ code: 'AB4751', message: undeclaredMessage })); + } finally { + await removeProjectFixture(root); + } +}); + +it.each([ + ['optionalDependencies', '{"name":"prebuilt-fixture","version":"1.0.0","optionalDependencies":{"sharp":"^0.33.0"}}'], + // npm 7+ installs a required peer for every consumer, the same reading AB7014 applies. + ['a required peer', '{"name":"prebuilt-fixture","version":"1.0.0","peerDependencies":{"sharp":"^0.33.0"}}'], +])('accepts a payload runtime dependency package.json installs through %s', async (_declared, packageJson) => { + const root = await createProject({ + files: { 'package.json': `${packageJson}\n` }, + payload: " payload: { runtime: { source: './built/runtime', runtimeDependencies: ['sharp'] } },", + }); + try { + // No payload error at all: an earlier AB474x would have skipped the runtime-dependency check + // (the AB4750 freshness nudge depends on fixture mtimes and is not a verdict). + expect((await validate({ root })).diagnostics.filter((diagnostic) => /^AB474\d$|^AB4751$/u.test(diagnostic.code))).toEqual([]); + } finally { + await removeProjectFixture(root); + } +}); + +it.each([ + "'sharp'", + "['']", +])('reports runtimeDependencies with %s as AB4740', async (runtimeDependencies) => { + const root = await createProject({ + payload: ` payload: { runtime: { source: './built/runtime', runtimeDependencies: ${runtimeDependencies} } },`, + }); + try { + expect((await validate({ root })).diagnostics).toContainEqual(expect.objectContaining({ + code: 'AB4740', + message: 'Payload "runtime" runtimeDependencies must be an array of package names.', + })); + } finally { + await removeProjectFixture(root); + } +}); + +it('normalizes string-form payload declarations with no runtime dependencies', async () => { + const root = await createProject({ + payload: " payload: { app: './built/app' },", + }); + try { + const built = await build({ output: join(root, 'out'), root }); + expect(built.model.payloads).toMatchObject([ + { name: 'app', runtimeDependencies: [] }, + ]); + } finally { + await removeProjectFixture(root); + } +}); + it('packages prebuilt payloads at stable paths and lowers prebuilt entries through every adapter', async () => { const root = await createProject({ hooks: standardHooksBlock, diff --git a/packages/agent-bundle/tests/prepack.test.ts b/packages/agent-bundle/tests/prepack.test.ts index 7fc021fb6..c9cc3fdb7 100644 --- a/packages/agent-bundle/tests/prepack.test.ts +++ b/packages/agent-bundle/tests/prepack.test.ts @@ -10,6 +10,7 @@ import { afterAll, beforeAll, expect, it } from '@rstest/core'; import { prepack } from '../src/api.ts'; import { runCli } from '../src/cli.ts'; import { type Diagnostic, DiagnosticError } from '../src/core/diagnostics.ts'; +import type { NormalizedPayload } from '../src/core/types.ts'; import { captureCliTerminal } from './support/cli-terminal.ts'; import { packInventoryDiagnostics, @@ -244,6 +245,37 @@ it('reports installed dependencies no packed JavaScript imports as AB7014, per f }, )); +it('accepts a dependency declared by a prebuilt payload runtimeDependencies list', () => withPackageDocument( + (document) => { + document.dependencies = { sharp: '^0.33.0', 'never-loaded': '^1.0.0' }; + }, + async () => { + const payload: NormalizedPayload = { + files: [], + id: 'payload:tools', + name: 'tools', + provenance: { kind: 'prebuilt', sourcePath: join(projectRoot, 'agent-bundle.config.ts') }, + runtimeDependencies: ['sharp'], + source: join(projectRoot, 'built', 'tools'), + targets: ['claude'], + }; + const reported = withCode(await packInventoryDiagnostics({ + artifactRoot: result.build.build.outputRoot, + model: { ...result.build.model, payloads: [payload] }, + packageBuild: result.build.packageBuild!, + packOutput: result.pack, + packerRewritesWorkspaceProtocols: false, + projectRoot, + }), 'AB7014'); + + expect(reported).toHaveLength(1); + expect(reported[0]?.message).toContain('"never-loaded"'); + expect(reported[0]?.message).not.toContain('"sharp"'); + expect(reported[0]?.message).toContain('no prebuilt payload declares'); + expect(reported[0]?.recovery).toContain('runtimeDependencies'); + }, +)); + it('reports an optional peer only for a protocol npm cannot parse, which fails the install before any fetch', () => withPackageDocument( (document) => { document.peerDependencies = { 'git-peer': 'github:owner/git-peer', 'workspace-peer': 'workspace:*', 'typo-peer': 'foo:bar' }; diff --git a/website/docs/en/guide/authoring/index.mdx b/website/docs/en/guide/authoring/index.mdx index d946d6aca..71b7e41b7 100644 --- a/website/docs/en/guide/authoring/index.mdx +++ b/website/docs/en/guide/authoring/index.mdx @@ -72,7 +72,7 @@ standard only through CLI translation, which is why its dedicated target remains | `routes` | Power-tier route-graph policy: `routes.servers` per-server modes, `routes.cli`, and `routes.mcpCommands` projection. | | `output` | `distPath` — where the artifact root lives. | | `runtime` | `node` — the minimum Node.js version for generated executables. | -| `payload` | Prebuilt directory trees packaged byte-for-byte. | +| `payload` | Prebuilt directory trees packaged byte-for-byte (`definePrebuilt`). | | `state` | `false` to disable the `src/state.ts` convention. | | `marketplace` | `true` adds Cursor's optional `.cursor-plugin/marketplace.json`. Claude Code and Codex always emit their local marketplace manifests regardless. | | `evals` | Eval suite discovery (`include`), the run directory (`runsDir`), and the optional `semanticGrader`. See [Evaluations](../development/evaluations.mdx). | diff --git a/website/docs/en/guide/authoring/package-entries.mdx b/website/docs/en/guide/authoring/package-entries.mdx index e04fc4453..e038ef111 100644 --- a/website/docs/en/guide/authoring/package-entries.mdx +++ b/website/docs/en/guide/authoring/package-entries.mdx @@ -310,6 +310,27 @@ each payload file lands in the artifact manifest with its SHA-256 and the `prebu and hashes into the project revision. Run your own build first — a missing or empty payload is a warning under `dev` so a clean checkout works, but `agent-bundle build` refuses it. +Because the compiler never opens payload files, use `definePrebuilt` and +`runtimeDependencies` to list the bare package names they load. A name npm would not read +as a bare package name, or one `package.json` does not install for a consumer +(`dependencies`, `optionalDependencies`, or a peer not marked optional), is `AB4751`; the +declaration check is skipped when `package.json` is missing (silent), unparsable, or outside +the root (`AB4011`). Declared names count as used for `AB7014`. + +```ts +import { defineConfig, definePrebuilt } from 'agent-bundle'; + +export default defineConfig({ + payload: { + tools: definePrebuilt({ + source: './built/tools', + targets: ['claude'], + runtimeDependencies: ['sharp'], + }), + }, +}); +``` + ## The bundler escape hatch `tools.rsbuild` (an Rsbuild environment-config fragment) and `tools.rspack` (an Rspack config diff --git a/website/docs/en/guide/distribution/validation.mdx b/website/docs/en/guide/distribution/validation.mdx index 08cce7ceb..c825227fc 100644 --- a/website/docs/en/guide/distribution/validation.mdx +++ b/website/docs/en/guide/distribution/validation.mdx @@ -235,7 +235,8 @@ are opaque to `AB6005`, and `AB7014` scans only those extensions, so an extensio module counts for nothing), one a packed file loads through a `require`, `createRequire`, or `import.meta.resolve` call that the compiler does not resolve as a module dependency, one a consumer-side install script runs, one whose `bin` a packed file executes, or one a -packed declaration file references (`.d.ts` outputs are not walked by `AB6005`) — and `AB7014` +packed declaration file references (`.d.ts` outputs are not walked by `AB6005`), or one a +prebuilt payload names in `runtimeDependencies` — and `AB7014` reports a declared dependency with none of that evidence, while `AB7015` reports one a consumer's npm cannot install. @@ -245,10 +246,11 @@ npm cannot install. | `AB7011` | An on-disk artifact file no longer matches its manifest SHA-256. Rebuild, and do not modify generated host packs. | | `AB7012` | A `package.json` bin points outside the packed `dist` output (including `src/`) or names a file npm omitted. Point it at the generated `dist/bin` file. | | `AB7013` | `package.json`, normalized plugin metadata, a host manifest, or artifact provenance reports a different release version. Make every release identity agree. | -| `AB7014` | A `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses — no packed JavaScript imports, requires, or resolves them (compiled bundles inline their imports and `AB6005` fails one they kept external, so `import` evidence comes from prebuilt payload modules and other scripts the framework did not compile, while `require`, `createRequire`, and `import.meta.resolve` calls count from any packed file), no packed declaration references them, no `imports` mapping or consumer install script reaches them (one diagnostic per field; optional peers are skipped here but their specifier is still checked by `AB7015`, and an `optionalDependencies` entry supersedes the same name under `dependencies`). Every consumer would install them for nothing. Move build-only packages to `devDependencies`. For `peerDependencies` this is a warning, since a required peer nothing imports may be a deliberate host-compatibility contract; mark it optional in `peerDependenciesMeta` if npm should stop installing it. | +| `AB7014` | A `dependencies`, `optionalDependencies`, or `peerDependencies` field names packages nothing in the pack uses — no packed JavaScript imports, requires, or resolves them (compiled bundles inline their imports and `AB6005` fails one they kept external, so `import` evidence comes from prebuilt payload modules and other scripts the framework did not compile, while `require`, `createRequire`, and `import.meta.resolve` calls count from any packed file), no packed declaration references them, no `imports` mapping or consumer install script reaches them, and no prebuilt payload names them in `runtimeDependencies` (one diagnostic per field; optional peers are skipped here but their specifier is still checked by `AB7015`, and an `optionalDependencies` entry supersedes the same name under `dependencies`). Every consumer would install them for nothing. Move build-only packages to `devDependencies`. For `peerDependencies` this is a warning, since a required peer nothing imports may be a deliberate host-compatibility contract; mark it optional in `peerDependenciesMeta` if npm should stop installing it. | | `AB7015` | A `dependencies`, `optionalDependencies`, or `peerDependencies` entry a consumer's npm cannot resolve through a registry. Name and specifier are read with `npm-package-arg`, npm's own parser, and come out as one of three kinds: registry (a version, range, dist-tag, or `npm:` alias of one), fetched (a git, GitHub-shorthand, remote-tarball, or path source — npm 12 refuses git and remote fetches by default (`allow-git`, `allow-remote`), and a path never exists on the consumer's disk), or unparseable (a name npm rejects, a scheme it lacks such as `link:`, `portal:`, or a typo, a selector that is neither a range nor a URL-safe dist-tag, an alias of a non-registry target, or an invalid URL — the manifest read itself fails, so this is reported even on an optional peer). A fetched `optionalDependencies` entry warns instead of failing, since npm continues without it; it stays an error when unparseable, or when a consumer install script needs the skipped package (runs its command — not merely mentions it — loads it from an inline `node -e` program — scanned for `require`, `createRequire`, and `import()` like a packed file — preloads it with `node -r`/`--require`/`--import`/`--loader`, or runs a packed file that imports it, `node .` running the root `main` included; each command after `&&`, `;`, or a newline is read on its own, shell quotes and backslash escapes resolved, and `node`'s options belong to `node` alone and end at the program — `node install.js --require x` preloads nothing, while a `NODE_OPTIONS=--require=x` assignment on the same `node` command does; every script word naming a packed JavaScript file counts as run, deliberately, so that runners the gate does not model — `tsx`, `zx`, `bun`, `deno run` — still have their file's dependencies traced, at the cost of a rare escalation for a word such as `echo install.js`, which the diagnostic makes visible by naming the file). A peer that `dependencies` or `optionalDependencies` also names is judged by that concrete entry; npm never reads the duplicate peer's selector. `workspace:`/`catalog:` are reported too unless the `prepack` lifecycle runs under pnpm, Yarn, or Bun, which rewrite them at pack time — `npm publish` does not, so under npm `AB7015` fires; entries the tarball itself carries — `bundleDependencies` npm actually packed (never peers), and a `file:` path inside the package whose packed source is installable (a directory with a parseable `package.json`, or a well-formed tarball whose `package.json` parses) — are not reported. Depend on a published version, or bundle the package and declare it under `devDependencies`. | -The dependency evidence is read from the packed bytes: every `.js`/`.mjs`/`.cjs` file npm would +A prebuilt payload's `runtimeDependencies` declaration counts as use, since the compiler never opens a +payload file. The remaining dependency evidence is read from the packed bytes: every `.js`/`.mjs`/`.cjs` file npm would publish is lexed for `import` specifiers and scanned for literal `require("…")`, `require.resolve("…")`, and `import.meta.resolve("…")` calls (a `createRequire(…)` binding counts as `require`, even with the factory renamed on import, as do direct `Module.createRequire(…)("…")` and `require("node:module").createRequire(…)("…")` calls; `path.resolve("…")` does not count), and every diff --git a/website/docs/zh/guide/authoring/index.mdx b/website/docs/zh/guide/authoring/index.mdx index 866a7411d..9195d7ece 100644 --- a/website/docs/zh/guide/authoring/index.mdx +++ b/website/docs/zh/guide/authoring/index.mdx @@ -69,7 +69,7 @@ export default defineConfig({ | `routes` | 进阶的路由图策略:`routes.servers` 逐服务器模式、`routes.cli`,以及 `routes.mcpCommands` 投影。 | | `output` | `distPath` —— 产物根目录的位置。 | | `runtime` | `node` —— 生成可执行文件的最低 Node.js 版本。 | -| `payload` | 按字节打包的预构建目录树。 | +| `payload` | 按字节打包的预构建目录树(`definePrebuilt`)。 | | `state` | 设为 `false` 可禁用 `src/state.ts` 约定。 | | `marketplace` | 设为 `true` 时额外输出 Cursor 可选的 `.cursor-plugin/marketplace.json`。Claude Code 与 Codex 无论如何都会输出各自的本地 marketplace 清单。 | | `evals` | Eval 套件发现(`include`)、运行目录(`runsDir`)与可选的 `semanticGrader`。见[评估](../development/evaluations.mdx)。 | diff --git a/website/docs/zh/guide/authoring/package-entries.mdx b/website/docs/zh/guide/authoring/package-entries.mdx index eeecfd0c5..735744b82 100644 --- a/website/docs/zh/guide/authoring/package-entries.mdx +++ b/website/docs/zh/guide/authoring/package-entries.mdx @@ -275,6 +275,25 @@ npx agent-bundle prepack --root . --output artifact --json 并参与项目 revision 的哈希。请先运行你自己的构建——缺失或为空的 payload 在 `dev` 下只是警告,以便 干净检出也能工作,但 `agent-bundle build` 会拒绝它。 +编译器不会打开 payload 文件,因此请用 `definePrebuilt` 和 `runtimeDependencies` 列出它们加载的裸包名。 +npm 不会将其读作裸包名的条目,或 `package.json` 未为消费者安装的包名(`dependencies`、 +`optionalDependencies`,或未标记为可选的 peer),会报 `AB4751`;`package.json` 缺失(静默)、 +无法解析或位于项目根之外(`AB4011`)时跳过声明检查。已声明的包名在 `AB7014` 中视为已使用。 + +```ts +import { defineConfig, definePrebuilt } from 'agent-bundle'; + +export default defineConfig({ + payload: { + tools: definePrebuilt({ + source: './built/tools', + targets: ['claude'], + runtimeDependencies: ['sharp'], + }), + }, +}); +``` + ## 打包器逃生舱 `tools.rsbuild`(一个 Rsbuild 环境配置片段)与 `tools.rspack`(一个 Rspack 配置对象、mutator 函数或 diff --git a/website/docs/zh/guide/distribution/validation.mdx b/website/docs/zh/guide/distribution/validation.mdx index 31fee23bd..6817da30a 100644 --- a/website/docs/zh/guide/distribution/validation.mdx +++ b/website/docs/zh/guide/distribution/validation.mdx @@ -200,8 +200,8 @@ CLI、MCP 入口、钩子包装层以及包构建的 JavaScript bundle——都 有据可证地需要从已编译 bundle 之外获取的内容——预构建的 `.js`/`.mjs`/`.cjs` payload 模块导入的包(预构建 文件对 `AB6005` 不透明,而 `AB7014` 只扫描这些扩展名,因此无扩展名的预构建模块不算任何证据),打包后的 文件通过未被编译器解析为模块依赖的 `require`、`createRequire` 或 `import.meta.resolve` 调用加载的包, -消费者侧安装脚本运行的包,打包后的文件执行其 `bin` 的包,或打包后的声明文件引用的包 -(`.d.ts` 输出不会被 `AB6005` 遍历)——`AB7014` 会报告不具备上述任何一种证据的已声明依赖,而 `AB7015` 会报告 +消费者侧安装脚本运行的包,打包后的文件执行其 `bin` 的包,打包后的声明文件引用的包 +(`.d.ts` 输出不会被 `AB6005` 遍历),或预构建 payload 在 `runtimeDependencies` 中声明的包——`AB7014` 会报告不具备上述任何一种证据的已声明依赖,而 `AB7015` 会报告 消费者的 npm 无法安装的依赖。 | 代码 | 含义 | @@ -210,10 +210,10 @@ CLI、MCP 入口、钩子包装层以及包构建的 JavaScript bundle——都 | `AB7011` | 磁盘上的某个产物文件与其清单 SHA-256 不再匹配。请重新构建,且不要修改生成的宿主包。 | | `AB7012` | 某个 `package.json` bin 指向了打包后的 `dist` 输出之外(包括 `src/`),或指名了一个被 npm 忽略的文件。请把它指向生成的 `dist/bin` 文件。 | | `AB7013` | `package.json`、规范化后的插件元数据、某份宿主清单或产物 provenance 报告了不同的发布版本。请让每处发布标识一致。 | -| `AB7014` | `dependencies`、`optionalDependencies` 或 `peerDependencies` 字段里列出的包在整个包里无人使用——没有打包后的 JavaScript 导入、require 或解析它(已编译的 bundle 会内联自己的导入,被保持外部化的导入会被 `AB6005` 判为失败,因此 `import` 证据只来自预构建 payload 模块以及其他框架没有编译的脚本,而 `require`、`createRequire` 与 `import.meta.resolve` 调用则可来自任何打包后的文件),没有打包后的声明文件引用它,也没有 `imports` 映射或消费者侧安装脚本触及它(每个字段一条诊断;可选 peer 在此被跳过,但其说明符仍由 `AB7015` 检查,`optionalDependencies` 中的同名项优先于 `dependencies`)。每位消费者都会白白安装它们。请把仅构建期需要的包移到 `devDependencies`。对 `peerDependencies` 而言这是警告:无人导入的必需 peer 可能是有意的宿主兼容性约束;若希望 npm 不再安装它,请在 `peerDependenciesMeta` 中标记为可选。 | +| `AB7014` | `dependencies`、`optionalDependencies` 或 `peerDependencies` 字段里列出的包在整个包里无人使用——没有打包后的 JavaScript 导入、require 或解析它(已编译的 bundle 会内联自己的导入,被保持外部化的导入会被 `AB6005` 判为失败,因此 `import` 证据只来自预构建 payload 模块以及其他框架没有编译的脚本,而 `require`、`createRequire` 与 `import.meta.resolve` 调用则可来自任何打包后的文件),没有打包后的声明文件引用它,也没有 `imports` 映射或消费者侧安装脚本触及它,也没有任何预构建 payload 在 `runtimeDependencies` 中声明它(每个字段一条诊断;可选 peer 在此被跳过,但其说明符仍由 `AB7015` 检查,`optionalDependencies` 中的同名项优先于 `dependencies`)。每位消费者都会白白安装它们。请把仅构建期需要的包移到 `devDependencies`。对 `peerDependencies` 而言这是警告:无人导入的必需 peer 可能是有意的宿主兼容性约束;若希望 npm 不再安装它,请在 `peerDependenciesMeta` 中标记为可选。 | | `AB7015` | `dependencies`、`optionalDependencies` 或 `peerDependencies` 中的某一项无法被消费者的 npm 经由注册表解析。包名与说明符一并交给 npm 自己的解析器 `npm-package-arg` 读取,结果分为三类:注册表类(版本、范围、dist-tag,或指向它们的 `npm:` 别名)、抓取类(git、GitHub 简写、远程 tarball 或路径来源——npm 12 默认拒绝 git 与远程抓取(`allow-git`、`allow-remote`),而路径在消费者磁盘上并不存在),以及不可解析类(npm 不接受的包名、它不支持的协议如 `link:`、`portal:` 或拼写错误、既非范围也非 URL 安全 dist-tag 的选择器、指向非注册表目标的别名,或非法 URL——清单本身就读不下去,因此即使出现在可选 peer 上也会被报告)。`optionalDependencies` 中的抓取类项只发出警告而不会失败,因为 npm 会继续安装;不可解析的项,或消费者安装脚本需要的项(运行其命令——而非仅仅提及它——在内联的 `node -e` 程序中加载它——该程序像已打包文件一样被扫描 `require`、`createRequire` 与 `import()`——用 `node -r`/`--require`/`--import`/`--loader` 预加载它,或运行某个导入它的已打包文件,包括通过根 `main` 运行的 `node .`;`&&`、`;` 或换行之后的每条命令单独判读,shell 引号与反斜杠转义均已解析,`node` 的选项只属于 `node` 且到程序为止——`node install.js --require x` 不会预加载任何东西,而同一条 `node` 命令上的 `NODE_OPTIONS=--require=x` 赋值会;脚本中每个点名已打包 JavaScript 文件的词都视为被运行,这是有意为之,使本闸门未建模的运行器——`tsx`、`zx`、`bun`、`deno run`——所运行文件的依赖仍被追踪,代价是像 `echo install.js` 这样的词偶有误升级,诊断会点名该文件使之可见),仍是错误。若 `dependencies` 或 `optionalDependencies` 也声明了同名 peer,则只按该具体条目判断;npm 从不读取重复 peer 的选择器。`workspace:`/`catalog:` 同样会被报告,除非 `prepack` 生命周期运行在 pnpm、Yarn 或 Bun 之下——它们会在打包时重写这些协议;`npm publish` 不会重写,因此在 npm 下 `AB7015` 会触发;tarball 自身携带的项——`bundleDependencies` 中 npm 实际打包进去的项(peer 除外),以及已打包来源可供安装的包内 `file:` 路径(带有可解析 `package.json` 的目录,或格式正确且其 `package.json` 可解析的 tarball)——不会被报告。请依赖已发布的版本,或把该包打包进产物并声明在 `devDependencies` 下。 | -依赖证据直接读取自打包后的字节:npm 将发布的每个 `.js`/`.mjs`/`.cjs` 文件都会被词法分析出 `import` +预构建 payload 的 `runtimeDependencies` 声明视为已使用,因为编译器从不打开 payload 文件。其余依赖证据直接读取自打包后的字节:npm 将发布的每个 `.js`/`.mjs`/`.cjs` 文件都会被词法分析出 `import` 说明符,并扫描字面量 `require("…")`、`require.resolve("…")` 与 `import.meta.resolve("…")` 调用(`createRequire(…)` 的绑定视同 `require`,即使导入时重命名了该工厂,直接调用的 `Module.createRequire(…)("…")` 与 `require("node:module").createRequire(…)("…")` 亦然;`path.resolve("…")` 不计入);每个打包后的 `.d.ts` 会被扫描其类型所引用的模块(`/// ` 指令同时计入该包及其 `@types/*` 对应包)。说明符先解码字符串转义再归约为包名,忽略 Node 内建模块。打包代码作为可执行文件运行的依赖——字符串字面量正是其已安装清单所声明的某个 `bin` 命令,如 `spawnSync("tsc", ["--version"])`——视为已使用;`#子路径` 导入计入 `imports` 映射所指向的每个包;消费者侧 `preinstall`/`install`/`postinstall` 脚本(不含 `prepare`——npm 从不为已发布的 tarball 运行它)或其通过 `npm run`(取 `run` 之后的第一个位置参数;`run` 前后的选项,带值与否均被跳过,其后的词如 `npm run setup -- dormant` 中的 `dormant` 是该脚本的参数)或直接的 `npm test`/`start`/`stop`/`restart`(没有 `restart` 脚本时,`npm restart` 依次运行 `stop` 与 `start`)委托的任何脚本点名(或运行其 `bin` 命令;字符串形式的 `bin` 以已安装清单的名字命名——清单按 npm 的方式读取,重复键以最后一个为准——依赖未在本地安装或其清单不是 JSON 时以去掉作用域的包名代替)的依赖 同样视为已使用。打包代码中的计算型 `import(表达式)` 或 `require(表达式)`(同样包括 `require.resolve`、`import.meta.resolve`、直接的 `createRequire(…)(…)` 或 `createRequire` 绑定)可能加载任何已声明的包,因此会整体撤回 `AB7014`;被 ESM 词法分析器拒绝的打包源码亦然,因为其中的 `import()` 调用无法被报告;把 `require` 当作值传递而非调用(`const load = require`、`fn(require)`)亦然,因为此后包可能以扫描看不到的名字被加载。注释中的提及只会保留某个依赖,绝不会报告它;`devDependencies` 永不检查,