Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/682-inspector-provenance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"agent-bundle": patch
---

Make Workbench Artifact file details render each file's own `compiler.provenance` source inputs and show `No source inputs recorded` for an empty record. Add an internal `AB6200` missing-row guard without remapping malformed on-disk manifests, which continue to fail parsing as `AB6001`. (#719)
6 changes: 3 additions & 3 deletions docs/diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ even when no error diagnostic was reported.
| `AB48xx`/`AB494x` | Route graph, state, layout (`AB4830`–`AB4832`), generated route declarations outside the TypeScript program (`AB4834`), route render budgets (`AB4835`), tool task support (`AB4836`), a route module that value-imports a compiler-carrying framework entry (`AB4837`), a CLI route `inputSchema` reference the static resolver cannot follow (`AB4838`) or that cycles (`AB4839`), an event route's `preflight` gate export (`AB4840`), an event route's declared provider keys (`AB4841`), a CLI surface projection of an MCP tool (`AB4843`–`AB4845`), and provider conventions (see below). |
| `AB5000` | General CLI and adapter failures (see below). |
| `AB60xx` | Built-artifact validation, including schema documents and referenced files (`AB6005`: the compiler finds a host-pack surface or package-build entry (`dist/bin/*.js`, the Flight workers, or the `lib` entry) that keeps something other than a Node built-in, `pnpapi`, or an emitted sibling external, or an MCP App view that keeps anything external; the emitted-module walk remains only for what the compiler cannot see — an expression `import()` in a compiled module, and the imports and syntax of JavaScript the framework did not compile or a `tools` hatch may have rewritten; a `dist` finding names `dist/<path>`; `AB6011`/`AB6012`: a target's required pinned-schema document is missing or invalid; `AB6025`: a manifest-declared `logo` path is missing from the artifact or escapes the deploy tree; `AB6034`: emitted Skill Markdown has no instruction body; `AB6035`–`AB6038`: Agent Plugins portable validation, see below). |
| `AB6200`–`AB6202` | Workbench artifact inspection over published epochs: `AB6200` the epoch does not validate or its provenance is inconsistent, `AB6201` an epoch reference could not be released, `AB6202` unsafe runtime metadata (see below). |
| `AB6200`–`AB6202` | Workbench artifact inspection over published epochs: `AB6200` the validator threw or an internal post-validation invariant failed, `AB6201` an epoch reference could not be released, `AB6202` unsafe runtime metadata. Artifact-validation diagnostics such as `AB6001` retain their original codes (see below). |
| `AB700x` | Host installation and uninstallation: bundle identity, host availability, scope, command failure, and collision checks (`AB7000`–`AB7004`: unsupported host, unreadable bundle identity, missing host, scope or mode refusal, host command failure — the same five codes are also the development project service's preparation failures; `AB7001` in detail: the composite root at `--from` cannot be resolved for the host from its `agent-bundle.manifest.json` — the manifest is missing or not canonical, has no `projections[]` row for the host, the row has no host plugin manifest pointer or the pointed file is missing, a `files[]` row is missing or its bytes, size, digest, or executable state differ from the row after npm normalization, `claude`/`codex` have no marketplace identity, or the `cursor` plugin name is not a safe local plugin name; `install`, `uninstall`, and `doctor` never probe `.claude-plugin/plugin.json` or look under `<from>/<host>`; `AB7005`: version collision, pre-receipt content collision, or foreign install; `AB7006`: the host lists the installed copy with load errors; see below), plus the `uninstall` refusals `AB7007`–`AB7009` (ownership or content mismatch, unconfirmed data purge, missing receipt; see below). |
| `AB7010`–`AB7015` | npm prepack inventory, artifact freshness, package bin targets, release-version agreement, and installed-dependency hygiene (`AB7014`: a dependency no consumer-runtime evidence requires; `AB7015`: a git, remote-tarball, path, or unrewritten workspace-protocol dependency specifier). |
| `AB7200`–`AB7202`, `AB7210`–`AB7211` | Development rebuilds and live host surfaces: rebuild admission and phase failures, development host install sync, and the dev-epoch contract gate (see below). |
Expand Down Expand Up @@ -1876,7 +1876,7 @@ therefore does not prove the absence of such a load.
| Code | Severity | Trigger | Recovery |
| --- | --- | --- | --- |
| `AB6000` | error | `Artifact root is not a readable directory.` — the artifact root cannot be walked; `Artifact manifest is missing or cannot be read.` — the tree could not be inspected, or `agent-bundle.manifest.json` is absent, is not a regular file, or could not be read (the manifest is read between two identity checks, so a manifest replaced mid-read reports here too). Validation stops at this code. | Restore a readable artifact root and canonical manifest, then rebuild the artifact. |
| `AB6001` | error | `Artifact manifest is not a strict canonical manifest.` — `agent-bundle.manifest.json` does not parse as a strict canonical artifact manifest. `Artifact manifest changed during validation.` — its bytes or identity differ between the first read and the re-read after validation. | Regenerate the strict canonical manifest without concurrent writes, then rerun validation. |
| `AB6001` | error | `Artifact manifest is not a strict canonical manifest.` — `agent-bundle.manifest.json` does not parse as a strict canonical artifact manifest. This includes a `files[]` row with no matching `compiler.provenance` row: the parser's exact-path rule rejects it before inspection, and the diagnostic keeps this generic message. `Artifact manifest changed during validation.` — its bytes or identity differ between the first read and the re-read after validation. | Regenerate the strict canonical manifest without concurrent writes, then rerun validation. |
| `AB6002`–`AB6003` | error | Reserved: both codes are declared in the artifact diagnostic registry, but no validator emits either today. | `AB6002`: Rebuild the artifact from complete project source, then rerun validation. `AB6003`: Rebuild the artifact with canonical generated output, then rerun validation. |
| `AB6004` | error | `Artifact files do not match the manifest.` — the regular files on disk differ from the manifest file table (a path, byte length, mode, or SHA-256; a missing or unmanifested file). `Artifact file changed during validation: "<path>".` — a file differed between the initial and final inspection, or between a validated staging tree and its re-check after `build` renamed it into place. `Artifact file table changed during validation.` — the final inspection could not be taken. | Rebuild the artifact so its file table and contents match the manifest. |
| `AB6005` | error | Primary compile-time form: `Compiled module "<asset>" keeps "<request>" external (<external type>) from <issuer module>; a generated executable bundles everything but Node built-ins.` `<request>` is the run-time load target; when an object-map external redirected the authored specifier, `, imported as "<specifier>",` follows the type, a relative target that names no emitted asset of the artifact (or escapes it) ends `; it names no module emitted by this artifact.` instead, and a request kept under an external type that does not load a module (`var`, `global`, `this`, `window`, `assign`, `umd`, `amd`, `system`, `jsonp`, `promise`, `script`, …) — even a Node built-in — ends `; external type <type> reads a variable instead of loading a module.` — the compiler service lowered a host-pack surface or package-build entry (`dist/bin/*.js`, the Flight workers, or the `lib` entry) and Rspack kept something other than a Node built-in, `pnpapi`, or an emitted sibling of that artifact external, whatever spelling Rspack emitted (`import`, `require`, or its `createRequire` shim); `generatedPath` names the asset. An expression request (`import(expr)`, `require(expr)`) is outside the compiler's view: Rslib's profile leaves it verbatim without parsing it, so it is neither bundled nor external; so is a literal import marked `rspackIgnore`/`webpackIgnore`, which Rspack leaves verbatim with no module, external, or warning. The emitted-module walk reports both in a compiled module the record proves: `Generated JavaScript import from "<module>" has a non-literal dynamic import.` and `Generated JavaScript import from "<module>" loads "<request>", which the compiler neither bundled nor recorded as an external; an import the build ignored is a run-time load outside the artifact.` (a literal request that is neither a Node built-in nor one of the record's externals for that file). MCP App view form (the `, imported as` clause applies to both forms): `Compiled MCP App view "mcp-apps/<name>.html" keeps "<request>" external (<external type>) from <issuer module>; a view inlines every module it loads.` — a browser document has no allowable external, so the view's Rsbuild compilation (which carries the same audit plugin) fails on any `ExternalModule`, whatever the hatch mapped it to. Residual walk forms, `Generated JavaScript import from "<module>" <finding>.`, reported only for what the compiler cannot see: `has a non-literal dynamic import` for an expression `import()` in any emitted module, compiled ones included, since Rslib's profile leaves that form verbatim; and, for a module the compile evidence record does not prove — JavaScript the framework did not compile (`install.mjs`, a copied script), every module of an artifact without a record, and every module of a build whose `tools` hatch may have rewritten the emitted bytes (`coverage.rewritable`) — `has invalid syntax` from a full parse, `uses unsupported specifier` or `uses invalid specifier` for a bare or malformed import, `cannot be read`, and the relative-target findings `is missing`, `resolves outside the artifact root`, `is not listed in the artifact manifest`, `does not resolve to a regular file`, `references invalid JSON`, and `uses unsupported target`. A compiled module the record covers with the same bytes is lexed for syntax and its literal imports are not resolved again — the compiler resolved them (bundled, built-in, or an emitted sibling). Prebuilt payloads are not walked; a `dist` finding names `dist/<path>`. | Bundle every JavaScript dependency into the artifact, then rebuild it. |
Expand Down Expand Up @@ -1909,7 +1909,7 @@ diagnostics in the response body.

| Code | Severity | Trigger | Recovery |
| --- | --- | --- | --- |
| `AB6200` | error | `Artifact inspection could not validate the published artifact.` — the strict artifact validator threw over the epoch; `Artifact file provenance references an unknown project source input.` — an output's `sourceInputs` name an input the manifest project does not declare; `Artifact manifest project inputs are invalid.` — the manifest's project inputs are structurally invalid. An epoch whose validation merely reports diagnostics is refused with those diagnostics instead of this code. | Rebuild the epoch from a project whose artifact validates cleanly. |
| `AB6200` | error | `Artifact inspection could not validate the published artifact.` — the strict artifact validator threw over the epoch; `Artifact file has no manifest provenance record.` — an internal defense-in-depth invariant failed after parsing (an on-disk missing row is rejected first as `AB6001`); `Artifact file provenance references an unknown project source input.` — an output's `sourceInputs` name an input the manifest project does not declare; `Artifact manifest project inputs are invalid.` — the manifest's project inputs are structurally invalid. An epoch whose validation merely reports diagnostics is refused with those diagnostics instead of this code. | Rebuild the epoch from a project whose artifact validates cleanly. |
| `AB6201` | error | `Artifact inspection could not release every acquired epoch reference.` — releasing an epoch reference after an inspection or diff failed. | None in the project: the failure is internal to the development server's epoch bookkeeping. |
| `AB6202` | error | Runtime metadata derived from the validated snapshot is unsafe: an MCP server's `entryPaths` name a file outside its target or absent from the manifest (`Validated MCP evidence references an unmanifested target file.`), or another runtime-evidence check named in the message failed. | Rebuild the epoch so its MCP runtime evidence references manifested target files. |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ export class ArtifactInspectionService {
);

const files = Object.freeze(manifest.files
.map((file) => this.#file(file, sourceInputs, provenanceByPath.get(file.path) ?? []))
.map((file) => this.#file(file, sourceInputs, provenanceByPath.get(file.path)))
.sort(comparePaths));
const filesByPath = new Map(files.map((file) => [file.path, file]));
const project = this.#project(manifest.compiler.project, sourceInputs);
Expand Down Expand Up @@ -224,8 +224,15 @@ export class ArtifactInspectionService {
#file(
file: ArtifactManifest['files'][number],
sourceInputs: ReadonlyMap<string, ArtifactInspectionSourceInput>,
provenanceInputs: readonly string[],
provenanceInputs: readonly string[] | undefined,
): ArtifactInspectionFile {
if (provenanceInputs === undefined) {
throw inspectionError(
'ARTIFACT_INSPECTION_INVALID',
'Artifact inspection requires a manifest provenance record for every output file.',
inspectionDiagnostic('AB6200', 'Artifact file has no manifest provenance record.', file.path),
);
}
const inputs = provenanceInputs.map((path) => sourceInputs.get(path));
if (inputs.some((input) => input === undefined)) {
throw inspectionError(
Expand Down
105 changes: 103 additions & 2 deletions packages/agent-bundle/tests/artifact-inspection-service.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { supportedCapabilities } from './support/adapter-capabilities.ts';
import { chmod, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
import { chmod, mkdir, mkdtemp, readFile, rename, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';

Expand All @@ -10,11 +10,12 @@ import type { TargetAdapter, TargetAdapterMetadata } from '../src/adapters/types
import {
artifactCompilerRecordVersion,
assembleArtifactManifest,
parseArtifactManifest,
type ArtifactManifestFileKind,
type ArtifactManifest,
} from '../src/build/manifest.ts';
import { validateArtifact, validateArtifactWithSnapshot } from '../src/build/validate-artifact.ts';
import { digest } from '../src/core/digest.ts';
import { digest, stableJson } from '../src/core/digest.ts';
import { ArtifactInspectionService } from '../src/dev/index.ts';
import { EpochStore } from '../src/dev/epoch-store.ts';
import type { ArtifactEpoch } from '../src/dev/types.ts';
Expand Down Expand Up @@ -603,6 +604,106 @@ it('revalidates an epoch on each inspection so post-publication corruption is vi
}
});

interface ProvenanceTamperCase {
readonly name: string;
readonly rule: string;
readonly tamper: (provenance: { path: string; sourceInputs: string[] }[]) => { path: string; sourceInputs: string[] }[];
}

const provenanceTamperCases: readonly ProvenanceTamperCase[] = [
{
name: 'missing: an output file has no provenance row',
rule: 'compiler.provenance paths must exactly match files.',
tamper: (provenance) => provenance.filter((entry) => entry.path !== 'mcp/runner.mjs'),
},
{
name: 'conflicting: a provenance row names an input the project never declared',
rule: 'compiler.provenance[mcp/runner.mjs].sourceInputs contains an undeclared project source input.',
tamper: (provenance) => provenance.map((entry) => entry.path === 'mcp/runner.mjs'
? { ...entry, sourceInputs: ['src/elsewhere.ts'] }
: entry),
},
{
name: 'conflicting: two provenance rows claim the same output file',
rule: 'compiler.provenance must be sorted with no duplicate entries.',
tamper: (provenance) => provenance.flatMap((entry) => entry.path === 'mcp/runner.mjs'
? [entry, { ...entry, sourceInputs: [configPath] }]
: [entry]),
},
{
name: 'relocated: a provenance row names an output the artifact does not carry',
rule: 'compiler.provenance paths must exactly match files.',
tamper: (provenance) => provenance.map((entry) => entry.path === 'mcp/runner.mjs'
? { ...entry, path: 'mcp/moved/runner.mjs' }
: entry),
},
];

it.each(provenanceTamperCases)('refuses to inspect an epoch whose provenance is $name', async ({ rule, tamper }) => {
// Every case is a manifest the compiler cannot emit (`createOutputProvenance`
// writes one row per output). A copy edited afterwards must fail closed at
// the manifest parser, so the inspector never renders a file with invented,
// empty, or borrowed source inputs.
const root = await mkdtemp(join(tmpdir(), 'agent-bundle-artifact-inspection-provenance-tamper-'));
const registry = runtimeRegistry();
const store = new TrackingEpochStore({ projectRoot: root });
const epochId = 'epoch-provenance-tamper';

try {
await publish({ files: runtimeFiles(), id: epochId, registry, root, store });
const manifestPath = join(root, '.agent-bundle', 'epochs', epochId, 'agent-bundle.manifest.json');
const published = JSON.parse(await readFile(manifestPath, 'utf8')) as {
compiler: { provenance: { path: string; sourceInputs: string[] }[] };
};
published.compiler.provenance = tamper(published.compiler.provenance);
const tampered = `${stableJson(published)}\n`;
expect(() => parseArtifactManifest(tampered)).toThrow(rule);
await writeFile(manifestPath, tampered);

await expect(new ArtifactInspectionService(store, registry).inspect(epochId)).rejects.toMatchObject({
code: 'ARTIFACT_INSPECTION_INVALID',
diagnostics: [expect.objectContaining({ code: 'AB6001', generatedPath: 'agent-bundle.manifest.json' })],
});
expect(store).toMatchObject({ acquired: 1, closed: 1 });
} finally {
await rm(root, { force: true, recursive: true });
}
});

it('inspects identical root-relative provenance after the published epochs are relocated', async () => {
const root = await mkdtemp(join(tmpdir(), 'agent-bundle-artifact-inspection-relocated-origin-'));
const relocated = await mkdtemp(join(tmpdir(), 'agent-bundle-artifact-inspection-relocated-'));
const registry = runtimeRegistry();
const epochId = 'epoch-relocated';

try {
await publish({ files: runtimeFiles(), id: epochId, registry, root, store: new EpochStore({ projectRoot: root }) });
const before = await new ArtifactInspectionService(new EpochStore({ projectRoot: root }), registry).inspect(epochId);

// The epoch directories (bytes and store metadata) move to another project
// root; the origin keeps its active-build pointer. Acquiring an epoch by id
// reads only its directory and metadata id, so the inspection depends on
// nothing but the relocated manifest bytes.
await mkdir(join(relocated, '.agent-bundle'), { recursive: true });
await rename(join(root, '.agent-bundle', 'epochs'), join(relocated, '.agent-bundle', 'epochs'));
const store = new TrackingEpochStore({ projectRoot: relocated });
const after = await new ArtifactInspectionService(store, registry).inspect(epochId);

expect(after).toEqual(before);
expect(after.provenance).toContainEqual({
outputPath: 'mcp/runner.mjs',
sourceInputs: [{ path: runnerSourcePath, sha256: fixtureInputs[1]!.sha256 }],
});
const serialized = JSON.stringify(after);
expect(serialized).not.toContain(root);
expect(serialized).not.toContain(relocated);
expect(store).toMatchObject({ acquired: 1, closed: 1 });
} finally {
await rm(root, { force: true, recursive: true });
await rm(relocated, { force: true, recursive: true });
}
});

it('returns deeply frozen detached inspection records', async () => {
const root = await mkdtemp(join(tmpdir(), 'agent-bundle-artifact-inspection-immutable-'));
const registry = runtimeRegistry();
Expand Down
Loading
Loading