Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/claude-orchestration-lineage-evidence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"agent-bundle": patch
---

Extend the Claude host capability table (`claude-2.1.250.json`, rendered on the hosts reference page) with live Claude Code 2.1.259 evidence on the `lineage.subagent-events`, `lineage.root`, `lineage.parent`, `lineage.depth`, and `lineage.mcp-correlation` rows: two `Agent` spawns issued in one message bind to their own spawn calls, `Explore` subagents carry the same lineage fields as `general-purpose` ones, `PostToolUseFailure` carries the subagent's `agent_id`, the root `session_id` survives `--resume` and `/compact`, and `claudecode/toolUseId` correlates MCP calls at depth 0, 1, and 2. Emitted host output and `adapterRevision` are unchanged. (#455)
23 changes: 22 additions & 1 deletion docs/audits/2026-09-03-claude-live-session-proofs.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,25 @@ is reproduced here beyond counts, codes, and field names.
the `native-host-smoke` workflow's Claude source leg will report
`harness-failure` on every run against 2.1.257.

## Re-run later the same day: unmodified harness, Claude Code 2.1.259

After the operator signed in again, `probe:install claude` — the checked-in
harness, byte-for-byte copy of `~/.claude/.credentials.json`, no local edits —
produced a signed-in isolated home at `/tmp/host-test/claude-home`
(`claude auth status` → `loggedIn: true`, `authMethod: claude.ai`). The three
proofs were re-run with `HOME=/tmp/host-test/claude-home` (raw logs under
`/tmp/claude-live/orch/`), right after that home had hosted the four-turn
orchestration capture (`docs/audits/2026-09-03-host-lineage-matrix.md`,
"orchestration run"):

| Proof | Result | Assertions that held |
| --- | --- | --- |
| `pnpm test:host-install:session:claude` | **pass** — 2/2, 0 skipped | the same assertions as above against Claude Code 2.1.259 |
| `pnpm test:packed:native:claude` | **pass** — 8/8, 0 skipped, no fix needed this time (defects 1–2 are merged) | `hosts = ['claude']`, `status: 'passed'`, Eval `pass: 1, fail: 0` |
| `AGENT_BUNDLE_NATIVE_CLAUDE_SMOKE=1 rstest …/native-claude-contract.test.ts` | **fail** — 17/18, `status: 'harness-failure'` | everything measured held again (`authentication.status: 'subscription-session'`, `version: '2.1.259'`, `validation.exitCode: 0`, `stream.activationEvidence: 'observed'`, `stream.authSource: 'non-environment'`, `stream.plugins: ['agent-bundle-native-smoke']`, `errorEnvelopes: []`, one `result`/`success`, `stderr.present: false`); sole diagnostic `claude-native.normal-home.changed` — defect 3, still open as [#439](https://github.com/ScriptedAlchemy/agent-bundle/issues/439), reproduced on 2.1.259 |

The real `~/.claude` was again never written by any of these runs.

## Cost and footprint

The two lineage captures reported `total_cost_usd` of roughly $0.44 and
Expand All @@ -112,4 +131,6 @@ isolated home used here was under `/tmp/claude-recapture/` or
claude` removed the probe's home (`/tmp/host-test/claude-home` and
`/tmp/host-test/claude-workspace` are gone; the captures under
`/tmp/host-test/claude/` remain) and `rm -rf /tmp/claude-recapture/proof-home`
removed the proofs' home.
removed the proofs' home. The 2.1.259 orchestration capture cost about $1.23
across its four turns (`total_cost_usd` 0.79 + 0.14 + 0.24 + 0.07); its
isolated home was removed by `probe:uninstall claude` after the re-run above.
172 changes: 148 additions & 24 deletions docs/audits/2026-09-03-host-lineage-matrix.md

Large diffs are not rendered by default.

26 changes: 26 additions & 0 deletions examples/host-test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ Two MCP servers ship in the plugin:
- `host-test` (generated routes, `src/mcp/host-test/tools/`): `dump` (filter by
any conversation/session/subagent id, `full` for raw lines) and `reset`. Each
`dump` call records the request context the generated server mounted for it.
A bare `dump` returns the newest 50 matching records — a whole log of a few
hundred records overflows the tool-result document — so pass `limit` (up to
5000) for more; `matched` and `total` always count the whole log.
- `host-test-raw` (hand-rolled stdio factory, `src/mcp/host-test-raw.ts`):
`probe` records the raw SDK request context — session id, JSON-RPC id,
`_meta`, lifted envelope, negotiated client info — so hook↔MCP correlation is
Expand Down Expand Up @@ -77,6 +80,29 @@ pnpm --filter @agent-bundle-example/host-test probe:uninstall claude
land in `/tmp/host-test/<host>/`, followed by a rendered `host-test dump`.
Earlier runs' records are never re-copied, and the command exits non-zero
when the host fails or when the run produced no hook record or no MCP record.
- `probe:capture <host> --scenario <file.json>` replaces the default prompt with
an ordered list of turns (`{ "turns": ["...", "..."] }`; a turn may also be
`{ "prompt": "..." }`). For Claude every turn after the first runs
`claude -p --resume <session_id>` against the session the first turn's
`system/init` envelope reported, so one capture holds a multi-turn session
with `SessionStart source: resume` and one `SessionEnd` per invocation.
`Stop` follows the prompt path rather than the invocation: a turn whose
background subagents finish re-prompts itself with a `<task-notification>`
and stops twice, while a `/compact` turn submits no prompt and never stops
(`fixtures/host-lineage/claude-2.1.259-orchestration.ndjson`). If the first
turn reports no `session_id`, the capture fails instead of running the
remaining turns as fresh sessions; `--prompt` and `--scenario` refuse each
other, and `--scripted-model` plays a fixed transcript and refuses both.
Codex and Cursor drivers take
the first turn only and refuse longer scenarios. `scenarios/claude-orchestration.json` is
the checked-in orchestration scenario (two parallel `Agent` spawns, one
sequential spawn that nests another, the `host-test:host-test` skill, a
plugin-command probe, a manual `/compact`, a final stop).
- Claude turns run with `--output-format stream-json --verbose`, so the model's
own tool-use stream is saved next to the hook payloads as
`session-<stamp>[.turn-N].stream.ndjson`; envelopes produced inside a
subagent carry `parent_tool_use_id` (the parent's `Agent` `tool_use_id`),
which is how the transcript and the hook log are cross-checked.
- Host processes get an allowlisted environment (PATH, locale, display, proxy,
TLS plumbing) plus the isolated `HOME`; nothing else from your shell is
inherited, and even allowlisted values are dropped when they carry a
Expand Down
10 changes: 10 additions & 0 deletions examples/host-test/scenarios/claude-orchestration.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "claude-orchestration",
"description": "Multi-turn Claude Code session that exercises the host's own orchestration surfaces: two parallel Agent spawns, one sequential spawn that nests another, the installed skill, a plugin-command probe, a manual /compact, and a final stop. Each turn after the first resumes the same session.",
"turns": [
"You are exercising the host-test probe plugin inside an isolated harness. Do exactly these steps in order, without asking questions and without stopping early.\n1. Run the shell command `pwd`.\n2. Call the `dump` tool of the host-test MCP server with {} as arguments and remember its `log.path`.\n3. In ONE assistant message, spawn TWO subagents with the Agent tool IN PARALLEL (two Agent tool calls in the same message):\n (a) subagent_type \"Explore\", prompt: \"Run the shell command `ls -la`. Then call the host-test MCP server's `dump` tool with {} and the host-test-raw MCP server's `probe` tool with {\\\"note\\\":\\\"parallel-explore\\\"} if those tools are available to you. Reply with every id you saw and state explicitly whether the MCP tools were available to you.\"\n (b) subagent_type \"general-purpose\", prompt: \"Run the shell command `pwd`, call the host-test MCP server's `dump` tool with {}, call the host-test-raw MCP server's `probe` tool with {\\\"note\\\":\\\"parallel-general\\\"}, then reply with every id you saw.\"\n4. After BOTH have returned, spawn ONE more subagent sequentially with the Agent tool, subagent_type \"general-purpose\", prompt: \"Run the shell command `pwd`, call the host-test MCP server's `dump` tool with {}, call the host-test-raw MCP server's `probe` tool with {\\\"note\\\":\\\"sequential\\\"}. Then spawn a nested subagent with the Agent tool (subagent_type \\\"general-purpose\\\") and instruct it to run `pwd`, call the host-test `dump` tool with {}, call the host-test-raw `probe` tool with {\\\"note\\\":\\\"nested\\\"}, and reply with every id it saw. Finally reply with every id you and the nested subagent saw.\"\n5. Reply with exactly one final line: HOST_TEST_TURN1_DONE <the log.path from step 2>",
"Same session, next turn. Do exactly these steps in order, without asking questions.\n1. Invoke the installed host-test skill through the Skill tool (its name is `host-test:host-test`) and follow the steps it lists, using {\"note\":\"root-skill\"} for the probe call; do NOT spawn any subagent this turn.\n2. If the host-test plugin provides any slash command or custom command other than that skill, run it once; if it provides none, write the single word NO_PLUGIN_COMMAND.\n3. Reply with exactly one final line: HOST_TEST_TURN2_DONE",
"/compact",
"Same session, final turn. Run the shell command `pwd` once more, then reply with exactly one final line: HOST_TEST_DONE <the log.path you learned earlier in this session>"
]
}
Loading
Loading