Prove successful native Claude Code / Codex host sessions in Workbench

[ScriptedAlchemy]

Priority and scope

P2 — native acceptance and qualification gap; PTY implementation is delivered. Follow-up to #675 and #600, preserving the install/cleanup fixes in #677/#678.

Audited commit: b435f7b9179271cbff81d3e40d14ee342cbd65dd.

Origin: owner-requested deep-dive, 2026-09-06. Expanded with the complete native acceptance and test-hardening criteria.

Problem

Fake-host tests establish useful PTY/transport/lifecycle behavior, not successful authenticated Claude/Codex model-driven tool-and-hook completion. The prior #675 acceptance run recorded Claude authentication and Codex usage-limit blockers; those are not successful native execution evidence.

The opt-in native-host-sessions.test.ts currently has two weaker conditions:

1. A requested native suite can skip a host when its --version invocation fails. enabled(host) combines the opt-in flag with executable availability and chooses it.skip otherwise.
2. Success requires only a correlated outgoing mcp.request whose summary includes tools/call. It does not require the matching successful response, expected tool value, or corresponding real hook.

A terminal launch, outgoing request, authentication refusal, usage-limit refusal, or all-skipped run must not be reported as completed native acceptance.

Existing implementation to preserve

HostSessionService is mounted by the production Workbench server and uses the PTY adapter and attached development installation. Existing fake-host acceptance remains valuable and should stay separately labelled. This issue does not request a host-session redesign or assert that the implementation is wholly missing.

Acceptance

• Default unauthenticated CI may explicitly skip native acceptance; it must label that outcome clearly.
• When AGENT_BUNDLE_NATIVE_HOST_CONTRACTS=1 requests native qualification, missing executable/authentication/quota prerequisites fail or explicitly block that qualification rather than producing an all-skipped success.
• Use a harmless deterministic plugin operation with a known result; avoid unnecessary dependence on a third-party service merely to prove host integration.
• For both logged-in Claude Code and Codex, attach the plugin through the supported native development-install path and complete a model-driven tool call.
• Match the outgoing tools/call to a successful response and expected value under the intended host/session identity.
• Observe a corresponding intended real hook receipt and correlate it with the session; do not substitute a test-injected receipt.
• Exercise cancellation, resize, restart, and termination; verify cleanup settles and no owned process or stale development registration remains after shutdown.
• Preserve native authentication, trust prompts, and permission behavior. No approval-bypass flags or synthetic success in place of native evidence.
• Record the repository SHA, exact host versions, relevant sanitized environment/prerequisite status, executed/skipped cases, and redacted evidence.
• Keep deterministic fake-host proof, real host installation/adoption proof, and authenticated tool/hook completion proof distinct in tests and documentation.

Test owners

• packages/agent-bundle/tests/native-host-sessions.test.ts
• packages/workbench/tests/sessions.e2e.test.ts
• packages/agent-bundle/tests/support/fake-host-cli/
• existing host-install/cleanup regression suites

Non-goals

No credentials in issues/logs/screenshots/artifacts, no weakening native permissions, no remote PTY service, no new host support, and no requirement to run paid authenticated model calls in every ordinary PR. Record a blocked run honestly rather than redefining acceptance to hide it.

[ScriptedAlchemy]

Native qualification completed on 2026-09-06; both real hosts passed.

Environment: Agent Bundle b435f7b917; Node 22.23.2; Claude Code 2.1.263 (claude.ai Max); Codex 0.153.4 (ChatGPT, one owner-authorized expiring usage reset redeemed); Workbench 1440×900; adopted epoch b3a538f3-bb2e-48ea-91d8-892cd43abb74; project examples/audiobook-curator. A temporary standalone session/start fixture was used so hook correlation was real rather than inferred.

Codex

• Host session hs_e408d8783707a924 attached to conversation 01a07897-9dbd-7e33-811b-c48b8bddae82.
• Trace: session.started → mcp initialize ok → tools/list ok → hook.received codex SessionStart / hook.completed → tools/call search_audible (trc_18) → tools/call search_audible ok (trc_19).
• Model response matched the tool result: 20 candidates, 0 errors, exit code 0; top result Dune / Frank Herbert / ASIN B002V1OF70; no mutation.
• Cancellation: a real sleep 60 turn was interrupted and terminal reported Conversation interrupted.
• Lifecycle: resize 110×35 → restart hs_95e7ef8698ccd095 with restartOf=hs_e408d8783707a924 → terminate SIGTERM → forget 204.

Claude

• Host session hs_e67892336a01939f attached to conversation 90dd0479-b9e2-46b8-aa32-ca3a3bd19a2f.
• Trace: real SessionStart hook received/completed; tools/call search_audible (trc_50) → tools/call search_audible ok (trc_51), with pre/post-tool hooks sharing the same conversation id.
• Terminal rendered the successful 20-candidate Dune result.
• Cancellation: a real sleep 60 turn reported Interrupted · What should Claude do instead?.
• Lifecycle: resize 112×36 → restart hs_a2c811ebca0d682d with the prior id → terminate (exit 143) → forget 204.

The qualification did expose two separate dev-path defects, now tracked as #716 (Codex’s persistent plugin app-server required a remove/add refresh before the installed MCP component became visible) and #717 (the watcher missed a route created under a previously absent nested directory; restart discovered it). They do not invalidate the successful native-session result after the dev install was refreshed.

[ScriptedAlchemy]

Follow-up #717 is merged in b805eec98b7ed75fe657b824c07bb160e9637cce via #720. A fresh built dev server detects a previously absent src/events/session/start.ts immediately and emits/removes both Claude and Codex hook outputs on successive epochs; the same acceptance covers an absent MCP tool root. The reported native miss came from the checkout's ignored stale dist, so no duplicate route-kind watcher was added.

[ScriptedAlchemy]

Follow-up #716 is fixed on main by 25bf66c7ae969554c8cdd1957dafc3a43ec3860e (#722). agent-bundle dev --install-host codex now refreshes an already-running Codex app-server through its documented local control transport and idempotent plugin/install operation before reporting the epoch attached. The persistent fake contract and real Codex 0.153.4 proof both passed with the app-server started first, MCP and hook components visible, the installed hook-document digest changing after rebuild, and exactly one registration retained; no app-server restart, process kill, or temporary plugin source is used.