fix: stream generated MCP/event Flight bytes before render completion

[ScriptedAlchemy]

Closes #686

What

Generated MCP and event-route Flight workers (generatedRouteFlightWorkerSource in src/build/entry-shell.ts) buffered renderAgentFlight with new Response(flight).arrayBuffer() and posted one complete message. They now read the Flight stream and post each chunk over the existing chunk/end/error transport the rendered CLI/script worker already speaks — no second renderer, mode, or queue.

Consumers switch from a resolved promise to a live ReadableStream:

• src/mcp-server-runtime.ts (createFlightWorkerHost) — pending render holds the stream controller; chunk enqueues, end closes, error errors, worker exit/error fail every pending stream.
• src/adapters/hook-contract.ts (standalone hook wrapper) — same shape.
• src/dev/routes/route-invocation-production.ts — still accepts complete (whole render in one message) from Flight workers compiled before this PR, since the epoch store restores such artifacts across dev-server restarts until the project rebuilds.

All three hosts (plus the production host, which already spoke chunk) gained a cancel hook on the stream: the dispatcher's Flight reconciler cancels the reader when its session closes, which with chunked delivery can precede the worker's end. Without the hook a late end/chunk hit a closed controller and crashed the host process (ERR_INVALID_STATE, surfaced as "Connection closed" on the next stdio call). Cancel now drops the pending entry and tells the worker to cancel, so the render is released and nothing reaches a closed stream.

Context, provider/state lifetime, byte order and flow control are unchanged: chunks are posted inside runAgentRequest's scope and before bindings.close(), postMessage preserves order, and demand is still bounded by createFlightDemand + the SSE route's drain pacing.

Coordination with #681 / #699

Events still flow one at a time through publishRender and the dispatcher; this PR adds no buffer, so #699's retainRenderEvent bound applies unchanged whichever lands first (no overlapping hunks in route-invocation-production.ts). Comment left on #681.

Acceptance

• tests/entry-shell.test.ts — worker source posts chunk inside the request scope and end after observed-render-finish; no arrayBuffer/complete; digest updated.
• tests/route-invocation-dev-server.test.ts — through the Workbench production surface (/api/routes/invocations + /stream): gated tool:status/live publishes its Agent.Progress fallback shell before the gate file exists, then shell → replace → complete whose document equals the ungated control run; cancel while blocked yields exactly ['shell'] and a cancelled final with nothing after the gate releases; tool:status/crash settles with exactly one complete and the same status/outcome/document as the non-streamed run; the same for the compiled event:tool/after route with manifest-selected claude host and preflight (shell → replace → complete, document/result/trace equal to the completed run).
• tests/generated-route-server.test.ts — real generated stdio server: notifications/progress for the fallback arrives before the gate file is written; the Decide and pin the thrown-route-error projection per surface instead of adopting error.tsx/loading.tsx file conventions (xref row 8) #492 thrown-route test now also proves the host survives a session-close-before-end race.
• packages/workbench/tests/streaming-render.e2e.test.ts (new, added to the integration pool) — Playwright at 1440×900: Rendered pane shows the streaming progress node with aria-busy="true" while the gate is closed, then stream complete with no fallback, equal to the already-open-gate control run. Verified to fail against the buffering worker (fallback never appears within 15 s) and pass after.
• Shared fixture: tests/helpers/gated-routes.ts.

Local gate (final SHA c89eef2, rebased onto origin/main @ b805eec; machine load avg 90–120 from unrelated Rust builds)

pnpm build && pnpm typecheck && pnpm lint                                            → pass
pnpm test:unit                                                                       → 4405 passed, 0 failed
rstest --config rstest.integration.config.ts route-invocation-dev-server.test.ts generated-route-server.test.ts
  streaming-render.e2e.test.ts dev-workbench.test.ts                                 → 56/56
pnpm test:packed (at e2b8db46f7, same tree minus #720's test-only commit)             → 39/39
pnpm test:integration:run (whole pool, at 43cc77af0c before the review fixes)         → 1164 passed, 0 failed, 4 skipped

Docs: website/docs/*/guide/authoring/mcp.mdx already describes streaming Suspense fallbacks and per-shell progress notifications; this PR makes the generated workers match it, no wording change. Changeset: .changeset/686-stream-generated-flight.md.

Deslop: Claude Fable 5.1, 3 edits (dropped try { controller.error() } catch {} wrappers — error() is a no-op on a non-readable stream; removed a redundant onError alias; extracted the duplicated gated fixture).

Also fixed on the way: a rejected progress.report() in the MCP host (session already completed/failed) used to be an unhandled rejection; it now cancels the worker render and settles a still-pending stream once. An already-aborted signal no longer registers a pending entry or an abort listener.

Self-review

Reviewer: GPT-5.6 Sol Max (generalPurpose; the change-risk-reviewer could not run because the TraceDecay daemon is down), two passes over origin/main...HEAD.

Pass 1 findings and disposition:

1. No backpressure between worker and host (High) — dismissed. Bytes in flight per request are bounded by one render's Flight output, exactly what arrayBuffer() previously held whole in the worker and then in a one-chunk stream; postMessage transfers the buffer without copying; downstream decode is paced by createFlightDemand and the SSE route by socket drain. A credit protocol would be the second queue/mode Stream generated MCP/event Flight bytes before render completion #686 forbids.
2. Unhandled progress.report() rejection in mcp-server-runtime.ts (High) — fixed: Promise.resolve(report).catch(...) cancels the worker render and settles a still-pending stream once; a late rejection after settlement is dropped.
3. Removing complete breaks restored pre-PR epochs (Medium) — fixed: route-invocation-production.ts accepts complete again with a comment naming the restored-epoch case.
4. Event fixture never gated: canonical.payload.toolInput?.gate is always undefined (payload fields are { nativeKey, value }) (Low) — fixed: fixture reads toolInput?.value?.gate; releaseGate returns the release instant and both the tool and the event stream assert completedAt >= releasedAt, so the fallback is proven to come from a blocked render, not a replay.
5. Listener registered on an already-aborted signal (Low) — fixed in mcp-server-runtime.ts and the hook wrapper: error the stream and return before registering.

Pass 2 (after fixes): one finding — a failed progress report settled the stream without cancelling the worker render (Medium) — fixed by posting cancel before settle. Reviewer reported no other material regressions (terminal/abort/cancel races settle once, legacy complete path safe, aborted early return does not touch the worker).

[changeset-bot]

🦋 Changeset detected

Latest commit: c89eef2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
agent-bundle	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-06T21:33:26.295987Z	43cc77a	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 43cc77af0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ScriptedAlchemy]

Coordination from #681: the surviving retention primitive is #715 (packages/agent-bundle/src/dev/routes/route-invocation-render-history.ts, retainRenderEvent, 256 events / 2 MiB, pinned newest + newest document-bearing event). #699 is closed as its duplicate; its unique tests were ported into #715.

Unlike #699, #715 does touch streamFromWorker in route-invocation-production.ts, so git merge-tree reports two conflicts with this branch:

• route-invocation-production.ts, streamFromWorker.execute: fix(workbench): bound producer / completed-envelope render history end to end #715 types execute as (dispatch: AgentRenderDispatch) and stores progress: dispatch.progress in the pending entry (worker progress messages become progress render events through entry.progress?.report). Keep this PR's cancelRender / ReadableStream({ cancel }) structure and fix(workbench): bound producer / completed-envelope render history end to end #715's signature plus the progress field on the entry. Your complete removal and fix(workbench): bound producer / completed-envelope render history end to end #715's progress branch don't overlap.
• route-invocation-dev-server.test.ts: both add imports/helpers and new it blocks; keep both sets.

#715's renderCompiled awaits publishRender per event and keeps only the complete document, which matches your "no buffer added" claim — chunked Flight delivery from this PR simply makes those events arrive live instead of at completion. Whichever lands second resolves as above.

[ScriptedAlchemy]

Ack on #715: this PR lands first (gate green on c89eef2 over origin/main @ b805eec). Two notes for the #715 rebase: (1) streamFromWorker here keeps complete as a legacy alias for epochs restored from pre-#718 builds, so keep that branch alongside #715's progress branch; (2) the stream's cancel hook and cancelRender must stay — the dispatcher's reconciler cancels the reader when its session closes, which with chunked delivery can precede the worker's end. Everything else resolves as described above.