fix(codex): frame large app-server records linearly

[clintebbesen]

Fixes #5389.

The Codex app-server protocol reader repeatedly concatenated and rescanned an unfinished JSONL record for every incoming chunk. Large records therefore incurred quadratic copying and could exhaust CPU and memory.

This change adds one incremental line-framing owner that retains unfinished records as fragments and joins them only at a newline or EOF. Both protocol stdin and Codex stderr now use it while preserving existing newline, CRLF, empty-line, parse-error, stream-failure, and EOF behavior.

Regression coverage includes a 20 MiB record split into irregular chunks, multiple records, CRLF, empty records, newline-boundary splits, final unterminated records, protocol EOF routing, and termination order. On the exact 20 MiB workload, the candidate completed in 7–13 ms versus 8,105 ms for the prior accumulator; the test rejects implementations exceeding 2,000 ms. Focused tests, package/server typechecks, targeted lint/formatting, and done-check passed.

Fork coordination: clintebbesen#2

Verification environment: Codex harness, gpt-5.6-sol.

---

Note

Medium Risk
Changes the Codex app-server JSONL read path and stderr framing; behavior is intended to match prior semantics but mistakes could drop or mis-parse wire messages.

Overview
Fixes pathological CPU/memory use when Codex sends large JSONL records split across many stdin chunks. The old reader kept re-concatenating and re-scanning the whole unfinished line on every chunk (quadratic work); very large lines could stall or exhaust resources.

This PR adds makeLineFramer in effect-codex-app-server, exported as ./lineFramer. It buffers incomplete lines as fragments, emits only on \n, strips trailing \r, and exposes finish() for a final unterminated line at EOF. Protocol stdin and Codex session stderr both use it instead of local remainder + split("\n") logic.

Regression tests cover LF/CRLF, empty lines, arbitrary chunk boundaries, a 20 MiB fragmented record (must complete under 2s), CRLF messages split across chunks, and routing the last line without a trailing newline before input stream end (notification before termination).

^{Reviewed by Cursor Bugbot for commit 3de1fe9. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Replace ad-hoc line buffering with makeLineFramer in codex app-server protocol and runtime

• Introduces makeLineFramer, a reusable framer that incrementally buffers string chunks and emits complete lines split on \n with trailing \r trimmed.
• Replaces Ref-based remainder accumulators in both protocol.ts and CodexSessionRuntime.ts with lineFramer.push() / lineFramer.finish() calls.
• Exports the new module as the effect-codex-app-server/lineFramer subpath in package.json.
• Adds tests covering LF, CRLF, empty lines, unterminated final lines, and large fragmented lines.

^{Macroscope summarized 3de1fe9.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6303765a-c880-4d9f-8e27-0ae6c3eb4bc9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Approved 3de1fe9

Refactoring that extracts existing line-framing logic into a reusable makeLineFramer() utility with better performance for large records. Behavioral semantics are preserved with comprehensive test coverage added.

^{You can customize Macroscope's approvability policy. Learn more.}

[t3dotgg]

Note

🤖 GPT-6 Astra (preview) responding on behalf of Theo

This was closed as part of an automated cleanup pass. If you believe it was closed in error, reply here and we will get it reopened.

Closing as superseded by #8605, which is on main. The protocol reader now stores partial records as fragments and joins them at a newline or EOF, removing the repeated copying behind the reported failure. The separate stderr optimization and extra framing cases remain available in this branch.