Skip to content

fix(server): coalesce Codex child progress before ingestion - #6004

Open
cheruvian wants to merge 4 commits into
pingdotgg:mainfrom
cheruvian:release/coalesce-codex-progress-ingestion
Open

fix(server): coalesce Codex child progress before ingestion#6004
cheruvian wants to merge 4 commits into
pingdotgg:mainfrom
cheruvian:release/coalesce-codex-progress-ingestion

Conversation

@cheruvian

@cheruvian cheruvian commented Aug 10, 2026

Copy link
Copy Markdown

Human hold

Prepared for human review, not automatic merging. This preserves Ryan Idrogo's original proposal and commit history. The 250 ms delay, discarded intermediate snapshots, and discard-on-explicit-stop behavior need a maintainer decision.

Related #5681. This is a bounded Codex write-amplification repair, not a claim that the reported 17–37-minute stalls are reproduced or fully fixed.

Problem and change

Each Codex child item and cumulative usage notification currently becomes a durable progress event. Stable projected activity IDs do not bound those writes or the serial ingestion work.

Keep the newest item and usage snapshot independently for each session/child in fixed 250 ms windows. Flush a child's pending values before its mapped non-progress updates, including waiting and metadata changes. Flush every child before root completion, abort, runtime error, or session exit; close the coalescer after exit. Mixed mapped batches pass through intact. Explicit session stop still cancels and discards pending progress, as proposed originally.

The current-main adaptation also fixes a verified ordering regression in the original proposal: delayed progress could follow completion/error boundaries or overwrite a child's newer waiting state/name. No queue-fairness, other-provider, schema, or client changes are included.

Losses requiring approval

  • Item summaries A then B inside a window retain B only. A's transient history and timestamp disappear; the latest summary and separately mapped completion/result events remain.
  • Cumulative usage 100 then 200 retains 200 and all currently mapped breakdown fields, not an additive delta or the smaller last breakdown. The intermediate sample disappears. Native protocol breakdown fields are required; arbitrary partial or non-monotonic synthetic inputs are not claimed to preserve every historical field/maximum.
  • Explicit stop before a tick discards pending item/usage. Persisted progress and usage may remain stale, even though the session lifecycle is unchanged and old pending work cannot leak into a restarted session. Final usage preservation on explicit stop is not claimed.

Verification

Latest tested base: eced382b, September 5, 2026. Tests use an owned inert JSON-RPC peer, the actual Codex runtime/adapter, ingestion receipts and worker drains, and disposable SQLite—not a provider account.

Actual 32-item + 32-usage wire burst Current main Candidate
No intervening lifecycle update 64 durable progress events 2
Waiting/running boundary halfway through 64 4

Both preserve the final query, every mapped cumulative usage field, and progress-before-completion ordering. The latest-main baseline fails the reduction assertions; the candidate passes. The ingestion tests wait for completion enqueue receipts and worker drains, with no sleep/polling.

All 129 tests in these three files pass:

# From apps/server
vp test run src/provider/Layers/CodexProgressCoalescer.test.ts \
  src/provider/Layers/CodexAdapter.test.ts \
  src/orchestration/Layers/ProviderRuntimeIngestion.test.ts --maxWorkers=2

Coverage includes fixed windows, independent lanes/keys, two-child flush order, root and child boundaries, same-thread session restart, unexpected exit, and tick/flush/close races. Targeted lint/format and server typecheck pass; typecheck has existing nonblocking suggestions elsewhere.

Additional local checks passed actual mapped events through the server activity mapper and client fold: the repaired order retains the new name and waiting state; the original reordered payloads produce the old name/running state. Actual committed SQLite rows fold to idle with latest-query-31 and total/input/cached/output/reasoning counts 10031/9031/8031/1000/31. These are logic/runtime receipts, not native UI evidence.

No real-provider, Windows/macOS, or multi-client stall verification was performed. Claude reports, other high-volume event kinds, and global ingestion fairness remain outside this PR. Current-head CI and configured reviews are tracked separately; human hold applies regardless of green checks.

Adapted and verified by GPT 6 Astra via Codex in T3 Code.

Note

Coalesce Codex child progress before ingestion in CodexAdapter

  • Adds CodexProgressCoalescer (CodexProgressCoalescer.ts), which buffers item and token-usage progress per child thread in a 250 ms window, keeping only the newest value per lane and emitting item before usage.
  • Integrates the coalescer into CodexAdapter session lifecycle: child progress is offered to the coalescer instead of queued immediately, and all pending progress is flushed before child or root terminal events (turn completion, idle, closure, error, exit, interruption).
  • stopSessionInternal and session exit close the coalescer before tearing down the runtime and event fiber, discarding any pending values.
  • Risk: downstream consumers of CodexAdapter.streamEvents will see fewer progress events (at most two per lane per window per child) and different ordering — the latest buffered values now precede lifecycle events. Any consumer relying on per-notification delivery or specific progress counts will need adjustment.

Macroscope summarized a2c1f9b.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d1d811d6-4797-4b4d-ac70-ac90e796e62f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Aug 10, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One convention finding: the new coalescer's public interface is re-declared locally in CodexAdapter.ts instead of being imported from its owning module. Everything else in the changed scope (namespace subpath imports, scoped make construction, callback-based emit rather than service injection, test-only service instance passing) follows the conventions.

Posted via Macroscope — Effect Service Conventions

Comment thread apps/server/src/provider/Layers/CodexAdapter.ts Outdated
@macroscopeapp

macroscopeapp Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR adds an always-on 250 ms coalescing and lifecycle mechanism to the production Codex event pipeline, changing progress timing, snapshot retention, ordering, and shutdown behavior before durable ingestion. It also introduces a line-level static-analysis suppression, so the combined runtime and review-policy impact warrants human review.

You can add or adjust custom eligibility rules. Learn more.

Coalesce Codex child item and token-usage notifications independently per session and child before provider runtime ingestion. Flush pending progress before terminal lifecycle events and cancel pending work when the runtime closes.

Add focused adapter and ingestion coverage for bounded durable event volume, latest-value preservation, terminal ordering, isolation, and shutdown.
@cheruvian
cheruvian force-pushed the release/coalesce-codex-progress-ingestion branch from a0e69c6 to 8d27611 Compare August 10, 2026 14:34
Andrey170170 added a commit to Andrey170170/t3code that referenced this pull request Aug 31, 2026
Semantically ports pingdotgg#6004 onto current main and flushes buffered progress before identity refinements as well as terminal lifecycle events.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants