Skip to content

fix(orchestration): exclude rolled-back work from bounded recovery - #8464

Open
saphid wants to merge 3 commits into
pingdotgg:t3code/codex-turn-mappingfrom
saphid:codex/fix-v2-recovery-scan
Open

fix(orchestration): exclude rolled-back work from bounded recovery#8464
saphid wants to merge 3 commits into
pingdotgg:t3code/codex-turn-mappingfrom
saphid:codex/fix-v2-recovery-scan

Conversation

@saphid

@saphid saphid commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

What Changed

Runtime recovery no longer materializes rolled-back history solely because abandoned command, tool, or subagent items retain a nonterminal status. Both SQL and memory selection exclude items owned by a rolled-back run while retaining orphaned items whose run is missing.

Current V2 already loads full projections only for the IDs returned by getRecoveryThreadIds("runtime"). This change preserves that API and adds a regression with 1,000 settled shells and one recovery candidate. It does not reintroduce the older duplicate recovery-query API.

Validation

On c0c3d6e61, based directly on V2 b9fa1399c:

  • CI=true vp test run apps/server/src/orchestration-v2/ProjectionStore.test.ts apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.test.ts apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.regression.test.ts apps/server/src/orchestration-v2/ProviderTurnControlService.test.ts: 30 tests pass across four files, exit 0. The rolled-back selection tests failed before the fix; orphan selection passes in both stores.
  • CI=true vp run --filter t3 typecheck: passes, exit 0, with existing suggestions.
  • Targeted formatting, lint, and git diff --check pass, exit 0.
  • Direct claude --model claude-opus-5 --effort high --print --tools '' --no-session-persistence --output-format json frozen review attempt exited 1 because OAuth expired before a model ran. No Claude review occurred.

The old branch was backed up before restacking. The final contribution is three files rather than the obsolete 926-file comparison.

Scope and limits

Targets Orchestrator V2 in #2829, not main. No UI changes, installed-runtime changes, or live database access. The production SQL recovery query retains its existing pending/running outbox selection. The in-memory projection store has no persisted outbox and this change makes no outbox-parity claim; the previous review referred to a helper removed during this restack.

Implemented and verified with GPT-6 in the Codex harness.

Current-base verification

Updated to exact V2 base 415ed0f73b97f1655b6282492f81d0b2bba3a9cc at head 97d295f475fed70f7c72374327d0d85a6d7efed7. The original leaf plus this base synthesizes cleanly as tree 2108d433dadef16b2cf38432e2c40f96805eec07; current V2 base still admits nonterminal items from rolled-back runs, so the recovery fix is not superseded.

  • 30/30 focused projection/recovery/control tests pass across four files.
  • Server typecheck and targeted lint pass.
  • Diff checks pass; the contribution remains the same three recovery files.

The production SQL query retains pending/running outbox recovery. The in-memory store still makes no persisted-outbox parity claim. No live database was accessed. Fresh independent Claude review remains unavailable because OAuth is signed out; no Claude review is claimed. Refreshed by GPT-6 in the Codex harness.

Note

Fix bounded recovery to exclude rolled-back runs in ProjectionStore

Runtime recovery no longer treats unfinished command, dynamic-tool, or subagent items as candidates when their projected run has a rolled-back status. Both the in-memory needsRecovery predicate and the SQL recovery-candidate query skip these items, while orphaned items with no matching run projection remain candidates. Tests cover rolled-back, orphaned, settled, and running threads across in-memory and SQL-backed stores.

  • Risk: needsRecovery and the SQL query in ProjectionStore.ts now skip unfinished items whose run is rolled back; verify no callers depend on recovering rolled-back work.

Macroscope summarized 97d295f.

@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 95edbe02-5ef5-4aa8-bf6b-fc452c4cc56d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Aug 28, 2026
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from 2d623ac to 37f40d9 Compare August 28, 2026 03:30
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 97c9bf7 to 1f46bcf Compare August 28, 2026 12:45
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from a5eeff5 to 112c206 Compare August 28, 2026 22:53
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 1f46bcf to 3177138 Compare August 28, 2026 23:03
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from a44ab73 to a110e68 Compare August 28, 2026 23:30
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 3177138 to 9be8e77 Compare August 28, 2026 23:47
Comment thread apps/server/src/orchestration-v2/ProjectionStore.ts Outdated
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from a110e68 to 81c6ac9 Compare August 29, 2026 00:13
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 9be8e77 to 8109404 Compare August 29, 2026 00:26
Comment thread apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.ts Outdated
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from 6733253 to 886b8db Compare August 29, 2026 02:40
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 8109404 to 658f2b8 Compare August 29, 2026 02:59
Comment thread apps/server/src/orchestration-v2/ProjectionStore.ts Outdated
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from 886b8db to a7452f6 Compare August 29, 2026 03:08
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 658f2b8 to ba89ee9 Compare August 29, 2026 03:21
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from a26d9ad to d6ed793 Compare August 29, 2026 06:34
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from ba89ee9 to 8efbd88 Compare August 29, 2026 21:48
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from 490318a to 373612e Compare August 30, 2026 00:13
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from 8efbd88 to a81d00a Compare August 30, 2026 00:56
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch 8 times, most recently from 877b9de to b82facd Compare September 5, 2026 04:52
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from b82facd to 2ac9bfe Compare September 5, 2026 04:56
@saphid
saphid force-pushed the codex/fix-v2-recovery-scan branch from a81d00a to c0c3d6e Compare September 5, 2026 11:05
@saphid saphid changed the title fix(orchestration): bound startup recovery reads fix(orchestration): exclude rolled-back work from bounded recovery Sep 5, 2026
@github-actions github-actions Bot added size:S 10-29 changed lines (additions + deletions). and removed size:L 100-499 changed lines (additions + deletions). labels Sep 5, 2026
@saphid
saphid marked this pull request as ready for review September 5, 2026 11:06
@macroscopeapp

macroscopeapp Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at c0c3d6e

Macroscope's review found this PR approvable — This is a localized recovery-selection bug fix that prevents abandoned work from being reprocessed after rollback while preserving recovery for orphaned work. Production changes are minimal and covered by focused in-memory and SQL-backed regression tests.

You can add or adjust custom eligibility rules. Learn more.

@cursor

cursor Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S 10-29 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant