Skip to content

fix(storage): wrap remaining saveAccounts call sites with retry helper - #443

Merged
ndycode merged 3 commits into
mainfrom
fix/saveAccounts-retry-windows-resilience
Apr 29, 2026
Merged

fix(storage): wrap remaining saveAccounts call sites with retry helper#443
ndycode merged 3 commits into
mainfrom
fix/saveAccounts-retry-windows-resilience

Conversation

@ndycode

@ndycode ndycode commented Apr 28, 2026

Copy link
Copy Markdown
Owner

Summary

Three production paths still wrote raw await saveAccounts(storage) without the EBUSY/EPERM retry helper that every other persistence boundary in this codebase uses. Wraps them in saveAccountsWithRetry so a transient Windows file-lock contention no longer silently drops a sync (or, in switch flows, fails the whole operation).

Why this matters

Per lib/AGENTS.md ("focus on auth rotation, windows filesystem io, and concurrency. verify every change … new queues handle ebusy/429 scenarios"), every persistence boundary should be EBUSY-tolerant. The forecast / report / best / rotation-reset / new rotation-reset-rate-limits paths all already use saveAccountsWithRetry from lib/codex-manager/forecast-report-shared.ts. These three were the holdouts I found while auditing the codebase after #442:

Site Function Failure mode
lib/accounts.ts:289 AccountManager.loadFromDisk source-of-truth sync persist A momentary file lock during startup silently drops the Codex CLI ↔ multi-auth sync. The catch already debug-logs but doesn't retry, so the in-memory state diverges from disk without user visibility.
lib/codex-manager.ts:2362 runHealthCheck post-mutation save A transient lock during codex auth check rejects the whole command instead of riding through.
lib/codex-manager.ts:3211 persistAndSyncSelectedAccount pre-Codex-CLI-sync save A transient lock during codex auth switch aborts the switch and may leave the active account out of sync.

Changes

  • lib/accounts.ts — import saveAccountsWithRetry, replace one raw call.
  • lib/codex-manager.ts — import saveAccountsWithRetry, replace two raw calls.
  • test/accounts-edge.test.ts — new regression: loadFromDisk retries source-of-truth persist on transient EBUSY.
  • test/codex-manager-cli.test.ts — existing does not mutate loaded quota cache when live check account save fails updated to use mockRejectedValue (unbounded) so the retry exhausts and the test still asserts its intended rejection path.

Retry policy (unchanged)

Same as the existing saveAccountsWithRetry helper:

  • Up to 3 retries on EBUSY / EPERM with backoff (10 * 2 ** attempt ms).
  • Any non-retryable error (no code, or a code outside the EBUSY/EPERM set) re-throws on the first attempt.
  • The existing test loadFromDisk tolerates sync persistence failures keeps its 1-call assertion: that error has no code, so it short-circuits as before.

Test plan

  • npm run typecheck
  • npx eslint
  • Full suite: 3745 / 3745 pass
  • Targeted: new EBUSY-retry regression in accounts-edge.test.ts confirms two attempts on a single transient failure.

Notes

This is the first follow-up from a broader reliability audit of the codebase that surfaced four PR-sized improvements. Subsequent PRs will cover the path-singleton try/finally hardening, a concurrency regression test for path-singleton races, and cleanup of the unused lastAccountEmail field. Submitting them as separate PRs so each can be reviewed in isolation.

note: greptile review for oc-chatgpt-multi-auth. cite files like lib/foo.ts:123. confirm regression tests + windows concurrency/token redaction coverage.

Greptile Summary

wraps the three remaining raw saveAccounts call sites in saveAccountsWithRetry so transient windows file-lock errors (EBUSY/EPERM) no longer silently drop a startup sync or abort a switch/health-check command. all three paths now match the retry policy already in place everywhere else in the codebase.

Confidence Score: 5/5

safe to merge — only P2 style finding, logic and token safety are correct

all three call sites correctly pass the locally-imported saveAccounts as a callback, matching existing usage; no circular dependency introduced by the accounts.ts → forecast-report-shared import; retry semantics and error propagation are correct; tests cover transient recovery, exhaustion, and the no-codex-cli-sync guarantee; only finding is a cosmetic import ordering issue

no files require special attention

Important Files Changed

Filename Overview
lib/accounts.ts adds import of saveAccountsWithRetry and wraps the loadFromDisk source-of-truth persist call; error catch block remains intact so transient windows EBUSY no longer silently drops the sync
lib/codex-manager.ts imports saveAccountsWithRetry and wraps the two raw saveAccounts calls in runHealthCheck and persistAndSyncSelectedAccount; import is placed mid-commands/ block (P2 style only)
test/accounts-edge.test.ts adds two new regression tests covering transient EBUSY recovery (2 calls) and persistent EPERM exhaustion (4 calls); both rely on real timers with small sleep totals (~70ms max), no fake-timer conflict
test/codex-manager-cli.test.ts updates existing EBUSY test to unbounded mockRejectedValue so retries exhaust before asserting rejection; adds two new regression tests for persistAndSyncSelectedAccount (transient recovery and retry exhaustion with no codex-cli sync)

Sequence Diagram

sequenceDiagram
    participant Caller
    participant saveAccountsWithRetry
    participant saveAccounts (fs)

    Caller->>saveAccountsWithRetry: saveAccountsWithRetry(storage, saveAccounts)
    loop attempt 0..3
        saveAccountsWithRetry->>saveAccounts (fs): saveAccounts(storage)
        alt success
            saveAccounts (fs)-->>saveAccountsWithRetry: resolved
            saveAccountsWithRetry-->>Caller: return
        else EBUSY / EPERM and attempt < 3
            saveAccounts (fs)-->>saveAccountsWithRetry: throw {code: EBUSY|EPERM}
            saveAccountsWithRetry->>saveAccountsWithRetry: sleep(10 * 2^attempt ms)
        else non-retryable OR attempt >= 3
            saveAccounts (fs)-->>saveAccountsWithRetry: throw error
            saveAccountsWithRetry-->>Caller: rethrow
        end
    end
Loading

Fix All in Codex

Prompt To Fix All With AI
This is a comment left during a code review.
Path: lib/codex-manager.ts
Line: 41-42

Comment:
**import inserted mid-`commands/` block**

`forecast-report-shared.js` sorts lexically after `commands/` (`f` > `c`), so it lands in the middle of the alphabetical `commands/` import group. consider moving it after the last `commands/` import to keep the block sorted and readable.

```suggestion
import { runConfigExplainCommand } from "./codex-manager/commands/config-explain.js";
import { runDebugBundleCommand } from "./codex-manager/commands/debug-bundle.js";
import { saveAccountsWithRetry } from "./codex-manager/forecast-report-shared.js";
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (3): Last reviewed commit: "test(codex-manager): cover persistAndSyn..." | Re-trigger Greptile

Context used:

  • Context used - lib/AGENTS.md (source)

…sWithRetry

Three production call sites still wrote raw `saveAccounts(storage)`
without the retry helper that absorbs transient Windows EBUSY/EPERM
contention:

- lib/accounts.ts:289 — `AccountManager.loadFromDisk` source-of-truth
  sync persist. The catch already debug-logs the failure, but a single
  attempt means a momentary file lock from another process drops the
  Codex CLI ↔ multi-auth sync silently.
- lib/codex-manager.ts:2362 — `runHealthCheck` post-mutation save.
- lib/codex-manager.ts:3211 — `persistAndSyncSelectedAccount`
  pre-Codex-CLI-sync save during account switch.

All three now use the same `saveAccountsWithRetry` helper that the
forecast / report / best / rotation-reset commands and (now) the
rotation reset-rate-limits subcommand use. Retry policy is unchanged:
up to 3 retries on EBUSY/EPERM with backoff, non-retryable errors
re-thrown immediately on the first attempt.

Tests:
- New regression: `loadFromDisk retries source-of-truth persist on
  transient EBUSY` — first call rejects with EBUSY, second succeeds,
  asserts both attempts.
- Existing `does not mutate loaded quota cache when live check account
  save fails` updated to mockRejectedValue (unbounded) so the retry
  exhausts and bubbles the error, asserting the rejection path that
  the test was always intending to verify.
- Existing `loadFromDisk tolerates sync persistence failures` keeps
  its single-call assertion: the helper only retries errors with an
  EBUSY/EPERM `code`, so the bare-`Error` rejection in that test
  short-circuits as before.

Full suite: 3745/3745 pass.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@coderabbitai

coderabbitai Bot commented Apr 28, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

walkthrough

routes account persistence through a retry-capable wrapper (saveAccountsWithRetry) in loadFromDisk, runHealthCheck, and persistAndSyncSelectedAccount, and adds tests that exercise transient (EBUSY) retry and retry exhaustion scenarios. (≈36 words)

changes

Cohort / File(s) Summary
persistence retry wrapper integration
lib/accounts.ts, lib/codex-manager.ts
replaced direct saveAccounts(...) calls with saveAccountsWithRetry(...) at persistence points (lib/accounts.ts:..., lib/codex-manager.ts:...). verify code paths that previously called saveAccounts now route through retry wrapper.
retry behavior test coverage
test/accounts-edge.test.ts, test/codex-manager-cli.test.ts
adds edge tests for AccountManager.loadFromDisk() retry behavior and tightens expectations for non-retryable failures (test/accounts-edge.test.ts:...). updates CLI tests to assert retry attempts and adds two regression tests for auth best account persistence with transient and persistent EBUSY (test/codex-manager-cli.test.ts:...).

sequence diagram(s)

mermaid
sequenceDiagram
participant cli as Client (CLI)
participant cm as CodexManager
participant am as AccountManager
participant retry as saveAccountsWithRetry
participant storage as Storage
cli->>cm: trigger action (auth check / switch)
cm->>am: persistAndSyncSelectedAccount / runHealthCheck
am->>retry: saveAccountsWithRetry(storage, saveAccounts)
retry->>storage: attempt saveAccounts()
alt transient EBUSY
storage-->>retry: reject EBUSY
retry->>storage: retry saveAccounts()
storage-->>retry: success
else persistent failure
storage-->>retry: repeated EBUSY / failure
retry-->>am: reject error
am-->>cm: propagate failure
end
retry-->>am: success
am-->>cm: ack persisted

estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

suggested labels

bug


review notes

  • concurrency risk: saveAccountsWithRetry is now called from lib/accounts.ts:... and lib/codex-manager.ts:.... confirm there are no race conditions if multiple save/sync flows run concurrently. add concurrent-write tests if concurrent operations are possible.
  • windows edge cases: tests and handling target EBUSY codes (test/accounts-edge.test.ts:..., test/codex-manager-cli.test.ts:...) but windows file-lock errors may surface with different codes (e.g., ERROR_SHARING_VIOLATION). verify saveAccountsWithRetry normalizes platform-specific errors or add windows-specific handling/tests.
  • missing regression tests: non-retryable failure behavior in runHealthCheck and persistAndSyncSelectedAccount should be explicitly covered. add a regression test that a permanent failure (non-EBUSY) still logs/aborts as expected (lib/codex-manager.ts:..., lib/accounts.ts:...).
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed Title follows conventional commits format with correct type (fix), scoped to storage, and summary under 72 chars in lowercase imperative.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed PR description is comprehensive, well-structured, and covers summary, changes, rationale, test plan, and validation. All required template sections are present.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/saveAccounts-retry-windows-resilience
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch fix/saveAccounts-retry-windows-resilience

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@lib/accounts.ts`:
- Line 2: The accounts module currently imports saveAccountsWithRetry from the
codex-manager namespace; extract saveAccountsWithRetry into a neutral shared
storage module (e.g., create a new storage/shared or utils/storage module) and
export it from there, then update lib/accounts.ts to import
saveAccountsWithRetry from the new shared module instead of codex-manager; also
update any codex-manager files that used the old location to import the helper
from the new shared module and ensure the moved function's tests/exports are
updated accordingly so account core logic no longer depends on the codex-manager
namespace.

In `@lib/codex-manager.ts`:
- Around line 2361-2363: Add a regression test that ensures
persistAndSyncSelectedAccount uses saveAccountsWithRetry's retry/exhaustion
behavior: mock the saveAccounts function (saveAccountsMock) to always reject
with EBUSY (and a separate case for EPERM) when persistAndSyncSelectedAccount is
invoked, call persistAndSyncSelectedAccount (via the same setup used in the
switch/best tests or in accounts-edge.test.ts), and assert that the promise
rejects with the retry-exhaustion error (i.e., the final propagated error)
rather than silently succeeding; reference the persistAndSyncSelectedAccount
function and saveAccountsWithRetry behavior to locate where to hook the mock and
assert propagation.

In `@test/accounts-edge.test.ts`:
- Around line 129-149: Add a deterministic Vitest regression that mirrors the
existing ebusy test but forces saveAccounts to always fail with an Error having
code "EPERM" so we exercise the windows-permission branch and retry exhaustion:
in the new test mockSaveAccounts.mockRejectedValue(eperm) (or rejectedValueOnce
three times plus one initial rejection) so AccountManager.loadFromDisk()
triggers the retry helper and ultimately resolves via the existing catch path in
lib/accounts.ts:291-295; assert mockSaveAccounts was called 4 times (initial + 3
retries) and that manager.getAccountCount() still returns the expected value,
reusing the same setup patterns (mockLoadAccounts,
mockSyncAccountStorageFromCodexCli, mockLoadCodexCliState) and naming consistent
with the existing ebusy test so the test remains deterministic and follows
Vitest conventions.

In `@test/codex-manager-cli.test.ts`:
- Around line 3218-3221: The test forces persistent EBUSY but doesn't prove
saveAccountsWithRetry actually retried; after the call that triggers
saveAccountsWithRetry, add an explicit assertion on saveAccountsMock call count
(e.g., expect(saveAccountsMock).toHaveBeenCalledTimes(<expectedRetries>) or at
minimum expect(saveAccountsMock.mock.calls.length).toBeGreaterThan(1)) so the
test fails if code falls back to a single-attempt save; reference the mocked
function saveAccountsMock and the retrying logic in saveAccountsWithRetry when
choosing the exact expected retry count.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0d112df5-c75d-4bf2-972c-05d403bc71d6

📥 Commits

Reviewing files that changed from the base of the PR and between 2e2211a and f1a9126.

📒 Files selected for processing (4)
  • lib/accounts.ts
  • lib/codex-manager.ts
  • test/accounts-edge.test.ts
  • test/codex-manager-cli.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Greptile Review
🧰 Additional context used
📓 Path-based instructions (2)
test/**

⚙️ CodeRabbit configuration file

tests must stay deterministic and use vitest. demand regression cases that reproduce concurrency bugs, token refresh races, and windows filesystem behavior. reject changes that mock real secrets or skip assertions.

Files:

  • test/accounts-edge.test.ts
  • test/codex-manager-cli.test.ts
lib/**

⚙️ CodeRabbit configuration file

focus on auth rotation, windows filesystem IO, and concurrency. verify every change cites affected tests (vitest) and that new queues handle EBUSY/429 scenarios. check for logging that leaks tokens or emails.

Files:

  • lib/accounts.ts
  • lib/codex-manager.ts
🔇 Additional comments (3)
lib/accounts.ts (1)

289-291: good windows file-lock hardening on source-of-truth persistence.

switching lib/accounts.ts:290 to saveAccountsWithRetry(...) correctly mitigates transient ebusy/eperm save failures while preserving existing fallback behavior in lib/accounts.ts:291-295.

as per coding guidelines lib/**: focus on auth rotation, windows filesystem IO, and concurrency. verify every change cites affected tests (vitest) and that new queues handle EBUSY/429 scenarios.

lib/codex-manager.ts (1)

2361-2363: retry wrapper is correctly applied at both remaining direct-save boundaries.

lib/codex-manager.ts:2362 and lib/codex-manager.ts:3211 now route through saveAccountsWithRetry(...), which is the right mitigation for transient windows lock contention in these mutation flows.

as per coding guidelines lib/**: focus on auth rotation, windows filesystem IO, and concurrency. verify every change cites affected tests (vitest) and that new queues handle EBUSY/429 scenarios.

Also applies to: 3209-3212

test/accounts-edge.test.ts (1)

125-127: the non-retryable branch assertion is solid.

test/accounts-edge.test.ts:125-127 clearly documents and asserts the single-attempt behavior when the error is not retryable.

Comment thread lib/accounts.ts
@@ -1,4 +1,5 @@
import type { Auth } from "@codex-ai/sdk";
import { saveAccountsWithRetry } from "./codex-manager/forecast-report-shared.js";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

decouple the retry helper from the codex-manager namespace.

lib/accounts.ts:2 imports a storage-persistence utility from lib/codex-manager/forecast-report-shared.ts. move saveAccountsWithRetry to a neutral storage/shared module so account core logic does not depend on a codex-manager feature namespace.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/accounts.ts` at line 2, The accounts module currently imports
saveAccountsWithRetry from the codex-manager namespace; extract
saveAccountsWithRetry into a neutral shared storage module (e.g., create a new
storage/shared or utils/storage module) and export it from there, then update
lib/accounts.ts to import saveAccountsWithRetry from the new shared module
instead of codex-manager; also update any codex-manager files that used the old
location to import the helper from the new shared module and ensure the moved
function's tests/exports are updated accordingly so account core logic no longer
depends on the codex-manager namespace.

Comment thread lib/codex-manager.ts
Comment thread test/accounts-edge.test.ts
Comment thread test/codex-manager-cli.test.ts
ndycode added 2 commits April 29, 2026 16:26
Adds the two regression cases CodeRabbit flagged on PR #443:

1. test/accounts-edge.test.ts gains a persistent-EPERM case that asserts
   loadFromDisk's source-of-truth save retries the full budget (initial + 3
   retries = 4 attempts) before catching and continuing. Pairs with the
   existing transient-EBUSY test to cover both retryable Windows codes.

2. test/codex-manager-cli.test.ts now asserts saveAccountsMock was called
   more than once during the 'auth check' EBUSY rejection path. Without
   this guard, a regression that swaps saveAccountsWithRetry for a raw
   saveAccounts call would slip through unnoticed.
Adds two regression tests for the saveAccountsWithRetry call inside
persistAndSyncSelectedAccount (lib/codex-manager.ts:3211), exercised here
through 'auth best':

1. transient EBUSY recovers — first save attempt rejects, second
   succeeds, the switch completes, and codex-cli is told about the new
   active selection. Without the retry helper this collapses to a
   single attempt and the switch fails.

2. persistent EBUSY exhausts the budget (initial + 3 retries = 4
   attempts) and propagates the error rather than silently succeeding,
   and setCodexCliActiveSelection is never called for a switch that
   did not persist.

Closes the CodeRabbit major comment on PR #443 about needing explicit
coverage for persistAndSyncSelectedAccount's retry/exhaustion path.
@ndycode
ndycode merged commit 25e1853 into main Apr 29, 2026
2 checks passed
@ndycode
ndycode deleted the fix/saveAccounts-retry-windows-resilience branch April 29, 2026 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant