Skip to content

The pinned-503 calls every recovery deadline a "limit", so provider outages read as blown quotas #675

Description

@possibilities

The pinned-503 calls every recovery deadline a "limit", so provider outages read as blown quotas

Summary

buildPinnedUnavailableErrorBody words its recovery deadline as the recorded limit resets at <t> for every blocker, but since #671 the deadline is max(rate-limit reset, cooldown end, circuit-breaker next-attempt). When the blocker is a tripped circuit breaker or a server-error cooldown — i.e. the provider is failing — the operator is told about a "limit", and a backend incident reads as a blown subscription quota. The parenthetical also leaks the internal token (circuit-open), which names a local breaker but reads as a fact about the account.

Observed during a real incident

During the OpenAI backend incident of 2026-08-20 (status page: incident created 00:02:08Z, "elevated errors"), Codex sessions printed:

■ unexpected status 503 Service Unavailable: Pinned account 2 is currently
unavailable (circuit-open); the recorded limit resets at
2026-08-20T00:13:02.254Z; the pin was set by this session's launcher, so
relaunch to select a different account., url: http://127.0.0.1:56220/responses

The evidence that this wording misdescribed the situation:

  • The usage ledger shows clean traffic through 00:02 UTC and the first 503s at 00:03 — exact correlation with the incident.
  • Zero HTTP 429s in the two hours around the incident; every failure row was the locally-minted codex_pinned_account_unavailable.
  • rateLimitResetTimes was null on both accounts — there was no rate-limit record at all. Account 2 carried cooldownReason: "server-error"; the printed deadline was a 30-second breaker's next-attempt time, ~13 seconds out.
  • Every busy app helper reported lastError: "fetch failed" — transport errors against the Codex backend.

The operator's read was "the backend is down but the error says my subscription is exhausted." That read was correct, and the skip-reason precedence in getManagedAccountRuntimeSkipReason even guarantees it: rate-limited is tested before circuit-open, so a reported circuit-open positively proves the account was not rate limited at evaluation time.

Expected

The human sentence derives its parenthetical and deadline noun from the blocker class:

  • rate-limited → quota phrasing ("the rate limit resets at …").
  • circuit-open, cooling-down:server-error, cooling-down:network-error → transient-failure phrasing naming upstream errors as the cause and the timestamp as the next attempt.
  • Permanent blockers keep their already-suppressed deadline.

The JSON contract stays byte-identical apart from messagereason keeps the raw token for machines, the status stays 503, and normalizeExhaustionStatus keeps 429 for genuine rate-limit exhaustion.

This is a follow-up in the same seam as #671, which made this sentence honest about pin source and reset time; the breaker deadline it correctly added to the recovery bound is what the noun "limit" no longer describes.

I have a fix ready and will open a PR shortly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions