Fall back to a configured sibling model when a provider is down (do not merge)

[goleary]

Do not merge. Opened so we can look at the actual size of doing model fallback inside opencode instead of around it.

What this does

When a step's LLM call fails with a status the coordinator lists (or never reaches a provider at all), and the same-provider retries are spent, the step swaps to the mapped sibling model and continues. The failing provider is marked degraded for the configured cooldown, so later steps in the same sandbox start on the healthy route instead of paying the retries again. The assistant message is updated to name the model that actually answered.

Config comes from REPLO_OPENCODE_FALLBACK_CONFIG, which the coordinator already injects into every sandbox (apps/cloudflare-agent-coordinator/src/lib/opencode-fallback-config.ts in andytown). Nothing in this fork read it before. No env var means no behaviour change.

Also fixes a caching bug seen in prod: applyCaching marked an empty trailing text part with cache_control, which Anthropic rejects (cache_control cannot be set for empty text blocks). An empty tail now falls through to the message-level marker.

Shape of the change

File	Lines	What
src/session/fallback.ts (new)	+120	Parse config once, qualifies(error), next(), degraded-provider map with cooldown, healthy() chain walk with cycle guard
src/session/retry.ts	+28 / -11	policy() gains a provider getter, an optional same-provider attempt cap, and a fallback hook that runs once retries are exhausted or the error is non-retryable
src/session/processor.ts	+41 / -5	Resolves the fallback model via Provider.Service, streams with ctx.model instead of the input model, rewrites providerID/modelID on the assistant message, logs [model-fallback]
src/session/prompt.ts	+7 / -1	Each step resolves SessionFallback.healthy() before getModel
src/provider/transform.ts	+4	Empty-text guard
tests	+212 / -3	test/session/fallback.test.ts (config, qualification, cooldown, cycle), a retry-policy handoff test, and an end-to-end processor test: two 503s from the test LLM server, third call answered by the fallback provider, message stamped fallback/fallback-model, test marked degraded

Decisions worth a look

• Fallback lives in the retry schedule, not a plugin. The schedule already owns "what happens after a failed attempt", so the hook is one extra branch there. The old model-fallback plugin approach needed the plugin to re-run the whole step.
• Per-step swap, per-sandbox cooldown. The swap decision is local to a step (bounded by maxFallbackAttempts); the cooldown is module state so the next step skips the dead provider. It is process memory, so a sandbox restart forgets it, which is fine.
• Which errors qualify: statuses in fallbackOnErrors, plus 402 (OpenRouter out of credits) and errors with no status (fetch failed, reset). Context overflow and aborts never swap.
• Same-provider retries keep today's behaviour when maxUpstreamRetryAttempts is unset, so upstream opencode semantics are unchanged without our env var.

Verification

• bun run typecheck clean.
• bun test test/session/fallback.test.ts test/session/retry.test.ts test/session/processor-effect.test.ts: 58 pass.
• bun test test/provider: 426 pass, 1 fail (chunkTimeout raises a response stream error when SSE body stalls), which fails identically on dev at 1ae9b5c with this branch's changes reverted, so it is pre-existing.

Follow-ups in andytown (not here)

• Add the OpenRouter ↔ direct Anthropic pairs to opencode-fallback-config.ts in both directions.
• Flip the coordinator default so agent Anthropic traffic routes through OpenRouter.

🤖 Generated with Claude Code

---

Summary by cubic

Fixes provider outages by falling back to a configured sibling model when retries are exhausted, and stops marking empty text parts for prompt caching.

• Reads REPLO_OPENCODE_FALLBACK_CONFIG; without it behavior is unchanged.
• After listed statuses or transport errors exhaust same-provider retries, the step swaps to the mapped model and continues.
• The failing provider is marked degraded for the cooldown, so later steps start on the healthy route.
• The assistant message records the model that actually answered.
• Fixes cache_control being set on empty trailing text, which Anthropic rejects.

^{Written for commit 242d693. Summary will update on new commits.}

[github-actions]

Hey! Your PR title Fall back to a configured sibling model when a provider is down (do not merge) doesn't follow conventional commit format.

Please update it to start with one of:

• feat: or feat(scope): new feature
• fix: or fix(scope): bug fix
• docs: or docs(scope): documentation changes
• chore: or chore(scope): maintenance tasks
• refactor: or refactor(scope): code refactoring
• test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

[github-actions]

The following comment was made by an LLM, it may be inaccurate: