Skip to content

fix(provider): default a stream-idle timeout for openai-compatible providers - #46741

Closed
beatakouchnir wants to merge 1 commit into
anomalyco:devfrom
beatakouchnir:fix/openai-compatible-stream-idle-timeout
Closed

fix(provider): default a stream-idle timeout for openai-compatible providers#46741
beatakouchnir wants to merge 1 commit into
anomalyco:devfrom
beatakouchnir:fix/openai-compatible-stream-idle-timeout

Conversation

@beatakouchnir

@beatakouchnir beatakouchnir commented Sep 2, 2026

Copy link
Copy Markdown

Issue for this PR

Fixes #46581

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Custom @ai-sdk/openai-compatible providers never got a chunkTimeout or headerTimeout default. wrapSSE() is a no-op when chunkTimeout is unset (if (typeof ms !== "number" || ms <= 0) return res), so these providers stream with no idle guard at all — a backend that stalls mid-stream hangs the session indefinitely. Named providers (openai, anthropic) already default a timeout; this gives openai-compatible providers the same protection via an overridable chunkTimeout default (300s, matching the header-timeout default).

I moved the existing inline includeUsage defaulting into a small exported applyOpenAICompatibleDefaults() and added the chunkTimeout default there. includeUsage behavior is unchanged.

This covers the stream-stall mode (no SSE bytes for the timeout window). The other mode in #46581 — the stream closing on [DONE] while the AI SDK's fullStream still awaits a usage chunk — is not addressed here, because a byte-idle timeout can't fire once the final byte has arrived; that one needs a change at the AI-SDK adapter and I'd rather keep it separate.

I hit this running unattended opencode run sessions against a local llama.cpp server; a couple stalled and hung until an external timeout. Forensics are in the comment on #46581.

How did you verify your code works?

  • New test/provider/openai-compatible-defaults.test.ts: default applied for openai-compatible, not applied to other providers, never overrides an explicit chunkTimeout, preserves includeUsage: false.
  • test/provider/header-timeout.test.ts already verifies that a set chunkTimeout turns a stalled SSE stream into a ResponseStreamError; this change makes that apply by default.
  • Full test/provider/ suite passes (593 tests). typecheck and prettier clean.

Screenshots / recordings

n/a — no UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

…oviders

Custom @ai-sdk/openai-compatible providers received no chunkTimeout or
headerTimeout default, so wrapSSE ran unwrapped and a backend that stalled
mid-stream would hang the session indefinitely (issue anomalyco#46581). Named providers
(openai, anthropic) already default a timeout; this extends the same protection
to openai-compatible providers via an overridable chunkTimeout default.

The openai-compatible option defaulting (previously an inline includeUsage
assignment) is extracted into a pure, exported applyOpenAICompatibleDefaults()
so the behavior is unit-testable in isolation. includeUsage handling is
unchanged; the chunkTimeout default is additive and overridable via
provider options.

The end-to-end "stall raises ResponseStreamError" behavior is already covered by
test/provider/header-timeout.test.ts; the new unit test covers that the default
is applied for openai-compatible, not applied to other providers, and never
overrides an explicit value.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@holny holny left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch — the ai-sdk layer already had the machinery, it just never got a default for custom openai-compatible providers. The no-op guard in wrapSSE makes an unset timeout a silent gap, so this is a real fix; the includeUsage extraction preserves behavior and the explicit-chunkTimeout override is covered by tests.

Context that's relevant here: the V2 native route path (packages/ai) has no chunk-idle guard at all — not just a missing default. That's tracked in #46692 (being worked separately), so no overlap with this PR; just noting the ai-sdk layer won't be the only surface needing this.

@beatakouchnir

Copy link
Copy Markdown
Author

Thanks for the review @holny. Looks like #46890 has since landed a global chunkTimeout default (also 5 min, disable-per-provider via chunkTimeout: false), which covers the openai-compatible case here and then some — so this PR is redundant now. Closing in favor of it. Glad the stall gap is addressed; the [DONE]-without-usage variant from #46581 is a separate mode that a byte-idle timeout won't catch, so that one's still open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenCode hangs indefinitely with @ai-sdk/openai-compatible although backend completes successfully

2 participants