fix(provider): default a stream-idle timeout for openai-compatible providers - #46741
fix(provider): default a stream-idle timeout for openai-compatible providers#46741beatakouchnir wants to merge 1 commit into
Conversation
…oviders Custom @ai-sdk/openai-compatible providers received no chunkTimeout or headerTimeout default, so wrapSSE ran unwrapped and a backend that stalled mid-stream would hang the session indefinitely (issue anomalyco#46581). Named providers (openai, anthropic) already default a timeout; this extends the same protection to openai-compatible providers via an overridable chunkTimeout default. The openai-compatible option defaulting (previously an inline includeUsage assignment) is extracted into a pure, exported applyOpenAICompatibleDefaults() so the behavior is unit-testable in isolation. includeUsage handling is unchanged; the chunkTimeout default is additive and overridable via provider options. The end-to-end "stall raises ResponseStreamError" behavior is already covered by test/provider/header-timeout.test.ts; the new unit test covers that the default is applied for openai-compatible, not applied to other providers, and never overrides an explicit value. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
holny
left a comment
There was a problem hiding this comment.
Nice catch — the ai-sdk layer already had the machinery, it just never got a default for custom openai-compatible providers. The no-op guard in wrapSSE makes an unset timeout a silent gap, so this is a real fix; the includeUsage extraction preserves behavior and the explicit-chunkTimeout override is covered by tests.
Context that's relevant here: the V2 native route path (packages/ai) has no chunk-idle guard at all — not just a missing default. That's tracked in #46692 (being worked separately), so no overlap with this PR; just noting the ai-sdk layer won't be the only surface needing this.
|
Thanks for the review @holny. Looks like #46890 has since landed a global |
Issue for this PR
Fixes #46581
Type of change
What does this PR do?
Custom
@ai-sdk/openai-compatibleproviders never got achunkTimeoutorheaderTimeoutdefault.wrapSSE()is a no-op whenchunkTimeoutis unset (if (typeof ms !== "number" || ms <= 0) return res), so these providers stream with no idle guard at all — a backend that stalls mid-stream hangs the session indefinitely. Named providers (openai,anthropic) already default a timeout; this gives openai-compatible providers the same protection via an overridablechunkTimeoutdefault (300s, matching the header-timeout default).I moved the existing inline
includeUsagedefaulting into a small exportedapplyOpenAICompatibleDefaults()and added thechunkTimeoutdefault there.includeUsagebehavior is unchanged.This covers the stream-stall mode (no SSE bytes for the timeout window). The other mode in #46581 — the stream closing on
[DONE]while the AI SDK'sfullStreamstill awaits a usage chunk — is not addressed here, because a byte-idle timeout can't fire once the final byte has arrived; that one needs a change at the AI-SDK adapter and I'd rather keep it separate.I hit this running unattended
opencode runsessions against a local llama.cpp server; a couple stalled and hung until an external timeout. Forensics are in the comment on #46581.How did you verify your code works?
test/provider/openai-compatible-defaults.test.ts: default applied for openai-compatible, not applied to other providers, never overrides an explicitchunkTimeout, preservesincludeUsage: false.test/provider/header-timeout.test.tsalready verifies that a setchunkTimeoutturns a stalled SSE stream into aResponseStreamError; this change makes that apply by default.test/provider/suite passes (593 tests).typecheckand prettier clean.Screenshots / recordings
n/a — no UI change.
Checklist