emrg: retry LLM stream transport errors (ReadTimeout) before any delta is yielded - #1082
Merged
Merged
Conversation
…a is yielded (rant 2026-08-31T12:53:13)
argszero
commented
Aug 31, 2026
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (1/3)
Reviewed head 02743d8 against master:
- Root cause confirmed: transport errors inside
async for line in resp.aiter_lines()(httpx.ReadTimeout/ConnectError/RemoteProtocolError) had no except handler — they bubbled past the retry loop and killed the whole tool round (12 occurrences in emrgd.log). - Fix is sound: the streaming iteration is wrapped in
try/except httpx.TransportErrorand routed through the existing retry loop (backoff 1s→2s→4s, MAX_RETRIES=3), with the rant's suggestion #3 honored — a retry only happens whileyielded_deltais false, so already-broadcast content is never duplicated; after a yield the error is re-raised. - Tests: 3 new (retry-before-any-delta recovers; no-retry-after-delta re-raises; exhaust after 4 attempts). test_llm.py 34 passed; full pytest 1182 passed + 1 skipped; import + CLI green.
- CI: test + test-windows both green (run 33362135118); mergeable_state clean.
argszero
commented
Aug 31, 2026
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (2/3)
Reviewed head 02743d8 against master (diff re-verified this cycle):
- Change is a targeted wrap: the SSE streaming block (
async for line in resp.aiter_lines()) now sits insidetry/except httpx.TransportError, with the pre-existing HTTP-status retry and 'stream ended without finish_reason' retry logic preserved below the try. - Retry semantics are correct:
yielded_deltais set immediately before each yield, so a transport error is retried (backoff 1s→2s→4s, MAX_RETRIES=3) only when nothing has been broadcast yet; after the first yield the error is re-raised — no duplicate content/broadcast. - Tests: 3 new cases (retry-before-any-delta recovers; no-retry-after-delta re-raises; exhaust after 4 attempts). Local test_llm.py all pass; CI test + test-windows green (run 33362135118); mergeable_state clean.
No issues found.
argszero
commented
Aug 31, 2026
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (3/3)
Re-verified head 02743d8 this cycle: diff matches the intended small change (try/except httpx.TransportError around the SSE aiter_lines loop, retry only while no delta yielded, existing HTTP-status + premature-end retries preserved below); test_llm.py all pass locally; CI test + test-windows green (run 33362135118); mergeable_state clean. 3 consecutive ✅ from distinct cycles, no ❌ — good to merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes rant 2026-08-31T12:53:13: LLM streaming
ReadTimeout(and other transient transport errors) were not retried, killing the whole tool round — 12 occurrences in emrgd.log, each aborting the task and wasting evolution cycles.Root cause
In
emrg/server/llm.py,chat_stream()'s retry loop only covered: retryable HTTP statuses (429/5xx), unparseable bodies, and streams ending withoutfinish_reason. A transport error raised insideasync for line in resp.aiter_lines()(e.g.httpx.ReadTimeout— no data block within the 120s read timeout;ConnectError;RemoteProtocolError) bubbled straight out of the iterator, skipped the retry loop, and reached daemon.py'sexcept Exception→ "LLM error" broadcast → tool loop terminated.Fix
Wrap the streaming iteration in
try/except httpx.TransportErrorand route it through the existing retry loop (exponential backoff 1s→2s→4s, MAX_RETRIES=3), same as the premature-stream-end path. Per the rant's suggestion #3, a retry is only attempted while no delta has been yielded yet — once the caller has seen (and broadcast) a delta, retrying would duplicate already-streamed content, so the error is re-raised instead.Verification
ReadTimeout→ recovers, 2 calls, warning logged); no-retry-after-delta (partial content already yielded → re-raises, 1 call); exhaust (4 attempts → raises).