Skip to content

emrg: retry LLM stream transport errors (ReadTimeout) before any delta is yielded - #1082

Merged
argszero merged 1 commit into
masterfrom
feature/llm-stream-transport-error-retry
Aug 31, 2026
Merged

emrg: retry LLM stream transport errors (ReadTimeout) before any delta is yielded#1082
argszero merged 1 commit into
masterfrom
feature/llm-stream-transport-error-retry

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

Fixes rant 2026-08-31T12:53:13: LLM streaming ReadTimeout (and other transient transport errors) were not retried, killing the whole tool round — 12 occurrences in emrgd.log, each aborting the task and wasting evolution cycles.

Root cause

In emrg/server/llm.py, chat_stream()'s retry loop only covered: retryable HTTP statuses (429/5xx), unparseable bodies, and streams ending without finish_reason. A transport error raised inside async for line in resp.aiter_lines() (e.g. httpx.ReadTimeout — no data block within the 120s read timeout; ConnectError; RemoteProtocolError) bubbled straight out of the iterator, skipped the retry loop, and reached daemon.py's except Exception → "LLM error" broadcast → tool loop terminated.

Fix

Wrap the streaming iteration in try/except httpx.TransportError and route it through the existing retry loop (exponential backoff 1s→2s→4s, MAX_RETRIES=3), same as the premature-stream-end path. Per the rant's suggestion #3, a retry is only attempted while no delta has been yielded yet — once the caller has seen (and broadcast) a delta, retrying would duplicate already-streamed content, so the error is re-raised instead.

Verification

  • New tests (tests/test_llm.py): retry-before-any-delta (ReadTimeout → recovers, 2 calls, warning logged); no-retry-after-delta (partial content already yielded → re-raises, 1 call); exhaust (4 attempts → raises).
  • test_llm.py: 34 passed (was 31). Full pytest: 1182 passed + 1 skipped (Agent.md count 1180 → 1183 synced).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (1/3)

Reviewed head 02743d8 against master:

  • Root cause confirmed: transport errors inside async for line in resp.aiter_lines() (httpx.ReadTimeout/ConnectError/RemoteProtocolError) had no except handler — they bubbled past the retry loop and killed the whole tool round (12 occurrences in emrgd.log).
  • Fix is sound: the streaming iteration is wrapped in try/except httpx.TransportError and routed through the existing retry loop (backoff 1s→2s→4s, MAX_RETRIES=3), with the rant's suggestion #3 honored — a retry only happens while yielded_delta is false, so already-broadcast content is never duplicated; after a yield the error is re-raised.
  • Tests: 3 new (retry-before-any-delta recovers; no-retry-after-delta re-raises; exhaust after 4 attempts). test_llm.py 34 passed; full pytest 1182 passed + 1 skipped; import + CLI green.
  • CI: test + test-windows both green (run 33362135118); mergeable_state clean.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (2/3)

Reviewed head 02743d8 against master (diff re-verified this cycle):

  • Change is a targeted wrap: the SSE streaming block (async for line in resp.aiter_lines()) now sits inside try/except httpx.TransportError, with the pre-existing HTTP-status retry and 'stream ended without finish_reason' retry logic preserved below the try.
  • Retry semantics are correct: yielded_delta is set immediately before each yield, so a transport error is retried (backoff 1s→2s→4s, MAX_RETRIES=3) only when nothing has been broadcast yet; after the first yield the error is re-raised — no duplicate content/broadcast.
  • Tests: 3 new cases (retry-before-any-delta recovers; no-retry-after-delta re-raises; exhaust after 4 attempts). Local test_llm.py all pass; CI test + test-windows green (run 33362135118); mergeable_state clean.
    No issues found.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (3/3)

Re-verified head 02743d8 this cycle: diff matches the intended small change (try/except httpx.TransportError around the SSE aiter_lines loop, retry only while no delta yielded, existing HTTP-status + premature-end retries preserved below); test_llm.py all pass locally; CI test + test-windows green (run 33362135118); mergeable_state clean. 3 consecutive ✅ from distinct cycles, no ❌ — good to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant