Skip to content

runtime: process exits on any failed agent fetch (UND_ERR_SOCKET / UND_ERR_BODY_TIMEOUT) #33

Description

@nanaknihal

Summary

Any failure of the runtime's HTTP request to the Python agent — a dropped socket, a slow turn, an agent restart — raises an unhandled rejection inside undici and kills the whole runtime process (server.ts). The Slack/Teams channel session dies with it, and every message until an operator restarts the process is silently lost.

Observed three times in one evening of local testing, two distinct signatures:

1. Agent restarted mid-request (also reproduced by killing the agent during a turn):

Agent execution failed: TypeError: terminated
    at Fetch.onAborted (node:internal/deps/undici/undici:11124:53)
  [cause]: SocketError: other side closed
    code: 'UND_ERR_SOCKET'

2. Agent turn exceeded undici's default 300s body timeout (a long tool-heavy turn with no stream activity):

node:internal/process/promises:394
    triggerUncaughtException(err, true /* fromPromise */);
TypeError: terminated
    code: 'UND_ERR_BODY_TIMEOUT'

Both escape as uncaught exceptions / unhandled rejections, so Node terminates the process.

Why it matters

The runtime is the long-lived member of the pair — it holds the Intelligence channel session. A transient agent-side hiccup (deploy, OOM, slow model call) shouldn't take down message delivery for the whole workspace. There's also a secondary effect: when the crash lands mid-turn, the agent's LangGraph state can be left with a dangling tool call, and the next turn on that thread fails with the Responses API's No tool output found for function call ….

Suggested fix

  • Wrap the agent fetch (and its stream consumption) so failures are caught, logged, and surfaced as a failed run — not an uncaught exception. A process.on('unhandledRejection') backstop that logs instead of exiting would also stop the bleeding.
  • Consider raising or configuring undici's bodyTimeout for the agent request (long agentic turns with sparse stream output can legitimately exceed 300s), or send periodic keep-alive events from the agent.

Happy to help test a fix — local repro is reliable: start a turn, then either kill the agent process or give it a tool that sleeps past 300s.

Environment: OpenTag main, Node v22.14.0, runtime via pnpm runtime / tsx server.ts, agent via uvicorn locally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions