fix(stream): prevent tool-call truncation, premature termination, upstream leaks + timeout hardening (#3)

* fix(stream): prevent tool-call truncation, premature end, and upstream leaks

Multiple bugs caused tool-call responses to be cut off mid-stream or
end abruptly with no finish_reason / message_stop. Each was reproduced
with a script before fixing and is covered by a regression test.

Critical fixes
- upstream: preserve NDJSON lines across backpressure. nodeReaderToStream
  returned out of read() when push() returned false, dropping the rest
  of the chunk's parsed lines forever. Tool-call deltas often arrive
  bundled in a single TCP packet, so this was the primary cause of
  "tool call truncated mid-stream".
- stream: synthesize a finish chunk (OpenAI) / message_delta + message_stop
  (Anthropic) when the upstream closes without a finish event (network
  drop, upstream crash mid-tool-call). Previously the client saw a clean
  end-of-stream with no terminal event.
- translate: set sawFinish=true on error events. Error events already
  emit a terminal chunk; without the flag, the server synthesized a
  second one on stream end (duplicate finish_reason chunks / duplicate
  message_stop, which the Anthropic SDK rejects).

Tool-call correctness
- openai: stable per-toolCallId streaming index. tool-call-delta events
  without an explicit index all defaulted to 0, so parallel tool calls
  were merged into one. Now allocate via Map<toolCallId, number> and
  reuse for subsequent deltas + final tool-call.
- anthropic: reuse the open tool_use block when a final tool-call arrives
  for the same toolCallId after deltas, instead of opening a second
  block (which produced duplicate tool_use blocks).

Resource hygiene
- upstream: cancel the upstream reader when the consumer stream is
  destroyed (client disconnect). Without this, CC kept generating
  tokens nobody read, burning the user's quota until upstream timeout.
- server: wire destroyStreamOnClientDisconnect on non-streaming paths
  too, so client disconnects abort the upstream fetch instead of
  silently draining into a discarded response.
- server: replace the ad-hoc stream.on('data'/'end'/'error') handlers
  with a pumpStream helper that applies client-side backpressure via
  pause-on-drain, wraps encoder.emit in try/catch (isolates encoder
  bugs instead of crashing the process), and consistently checks
  res.writableEnded.

Tests
- 4 new upstream tests (backpressure, reader-cancel-on-destroy).
- 5 new e2e tests (truncated OpenAI stream, truncated Anthropic stream,
  upstream error without duplicate finish, duplicate message_stop,
  stable tool-call indices).
- 3 new unit tests for the encoders (sawFinish on error, stable indices,
  tool-call reuses open block).
- All 144 tests pass; tsc --noEmit clean.

* fix(upstream): add streaming idle timeout + make timeouts configurable

The previous 5-minute timeout only protected the time-to-first-byte
phase: clearTimeout() was called the moment fetch() resolved (response
headers), leaving the entire streaming phase with no protection at all.
Two new failure modes resulted:

- A stalled upstream (TCP open, no chunks arriving mid-tool-call) would
  hang the consumer forever. The client's own timeout would eventually
  fire, but the proxy kept the upstream connection open and the request
  slot occupied until CC's own server-side timeout (if any) released it.
- The 5-minute connect timeout was hardcoded — slow reasoning models
  with long initial processing would be killed at the boundary with no
  way for the operator to bump it.

Fixes
- config: add CC_UPSTREAM_TIMEOUT_MS (default 600000 / 10 min, up from
  5 min) and CC_IDLE_TIMEOUT_MS (default 120000 / 2 min, 0 disables).
  Both parse-positive-int guarded.
- upstream: nodeReaderToStream now arms an idle timer before each
  reader.read() and disarms it on chunk arrival. If no data arrives
  within idleTimeoutMs, the reader is cancelled with an
  IdleTimeoutError that propagates through the stream's error path
  (which the existing pumpStream turns into a clean finish for the
  client). The timer is unref'd so it never keeps the event loop alive.
- upstream: caller's AbortSignal is now plumbed into nodeReaderToStream
  so a client disconnect during a stalled read immediately destroys the
  stream instead of waiting for the idle timer.
- server: both /v1/chat/completions and /v1/messages pass the new
  timeout options through.

Docs
- README + .env.example document the two new env vars.

Tests
- upstream: idle timeout fires after the configured interval and
  surfaces an IdleTimeoutError; idleTimeoutMs=0 disables it.
- config: defaults, env-var override, 0-disabled, and invalid-fallback
  cases.
- All 149 tests pass; tsc --noEmit clean.

* fix(translate): non-streaming tool-call dedup, content order, message_start on empty stream

Final-pass audit surfaced eight remaining correctness and resource
hygiene issues across the OpenAI/Anthropic translation layers and the
streaming server. None of them are the original 'tool call truncated'
class — they're separate contract violations and edge cases.

Critical
- openai: buildNonStreamingResponse now deduplicates tool-call-delta +
  final tool-call with the same id (the streaming encoder was fixed in
  the previous commit but the non-streaming builder was not). Without
  this, clients received two tool_calls entries with the same id, which
  some clients call twice and some dedupe wrong.
- anthropic: handleFinish now emits message_start when no content event
  arrived first (empty response, max_tokens=0, immediate refusal). The
  Anthropic SDK requires message_start as the first event of a stream
  and throws on its absence — the error and finishRecords paths already
  guarded this, but handleFinish did not.

High
- anthropic: non-streaming response now orders content blocks
  [thinking, text, tool_use] per the extended-thinking contract.
  Previously emitted [text, thinking, tool_use], which broke Claude
  Code's thinking-block continuation logic. Existing test asserted the
  wrong order; fixed.
- server: post-pumpStream [DONE] write now guards res.destroyed, not
  just res.writableEnded, preventing ERR_STREAM_DESTROYED on a socket
  torn down by mid-stream client disconnect.
- server: parseBody now calls req.destroy() on 413 so the client
  socket is freed immediately instead of lingering until the upload
  completes (was holding the connection open for the full oversized
  body even after rejecting).
- openai/server: pumpStream onError no longer mixes a non-chunk
  {error:...} envelope with valid chunks — both error paths (encoder
  error event + stream-level error) now emit uniform content+finish
  chat.completion.chunk records. Some clients were parsing the bare
  envelope as a tool call named 'error'.

Medium
- server: removed dead destroyStreamOnClientDisconnect calls in the
  streaming paths (mid-stream disconnect is already handled via the
  abort signal plumbed into nodeReaderToStream; the call after
  pumpStream returns is a no-op since the stream has ended/errored).

Low
- anthropic: top_p is now propagated into ccBody.params (was silently
  dropped, affecting Anthropic clients that tune sampling).

Tests
- translate: tool-call-delta + tool-call same-id merge in non-streaming
  builder.
- translate-anthropic: message_start ordering when finish arrives with
  no prior content; content block ordering thinking-before-text.
- e2e: stream-level error produces uniform chunks (no out-of-band
  envelope).
- All 152 tests pass; tsc --noEmit clean.

* chore: remove Docker support (unnecessary for a localhost proxy)

Author: thaolaptrinh

.env.example

@@ -5,5 +5,16 @@ CC_API_KEY=
 HOST=127.0.0.1
 PORT=8787
 
+# Upstream timeouts (milliseconds)
+# Max wall-clock time for CC to return response headers + first byte.
+# Bump this for slow reasoning models that take a while to start streaming.
+# CC_UPSTREAM_TIMEOUT_MS=600000   # default: 10 minutes
+
+# Max gap between consecutive data chunks during streaming. If CC opens the
+# connection but stops sending data mid-response (stalled tool call, dropped
+# TCP, etc.), the proxy aborts the stream and the client gets a clean
+# error/finish instead of hanging forever. Set to 0 to disable.
+# CC_IDLE_TIMEOUT_MS=120000       # default: 2 minutes
+
 # Anthropic model mapping (optional — prefer --setup-claude-code)
 # ANTHROPIC_DEFAULT_MODEL=deepseek/deepseek-v4-pro

.github/workflows/ci.yml

@@ -34,19 +34,3 @@ jobs:
       - name: Build
         run: pnpm build
 
-  docker:
-    runs-on: ubuntu-latest
-    needs: quality
-    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
-
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Build Docker image
-        uses: docker/build-push-action@v6
-        with:
-          push: false
-          tags: commandcode-api-proxy:latest

DEVELOPMENT.md

@@ -72,26 +72,6 @@ tests/
 └── e2e.test.ts           # End-to-end integration tests
 ```
 
-## Docker
-
-```bash
-# Build
-docker build -t commandcode-api-proxy .
-
-# Run with env var
-docker run --rm -p 8787:8787 \
-  -e CC_API_KEY=user_xxx \
-  commandcode-api-proxy
-
-# Or mount auth.json
-docker run --rm -p 8787:8787 \
-  -v ~/.config/commandcode-api-proxy:/home/node/.config/commandcode-api-proxy:ro \
-  commandcode-api-proxy
-
-# Using docker compose
-docker compose up -d
-```
-
 ## Tech stack
 
 - **Runtime:** Node.js (zero runtime dependencies)

Dockerfile

@@ -1,4 +1,4 @@
-.PHONY: build dev start test test-watch test-coverage fmt lint check clean docker-build docker-run docker-run-detached release help
+.PHONY: build dev start test test-watch test-coverage fmt lint check clean release help
 
 APP_NAME := commandcode-api-proxy
 VERSION := $(shell node -p "require('./package.json').version")
@@ -32,21 +32,6 @@ clean: ## Clean build artifacts
 	rm -rf dist/
 	rm -rf coverage/
 
-docker-build: ## Build Docker image
-	docker build -t $(APP_NAME):$(VERSION) .
-	docker tag $(APP_NAME):$(VERSION) $(APP_NAME):latest
-
-docker-run: ## Run Docker container
-	docker run --rm -p 8787:8787 \
-		-e CC_API_KEY=$(or $(CC_API_KEY),) \
-		$(APP_NAME):latest
-
-docker-run-detached: ## Run Docker container in background
-	docker run -d --name $(APP_NAME) \
-		-p 8787:8787 \
-		-e CC_API_KEY=$(or $(CC_API_KEY),) \
-		$(APP_NAME):latest
-
 release: build test ## Build and test for release
 	@echo "Ready for release: pnpm publish"
 

Makefile

@@ -73,15 +73,17 @@ commandcode-api-proxy auth logout
 
 Equivalent env vars (lower priority than CLI flags):
 
-| Env var          | Description                                                                                                             |
-| ---------------- | ----------------------------------------------------------------------------------------------------------------------- |
-| `HOST`           | Bind address                                                                                                            |
-| `PORT`           | Port                                                                                                                    |
-| `CC_API_KEY`     | Command Code API key                                                                                                    |
-| `CC_API_BASE`    | Upstream API base URL                                                                                                   |
-| `CC_CLI_VERSION` | CLI version sent upstream                                                                                               |
-| `LOG_LEVEL`      | Log level (`info`, `debug`, etc.)                                                                                       |
-| `CORS_ORIGIN`    | `Access-Control-Allow-Origin` value. `*` by default; empty string disables CORS. Restrict before exposing on a network. |
+| Env var                  | Description                                                                                                             |
+| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- |
+| `HOST`                   | Bind address                                                                                                            |
+| `PORT`                   | Port                                                                                                                    |
+| `CC_API_KEY`             | Command Code API key                                                                                                    |
+| `CC_API_BASE`            | Upstream API base URL                                                                                                   |
+| `CC_CLI_VERSION`         | CLI version sent upstream                                                                                               |
+| `CC_UPSTREAM_TIMEOUT_MS` | Max ms for upstream to return response headers + first byte (default `600000` / 10 min). Bump for slow reasoning models |
+| `CC_IDLE_TIMEOUT_MS`     | Max ms between consecutive stream chunks (default `120000` / 2 min). `0` disables — detects stalled upstreams            |
+| `LOG_LEVEL`              | Log level (`info`, `debug`, etc.)                                                                                       |
+| `CORS_ORIGIN`            | `Access-Control-Allow-Origin` value. `*` by default; empty string disables CORS. Restrict before exposing on a network. |
 
 > **Security:** the proxy forwards your paid Command Code key upstream and
 > accepts any auth token from clients (`proxy-managed`), so it is designed for

README.md

@@ -15,6 +15,10 @@ export interface Config {
   ccVersion: string;
   logLevel: string;
   corsOrigin: string;
+  /** Max wall-clock ms for upstream to send response headers + first byte. */
+  upstreamTimeoutMs: number;
+  /** Max ms between consecutive chunks during streaming. 0 = disabled. */
+  idleTimeoutMs: number;
 }
 
 /**
@@ -86,5 +90,34 @@ export function loadConfig(): Config {
   // empty to disable) before exposing the proxy on a network.
   const corsOrigin = process.env.CORS_ORIGIN ?? "*";
 
-  return { host, port, apiKey, ccApiBase, ccVersion, logLevel, corsOrigin };
+  // Upstream timeouts. The connection timeout covers the wall-clock time
+  // until the upstream returns response headers + first byte — bump it for
+  // slow reasoning models. The idle timeout catches stalled streams where
+  // the upstream opened the connection but stopped sending chunks
+  // mid-response (e.g. tool call hung on the upstream side). Set
+  // CC_IDLE_TIMEOUT_MS=0 to disable idle detection entirely.
+  const upstreamTimeoutMs = parsePositiveInt(
+    process.env.CC_UPSTREAM_TIMEOUT_MS,
+    600_000, // 10 minutes — covers high-effort reasoning models
+  );
+  const idleTimeoutMs = parsePositiveInt(process.env.CC_IDLE_TIMEOUT_MS, 120_000); // 2 minutes
+
+  return {
+    host,
+    port,
+    apiKey,
+    ccApiBase,
+    ccVersion,
+    logLevel,
+    corsOrigin,
+    upstreamTimeoutMs,
+    idleTimeoutMs,
+  };
+}
+
+function parsePositiveInt(raw: string | undefined, fallback: number): number {
+  if (raw == null || raw === "") return fallback;
+  const n = Number(raw);
+  if (!Number.isFinite(n) || n < 0) return fallback;
+  return Math.floor(n);
 }