Skip to content

instrumentDurableObjectWithSentry prevents WebSocket Hibernation — a DO holding an accepted socket is billed ~100% wall-clock duration #22328

Description

@rodrigoceloto

Package: @sentry/cloudflare
Version: 10.43.0
Runtime: Cloudflare Workers / Durable Objects, hibernatable WebSockets (ctx.acceptWebSocket + ctx.setWebSocketAutoResponse)

Summary

Wrapping a Durable Object class with instrumentDurableObjectWithSentry prevents the WebSocket Hibernation API from ever hibernating the object. A DO holding one idle, edge-ping-answered socket bills exactly the bucket length of activeTime in every Cloudflare analytics bucket (e.g. 300.0s per 5-min bucket) with cpuTime ≈ 0.01s and subrequests = 0, indefinitely. Since the cost is per connected socket, it scales linearly with concurrent users.

Production A/B evidence (matched 21h workload windows)

Removing ONLY the wrapper (same app code, same clients, deploy-minute-aligned collapse):

wrapped unwrapped
total wakes (alarm+fetch+ws+jsrpc) 3,279 2,668 (−19%)
GB·s duration 9,450 184.2 (−98.1%)
GB·s per wake 2.882 0.069 (41.7× cheaper)
constructions ~1 per deploy (pinned in memory) ~1 per wake (hibernating)

Pre-fix the burn was exactly 3600.0s/hour regardless of event count — residency, not work.

Source observations (v10.43.0) that plausibly contribute

  1. CloudflareClient.flush(timeout) does not bound the flush-lock wait. It runs await this._flushLock.finalize() before any timeout applies; the timeout only bounds span-completion and transport. Any promise handed to ctx.waitUntil that never settles wedges every subsequent flush in the object forever.
  2. Unbounded waitUntil override stacking. The construct trap substitutes an Object.create shim for ctx (instrumentContext); every init() runs makeFlushLock(context), which reassigns context.waitUntil, one closure layer per init, never removed — unbounded on an instance that lives for a socket's lifetime.
  3. The 101-upgrade client leaks to the global default scope and is then disposed out from under the socket. wrapRequestHandler's 101 branch deliberately skips dispose() so webSocketMessage handlers can still capture — but initAndBindsetCurrentClient() writes the client to the ambient current scope, which at invocation entry (no ALS store active yet) is the global default scope. The next wrapped invocation (wrapMethodWithSentry) finds it via getClient(), skips init, and its terminal flushAndDispose disposes it. Likely the root cause of @sentry/cloudflare not reporting errors thrown from Durable Objects #16786 (DO errors silently unreported).
  4. Per-event global re-instrumentation: every init() re-registers console/fetch handlers and re-runs setAsyncLocalStorageAsyncContextStrategy().
  5. markAsInstrumented flags prototype methods for webSocketMessage/Close/Error, so the 2nd+ instance constructed in the same isolate silently gets unwrapped socket handlers (inconsistent instrumentation).

We could not pin the exact workerd-side hibernation-readiness clause from JS-side reading alone (every individual flush path is time-bounded under nominal promise settlement, and DurableObjectState.waitUntil is a documented no-op) — but the wrapper-boundary causality is production-proven. A minimal echo-DO ± wrapper repro should reproduce it directly.

Related

Metadata

Metadata

Assignees

Fields

No fields configured for issues without a type.

Projects

Status
No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions