Skip to content

@sentry/cloudflare: scheduled and queue handlers reuse one trace ID per isolate (no propagation-context reset) #24118

Description

@msnelling

Is there an existing issue for this?

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/cloudflare

SDK Version

10.72.0 (code unchanged on develop as of 10.73.0)

Framework Version

Cloudflare Workers (workerd), nodejs_compat

Link to Sentry event

No single event — see the production evidence below; happy to share trace IDs privately.

Reproduction Example/SDK Setup

Minimal reproduction against the installed SDK (Node, no DSN needed — it only exercises scope/propagation-context handling):

// repro.cjs — run with: node repro.cjs
const path = require('path');
const core = require('@sentry/core');
const cf = require.resolve('@sentry/cloudflare');
const { setAsyncLocalStorageAsyncContextStrategy } = require(path.join(path.dirname(cf), 'async.js'));

setAsyncLocalStorageAsyncContextStrategy();

// What wrapScheduledHandler / wrapQueueHandler do before startSpan():
const ids = [];
for (let i = 0; i < 3; i++) {
  core.withIsolationScope((iso) => {
    const merged = { ...iso.getPropagationContext(), ...core.getCurrentScope().getPropagationContext() };
    ids.push(merged.traceId); // the traceId createChildOrRootSpan() gives the root span
  });
}
console.log('scheduled/queue path:', ids, '| all identical:', new Set(ids).size === 1);

// What wrapRequestHandler does (continueTrace with empty headers):
const fetchIds = [];
for (let i = 0; i < 3; i++) {
  core.withIsolationScope(() => {
    core.continueTrace({ sentryTrace: '', baggage: null }, () => {
      fetchIds.push(core.getCurrentScope().getPropagationContext().traceId);
    });
  });
}
console.log('fetch path:', fetchIds, '| all distinct:', new Set(fetchIds).size === 3);

// Proposed fix: startNewTrace outside withIsolationScope
const fixed = [];
for (let i = 0; i < 3; i++) {
  core.startNewTrace(() => {
    core.withIsolationScope((iso) => {
      fixed.push({ ...iso.getPropagationContext(), ...core.getCurrentScope().getPropagationContext() }.traceId);
    });
  });
}
console.log('with startNewTrace:', fixed, '| all distinct:', new Set(fixed).size === 3);

Output:

scheduled/queue path: [ '2bdf65cd…', '2bdf65cd…', '2bdf65cd…' ] | all identical: true
fetch path:           [ '41000aa9…', '00a18b63…', 'd5ff2bb2…' ] | all distinct: true
with startNewTrace:   [ 'a8f471f5…', 'f63834f3…', '46f93be6…' ] | all distinct: true

Steps to Reproduce

  1. Export a Worker with withSentry(env => ({ dsn, tracesSampleRate: 1 }), { scheduled, queue }).
  2. Let the cron trigger fire several times on a warm isolate (or consume several queue batches).
  3. Look at the resulting faas.cron / queue.process transactions in Sentry.

Expected Result

Each scheduled invocation and each queue batch starts a new trace, the same way each fetch invocation does and the same way wrapMethodWithSentry handles Durable Object alarms and RPC methods (startNewTrace: true).

Actual Result

Every scheduled invocation and every queue batch on the same isolate shares one trace ID. The ID only changes when the isolate is replaced.

Production evidence (24 h window, span.op:faas.cron, three separate Workers):

Worker Trace ID Consecutive scheduled runs on it
A (*/5 * * * *) 1fb7da5c… 17 runs, 11:25 → 12:45
B (*/5 * * * *) ca086e7b… 11:25 → 12:35, then rotates on isolate replacement
C (*/15 * * * *) 3063794a… 11:15 → 12:30

Queue consumers show the same: unrelated batches from different tenants (process <queue> spans over ~10 minutes) land in one trace. A cron-monitor check-in failure that links to "the trace" links to a 75-minute mega-trace covering every tick.

Root cause

wrapScheduledHandler (packages/cloudflare/src/instrumentations/worker/instrumentScheduled.ts) and wrapQueueHandler (instrumentQueue.ts) do:

withIsolationScope(isolationScope => {
  
  return startSpan({}, );
});

with no continueTrace / startNewTrace / setPropagationContext. The Cloudflare async-context strategy's withIsolationScope (packages/cloudflare/src/async.ts) clones only the isolation scope and reuses the current scope object:

function withIsolationScope(callback) {
  const scope = getScopes().scope;                 // not cloned, not reset
  const isolationScope = getScopes().isolationScope.clone();
  
}

At a cron/queue entry point there is no ALS store, so getScopes().scope is getDefaultCurrentScope() — a getGlobalSingleton('defaultCurrentScope'), one per isolate — whose _propagationContext.traceId is generated once in the Scope constructor and never reset on this path. createChildOrRootSpan then takes the root span's traceId from { ...isolationScope.getPropagationContext(), ...scope.getPropagationContext() }, i.e. the pinned one.

The fetch path escapes because wrapRequestHandler calls continueTrace({ sentryTrace, baggage }), which sets a fresh propagation context per request even when the headers are absent.

Proposed fix

Mirror what wrapMethodWithSentry already does for DO alarms / RPC — wrap the body in startNewTrace (outside withIsolationScope, or reset the propagation context on the forked scope before startSpan):

function wrapScheduledHandler(controller, options, context, fn) {
  return startNewTrace(() =>
    withIsolationScope(isolationScope => {
      
      return startSpan({}, );
    }),
  );
}

Same for wrapQueueHandler (and, by inspection, instrumentEmail / instrumentTail, which follow the same shape). We're carrying an application-side startNewTrace wrapper around the exported scheduled/queue handlers in the meantime, with a control test pinning the current SDK behaviour so we can drop the wrapper once this lands.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions