Skip to content

cancellation token#2414

Open
GarrettBeatty wants to merge 6 commits into
feature/durablefunctionfrom
gcbeatty/cancellationtoken
Open

cancellation token#2414
GarrettBeatty wants to merge 6 commits into
feature/durablefunctionfrom
gcbeatty/cancellationtoken

Conversation

@GarrettBeatty

@GarrettBeatty GarrettBeatty commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Issue #, if available:

Description of changes:

Threads a CancellationToken into every user Func accepted by IDurableContext so step bodies can observe both upstream caller cancel intent and SDK-driven workflow teardown.

What changed

The four IDurableContext methods that take a user Func now pass a CancellationToken into that Func:

  • StepAsync (both overloads): Func<IStepContext, CancellationToken, Task<T>>
  • RunInChildContextAsync (both overloads): Func<IDurableContext, CancellationToken, Task<T>>
  • WaitForCallbackAsync submitter: Func<string, IWaitForCallbackContext, CancellationToken, Task>
  • WaitForConditionAsync check: Func<TState, IConditionCheckContext, CancellationToken, Task<TState>>

WaitAsync, CreateCallbackAsync, and InvokeAsync are unchanged — they take no user Func.

The token is a linked source combining:

  1. The CancellationToken parameter passed to the IDurableContext method (the caller's intent).
  2. A new SDK-owned workflow-shutdown signal (WorkflowCancellation) that fires when TerminationManager resolves — i.e. when the workflow is being torn down for any reason (wait scheduled, callback pending, retry scheduled, invoke pending, checkpoint failed).

Semantics

  • An OperationCanceledException thrown out of a step body via the linked token (linked.IsCancellationRequested == true) is treated as cancellation: no FAIL checkpoint is written, no retry is consulted, the exception propagates up. The when (linked.IsCancellationRequested) filter distinguishes our cancel from a stray user-thrown OCE.
  • An OperationCanceledException thrown by user code for unrelated reasons (token never fired) continues to be treated as a normal step failure: FAIL checkpoint, retry per the configured RetryStrategy.
  • SDK-internal writes (checkpoint serialization, batcher flush, response build) do not observe the workflow token. Successful work is never lost to teardown.
  • Cached operation replays short-circuit before the user Func is invoked, so a cancelled token cannot affect replay determinism.

Why now

We're in preview. Landing the breaking delegate-shape change before GA costs every preview user one mechanical lambda update; landing it post-GA would be much more disruptive.

Public API impact

This is a breaking change to the user-facing delegate signatures on IDurableContext. Every existing user lambda must add a parameter (or _):

// Before
await ctx.StepAsync(async step => await DoWork(), name: "work");

// After
await ctx.StepAsync(async (step, ct) => await DoWork(ct), name: "work");

Per .autover/changes/: bumped as a Patch because the package is unreleased preview.

Tests

  • All 318 existing unit tests updated and passing on net8.0.
  • New WorkflowCancellationTests (10 tests) covering: token primitive, caller-token propagation, termination → cancel, user-OCE-as-failure, no-checkpoint-on-cancel, child-context propagation, condition-check token, replay short-circuit.
  • All 34 integration test function projects + the AOT publish test compile cleanly with the new signatures.

Docs

  • XML doc updated on every changed IDurableContext Func parameter to describe the linked-token contract.
  • docs/core/{steps,child-contexts,callbacks,wait-for-condition}.md examples updated to take and forward the token.
  • New docs/core/cancellation.md documenting the contract and user guidance ("don't branch workflow logic on IsCancellationRequested", "don't catch and continue", "do pass ct to every cancellation-aware API").
  • Design doc at Libraries/src/Amazon.Lambda.DurableExecution/docs/design/cancellation-design.md.

Out of scope

  • Source generator changes (the Amazon.Lambda.Annotations generator's emitted entry point will pass the workflow token into the user's top-level handler — separate change).
  • Parallel/map operations (their cancellation rides on the same WorkflowCancellation; semantic decisions belong to that design).
  • Lambda deadline timer (considered, rejected — Lambda's timeout is the deadline backstop).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@GarrettBeatty GarrettBeatty force-pushed the gcbeatty/cancellationtoken branch from 33bcf64 to a92f8ef Compare June 10, 2026 18:33
@GarrettBeatty GarrettBeatty requested a review from Copilot June 10, 2026 18:43
@GarrettBeatty GarrettBeatty marked this pull request as ready for review June 10, 2026 18:46
@GarrettBeatty GarrettBeatty requested review from a team as code owners June 10, 2026 18:46
@GarrettBeatty GarrettBeatty requested review from normj and philasmar and removed request for a team June 10, 2026 18:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces workflow-scoped cancellation semantics for the DurableExecution SDK by threading a linked CancellationToken into every user delegate accepted by IDurableContext, allowing user code to unwind promptly when a termination signal occurs (e.g., suspend/pending) while keeping durable checkpointing deterministic.

Changes:

  • Add an internal WorkflowCancellation source that cancels when TerminationManager.TerminationTask resolves.
  • Update public IDurableContext APIs so user delegates receive (…, CancellationToken) and operations link the caller token with the workflow-shutdown token.
  • Update docs, unit tests, and integration tests to the new delegate signatures and document cancellation behavior.

Reviewed changes

Copilot reviewed 54 out of 54 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
Libraries/test/Amazon.Lambda.DurableExecution.Tests/WorkflowCancellationTests.cs Adds unit tests covering workflow-shutdown cancellation and linked-token semantics.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/WaitForConditionOperationTests.cs Updates tests to the new WaitForConditionAsync delegate signature and context construction.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/WaitForCallbackTests.cs Updates callback tests to the new submitter signature and updated context construction.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/InvokeOperationTests.cs Updates tests to new StepAsync signature and updated context construction.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/DurableFunctionTests.cs Updates workflow tests to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/DurableContextTests.cs Updates context tests to new StepAsync signature and updated context construction.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/ChildContextOperationTests.cs Updates child-context tests to new RunInChildContextAsync signature and updated context construction.
Libraries/test/Amazon.Lambda.DurableExecution.Tests/CallbackOperationTests.cs Updates callback tests to updated context construction.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForConditionUserCheckThrowsFunction/Function.cs Updates integration workflow to new WaitForConditionAsync check signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForConditionReplayDeterminismFunction/Function.cs Updates integration workflow to new StepAsync and WaitForConditionAsync signatures.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForConditionMaxAttemptsFunction/Function.cs Updates integration workflow to new WaitForConditionAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForConditionHappyPathFunction/Function.cs Updates integration workflow to new WaitForConditionAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForConditionExponentialFunction/Function.cs Updates integration workflow to new WaitForConditionAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForCallbackSubmitterFailsFunction/Function.cs Updates integration workflow submitter signature to include CancellationToken.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/WaitForCallbackHappyPathFunction/Function.cs Updates integration workflow submitter signature and passes token into SDK client call.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/StepWaitStepFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/StepFailsFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/RetryFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/RetryExhaustionFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ReplayDeterminismFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ReplayAwareLoggerFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/MultipleStepsFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/LongRetryChainFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/LongerWaitFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/InvokeReplayDeterminismParentFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/InvokeReplayDeterminismChildFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/InvokeHappyPathChildFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/InvokeFailureChildFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/InvokeChildTenantFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/CreateCallbackHappyPathFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ChildContextRetryFailsFunction/Function.cs Updates integration workflow to new RunInChildContextAsync/StepAsync signatures.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ChildContextFunction/Function.cs Updates integration workflow to new RunInChildContextAsync/StepAsync signatures.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ChildContextFailsFunction/Function.cs Updates integration workflow to new RunInChildContextAsync/StepAsync signatures.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/CallbackFailedFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/AtMostOnceCrashFunction/Function.cs Updates integration workflow to new StepAsync signature.
Libraries/test/Amazon.Lambda.DurableExecution.AotPublishTest/Program.cs Updates AOT publish test workflow to new StepAsync signature.
Libraries/src/Amazon.Lambda.DurableExecution/WaitForCallbackConfig.cs Updates XML doc signature reference for WaitForCallbackAsync to include token.
Libraries/src/Amazon.Lambda.DurableExecution/README.md Documents the new linked cancellation token and updates sample code to use it.
Libraries/src/Amazon.Lambda.DurableExecution/IWaitForCallbackContext.cs Updates XML doc signature reference for WaitForCallbackAsync to include token.
Libraries/src/Amazon.Lambda.DurableExecution/Internal/WorkflowCancellation.cs Adds workflow-scoped cancellation source tied to termination.
Libraries/src/Amazon.Lambda.DurableExecution/Internal/WaitForConditionOperation.cs Threads linked token into condition check and treats linked-token cancellation as non-failure.
Libraries/src/Amazon.Lambda.DurableExecution/Internal/StepOperation.cs Threads linked token into step body and treats linked-token cancellation as non-failure.
Libraries/src/Amazon.Lambda.DurableExecution/Internal/ChildContextOperation.cs Threads linked token into child-context delegate and treats linked-token cancellation as non-failure.
Libraries/src/Amazon.Lambda.DurableExecution/IDurableContext.cs Updates public API delegate signatures and documents linked-token semantics.
Libraries/src/Amazon.Lambda.DurableExecution/DurableFunction.cs Creates/disposes WorkflowCancellation per invocation and injects into DurableContext.
Libraries/src/Amazon.Lambda.DurableExecution/DurableContext.cs Wires WorkflowCancellation through operation construction and updates WaitForCallback composition.
Libraries/src/Amazon.Lambda.DurableExecution/docs/core/wait-for-condition.md Updates docs to include cancellation token in check delegate and narrative.
Libraries/src/Amazon.Lambda.DurableExecution/docs/core/steps.md Updates docs to include cancellation token in step delegate and narrative.
Libraries/src/Amazon.Lambda.DurableExecution/docs/core/child-contexts.md Updates docs to include cancellation token in child-context delegate signature and example.
Libraries/src/Amazon.Lambda.DurableExecution/docs/core/cancellation.md Adds new cancellation documentation describing linked-token semantics and guidance.
Libraries/src/Amazon.Lambda.DurableExecution/docs/core/callbacks.md Updates docs to include cancellation token in submitter delegate and example.
Libraries/src/Amazon.Lambda.DurableExecution/ChildContextConfig.cs Updates XML doc signature reference for RunInChildContextAsync to include token.
Libraries/src/Amazon.Lambda.DurableExecution/CallbackException.cs Updates XML doc signature references for WaitForCallbackAsync to include token.
.autover/changes/110c67f8-15d8-44b5-beae-6f1223933c27.json Adds a changelog entry for the linked cancellation token behavior and API changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Libraries/src/Amazon.Lambda.DurableExecution/DurableContext.cs Outdated
Comment thread .autover/changes/110c67f8-15d8-44b5-beae-6f1223933c27.json

@normj normj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved assuming my understand is correct that when this is merged in ParallelAsync PR that the `WorkflowCancellation will also be cancelled when we are awaiting the inflight tasks in the finally block.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants