Skip to content

fix(kernel): release an event claim when a panic unwinds past it (#173) - #182

Merged
kjgbot merged 1 commit into
mainfrom
fix/173-claim-panic-guard
Sep 5, 2026
Merged

fix(kernel): release an event claim when a panic unwinds past it (#173)#182
kjgbot merged 1 commit into
mainfrom
fix/173-claim-panic-guard

Conversation

@kjgbot

@kjgbot kjgbot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Closes #173, deferred from #171 on purpose so it would get its own review rather than riding under an exactly-once fix.

The gap

claim_event tells every concurrent delivery that an event is a duplicate on the strength of a claim row, so whoever takes that claim owes the event either a registered run or the claim back.

The Err path handed it back. A panic did not — an unwind passes every match arm, and a claim leaked that way survived for the life of the boot: each later delivery answered "duplicate" by the same-boot rule, with no run to carry the event. Only a restart cleared it, because the claim then belonged to a previous boot and was repaired.

The shape of the fix

A Drop guard covers the exit that has no ? to take. The ordinary Err path still releases explicitly and propagates — a release that fails there strands the claim while later deliveries are told "deduped" with nothing behind them, and that is a runtime failure the caller must see, not one to report and swallow. Release happens before disarming, so a failing release leaves the guard armed and the unwind still attempts best-effort cleanup on the way out.

This is the crate's first Drop impl, so the trade is stated in the code: it cannot return an error or be ?-ed, so a failed release there is reported via eprintln! — the convention already used in server/liveness.rs — and swallowed, leaving the event exactly where a leak would have left it, no worse. It must not panic: panicking in Drop during an unwind aborts the process, turning a stranded event into a dead daemon.

Evidence

Four tests, and the mutation that gates them. Making Drop a no-op (if true || !self.armed) fails exactly the two that should fail — an_armed_guard_releases_the_claim_when_dropped and a_panic_between_claim_and_register_still_releases — while a_disarmed_guard_leaves_the_claim_alone and a_guard_only_releases_its_own_run keep passing. So they pin the release without also pinning its absence.

The panic case runs through catch_unwind and asserts the panic actually happened, so it cannot pass by never unwinding.

Full transcript with runnable commands and sha256 before/mutated/restored is in the commit message. Kernel workspace: 156 passed, 0 failed, no warnings.

Known boundary, stated rather than glossed

The panic test constructs ClaimGuard directly rather than injecting a panic through submit_event, so it pins the guard's contract, not its placement. Placement is pinned by reading — armed immediately after the claim, disarmed only after spawn_claimed_run returns Ok — and injecting a panic into production code would need a test-only seam in the very span this change exists to protect.

Review history

Four drafts were rejected, each fairly, and one was about the code: routing the ordinary Err path through Drop silently downgraded a propagating release to best-effort. That was a real regression the review caught, not a wording problem. The other three were evidence claims — describing output instead of showing it, abbreviating cargo's test paths while calling the block literal, and printing commands without the cd kernel they were actually run in.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR

#173, deferred from #171 on purpose so it would get its own review.

`claim_event` tells every concurrent delivery that an event is a
duplicate on the strength of a claim row, so whoever takes that claim
owes the event either a registered run or the claim back. The `Err` path
handed it back; a PANIC did not. An unwind passes every match arm, and a
claim leaked that way survived for the life of the boot -- each later
delivery answered "duplicate" by the same-boot rule, with no run to carry
the event. Only a restart cleared it, because the claim then belonged to
a previous boot and was repaired.

A `Drop` guard covers the exit that has no `?` to take. The ORDINARY
`Err` path still releases explicitly and propagates: a release that fails
there strands the claim while later deliveries are told "deduped" with no
run behind them, and that is a runtime failure the caller must see rather
than one to report and swallow. Release happens before disarming, so a
failing release leaves the guard armed and the unwind still attempts a
best-effort cleanup on the way out. Best-effort is the right trade only
where the alternative is no cleanup at all.

First `Drop` impl in the crate, so the trade is stated in the code: it
cannot return an error or be `?`-ed, so a failed release there is
reported via `eprintln!` (the convention already used in
`server/liveness.rs`) and swallowed -- leaving the event exactly where a
leak would have left it, no worse. It must not panic: panicking in `Drop`
during an unwind aborts the process, turning a stranded event into a dead
daemon.

Evidence. Every command below is literally runnable from the repository
root, and the output is complete:

  $ shasum -a 256 kernel/relayflowd/src/engine/wake.rs
  bdc6d17b93f1aa5c3f256deeb5377943a68dec1fad6f018dee4dab88eaeb1cd3  kernel/relayflowd/src/engine/wake.rs

  $ (cd kernel && PATH="$HOME/.cargo/bin:$PATH" RUSTUP_TOOLCHAIN=stable cargo test -p relayflowd --lib claim_guard)
      Finished `test` profile [unoptimized + debuginfo] target(s) in 0.33s
       Running unittests src/lib.rs (target/debug/deps/relayflowd-01b0ec98cb5c81e7)

  running 4 tests
  test engine::wake::claim_guard_tests::a_disarmed_guard_leaves_the_claim_alone ... ok
  test engine::wake::claim_guard_tests::a_guard_only_releases_its_own_run ... ok
  test engine::wake::claim_guard_tests::an_armed_guard_releases_the_claim_when_dropped ... ok
  test engine::wake::claim_guard_tests::a_panic_between_claim_and_register_still_releases ... ok

  test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 25 filtered out; finished in 0.02s

  # MUTATION: Drop made a no-op -- `if true || !self.armed`
  $ shasum -a 256 kernel/relayflowd/src/engine/wake.rs
  548550ba2430c59fa1dd1760bb763b1d14e9f6a234dbe667b62cb73721bbb524  kernel/relayflowd/src/engine/wake.rs

  $ (cd kernel && PATH="$HOME/.cargo/bin:$PATH" RUSTUP_TOOLCHAIN=stable cargo test -p relayflowd --lib claim_guard)
     Compiling relayflowd v0.1.0 (/Users/khaliqgant/AgentWorkforce/flows-173/kernel/relayflowd)
      Finished `test` profile [unoptimized + debuginfo] target(s) in 2.24s
       Running unittests src/lib.rs (target/debug/deps/relayflowd-01b0ec98cb5c81e7)

  running 4 tests
  test engine::wake::claim_guard_tests::a_disarmed_guard_leaves_the_claim_alone ... ok
  test engine::wake::claim_guard_tests::an_armed_guard_releases_the_claim_when_dropped ... FAILED
  test engine::wake::claim_guard_tests::a_guard_only_releases_its_own_run ... ok
  test engine::wake::claim_guard_tests::a_panic_between_claim_and_register_still_releases ... FAILED

  failures:

  ---- engine::wake::claim_guard_tests::an_armed_guard_releases_the_claim_when_dropped stdout ----

  thread 'engine::wake::claim_guard_tests::an_armed_guard_releases_the_claim_when_dropped' (18172736) panicked at relayflowd/src/engine/wake.rs:367:9:
  a dropped armed guard must hand the event back

  ---- engine::wake::claim_guard_tests::a_panic_between_claim_and_register_still_releases stdout ----

  thread 'engine::wake::claim_guard_tests::a_panic_between_claim_and_register_still_releases' (18172735) panicked at relayflowd/src/engine/wake.rs:397:13:
  spawn_claimed_run blew up between the claim and register
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

  thread 'engine::wake::claim_guard_tests::a_panic_between_claim_and_register_still_releases' (18172735) panicked at relayflowd/src/engine/wake.rs:401:9:
  a claim leaked by a panic strands the event until the process restarts

  failures:
      engine::wake::claim_guard_tests::a_panic_between_claim_and_register_still_releases
      engine::wake::claim_guard_tests::an_armed_guard_releases_the_claim_when_dropped

  test result: FAILED. 2 passed; 2 failed; 0 ignored; 0 measured; 25 filtered out; finished in 0.01s

  error: test failed, to rerun pass `-p relayflowd --lib`

  # RESTORED
  $ shasum -a 256 kernel/relayflowd/src/engine/wake.rs
  bdc6d17b93f1aa5c3f256deeb5377943a68dec1fad6f018dee4dab88eaeb1cd3  kernel/relayflowd/src/engine/wake.rs
Exactly the two tests that should fail do, while the disarmed and
wrong-run cases keep passing -- so they pin the release without also
pinning its absence. The panic case runs through `catch_unwind` and
asserts the panic actually happened, so it cannot pass by never
unwinding. Kernel workspace at this head: 156 passed, 0 failed.

Known coverage boundary, stated rather than glossed: the panic test
constructs `ClaimGuard` directly instead of injecting a panic through
`submit_event`, so it pins the guard's contract, not its placement. The
placement is pinned by reading -- armed immediately after the claim,
disarmed only after `spawn_claimed_run` returns Ok -- and injecting a
panic into production code would need a test-only seam in the very span
this change exists to protect.

Four earlier drafts were rejected by review, each fairly, and only one
was about the code: the first described the evidence instead of showing
it; the second abbreviated cargo's test paths while calling the block
literal; the third routed the ordinary `Err` path through `Drop`, which
silently downgraded a propagating release to best-effort -- a real
regression; the fourth printed commands without the `cd kernel` they were
actually run in, so "runnable from the root" was false.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR

Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 12 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: afeb0930-a6fc-4ca4-9a93-b6e421b7aee0

📥 Commits

Reviewing files that changed from the base of the PR and between a93a6f7 and bc41f9e.

📒 Files selected for processing (1)
  • kernel/relayflowd/src/engine/wake.rs

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

kjgbot pushed a commit that referenced this pull request Sep 5, 2026
…eview rounds

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR

Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
@kjgbot
kjgbot merged commit ba9e68c into main Sep 5, 2026
2 of 3 checks passed
@kjgbot
kjgbot deleted the fix/173-claim-panic-guard branch September 5, 2026 17:41
kjgbot pushed a commit that referenced this pull request Sep 5, 2026
…ssues surfaced

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR

Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
kjgbot added a commit that referenced this pull request Sep 5, 2026
Closes #167.

The row claimed gate 2 was missing "a test proving a duplicate event does not double-execute". That was already false when #167 was filed, and tonight's work went further, so the row understated progress and the gate at once.

Done, each claim checked against main rather than remembered: sequential duplicates (#14); concurrent racing deliveries under the production topology of one `Engine` per protocol request (#171); claims surviving the process that made them (#171 boot id, #182 panic unwind); resume adopting only a journal it can actually use (#177, #186).

Still missing, narrowly: the RFC-0001 Appendix A wake-time context contract — nothing specifies what `wake_context` guarantees, or that a resumed run observes the same context rather than a recomputed one.

And the correction #167 cared about most: the row was understating the gate. RFC-0001 §3's bar is `hn-monitor` running as a relayflow in production on its real events with zero bespoke persistence, not a passing test suite.

Evidence at the merged head 8e57b17:
- signoff: local 3-lens preswarm, maintainability / history / structure all REVIEW_PASSED
- CI: none applies. `cloud-runtime-artifact.yml` filters on `kernel/**`, `sdk/**`, `testdata/**` and `scripts/cloud-artifact*`; this touches only `ops/SCOREBOARD.md`, so no artifact run was triggered — verified by reading the workflow's `paths:` rather than waiting on a run that was never going to start. The `review` check is red for the reason common to every flows PR: the gate invokes `agent-relay` and no step installs it.

No counts in the row, deliberately — counts drift with the base, PR numbers and test names do not.
kjgbot added a commit that referenced this pull request Sep 5, 2026
Nothing verified main. `cloud-runtime-artifact.yml` triggered only on `pull_request` and `workflow_dispatch`, and `review-swarm.yml` only on `pull_request` — so every PR is checked at its own head and never as merged, and since we squash-merge onto a main that has moved since that CI ran, the composed result went unverified.

Not theoretical: twelve PRs merged on 2026-09-05 across the exactly-once claim path (#171, #182), resume adoption (#177, #186), the authored-flow executor (#184, #187) and the CLI run loop (#180) — each green on its own branch, the composed tree never run until dispatched by hand:

```
run 33987924703  workflow_dispatch  main  completed/success  ed917bf
```

Main is fine. But nobody knew that, and finding out required knowing to ask.

Without this, a bad compose surfaces as an unrelated PR going red — the most expensive way to find it, since the author debugs their own change first. Tonight already produced three failures on PRs that belonged to something else (#179, #185, the #174 chain), and each cost a tick to attribute.

Paths are deliberately not filtered on the push trigger: on a PR the question is "does this change affect the runtime", on main it is "is the tree good", and a docs-only merge can land on a tree someone else broke.

Evidence at the merged head d1cb32a:
- signoff: local 3-lens preswarm, maintainability / history / structure all REVIEW_PASSED
- CI: run 33988314276 success on d1cb32a, verified by headSha and event=pull_request
- YAML parsed and asserted: triggers are `pull_request`, `push`, `workflow_dispatch`; push is branch-scoped to main; the pull_request path filter is unchanged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claim between claim_event and register is not panic-safe

1 participant