Skip to content

fix(server): stop reaping sessions with live background work - #5690

Closed
liusqu wants to merge 1 commit into
pingdotgg:mainfrom
liusqu:fix/session-reaper-background-tasks
Closed

fix(server): stop reaping sessions with live background work#5690
liusqu wants to merge 1 commit into
pingdotgg:mainfrom
liusqu:fix/session-reaper-background-tasks

Conversation

@liusqu

@liusqu liusqu commented Aug 8, 2026

Copy link
Copy Markdown

Problem

ProviderSessionReaper measures idleness from ProviderSessionDirectory.lastSeenAt, which only advances on turns (ProviderService.sendTurn). But a Claude session keeps working between turns — subagent fleets, workflow runs, background shells all outlive the turn that spawned them. A session with no user turn for 30 minutes gets stopSession'd while its agents are still running, silently killing the in-flight work.

Hit this in production use tonight: a 30+ minute background implementation agent was killed mid-run twice; the only surviving evidence was the worktree state.

Fix

The projection already answers "is native background work still alive": ThreadBackgroundLiveness is fed by the canonical task lifecycle stream for every provider, drops terminal/idle/inert tasks, and is cleared on session.exited. Its backgroundLiveness verdict already rides on the OrchestrationThreadShell the reaper fetches for its existing activeTurnId check — so this is a read of an existing field, not new plumbing. Two files touched.

  • Idle-by-lastSeenAt sessions are spared while backgroundLiveness === "working".
  • "monitoring" deliberately does not earn a reprieve: watch loops tick forever by design and would make sessions immortal, turning the sweep into a no-op as a leak backstop. Losing a watch loop is recoverable; losing a running agent is not.
  • The reprieve is bounded by backgroundExtensionLimitMultiplier (48× threshold = 24 h by default, configurable like the other two options), so a wedged-but-chatty agent cannot pin its session open forever.
  • The activeTurnId skip is unchanged.

Tests

Four new cases in ProviderSessionReaper.test.ts (idle+working → spared; monitoring → reaped; null → reaped; working past ceiling → reaped). Red-green verified against two deliberately unpatched variants — each new guard is independently load-bearing. Full src/provider suite: 490 passed, 6 skipped. Lint/format clean.

🤖 Generated with Claude Code


Note

Medium Risk
Changes when long-idle provider sessions are stopped, which can affect in-flight subagents vs. session leaks; behavior is bounded by a configurable ceiling and covered by new tests.

Overview
Provider session reaper no longer stops sessions that are idle on lastSeenAt but still have real background agent work, using the thread shell’s existing backgroundLiveness field.

After the usual inactivity and activeTurnId checks, the sweep skips stopSession when backgroundLiveness === "working" and idle time is below a new cap (inactivityThresholdMs × backgroundExtensionLimitMultiplier, default 48× → 24h). "monitoring" does not get that reprieve (watch loops would otherwise keep sessions open indefinitely). null or idle past the cap is still reaped, including wedged agents that keep reporting "working".

Tests cover working (spared), monitoring/null (reaped), and working past a lowered multiplier ceiling (reaped); the harness can pass backgroundExtensionLimitMultiplier.

Reviewed by Cursor Bugbot for commit 23f9b8b. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Stop reaping provider sessions with live background work

  • The session reaper now skips stopping sessions when backgroundLiveness is 'working' and the idle duration is below a configurable ceiling (inactivityThresholdMs × backgroundExtensionLimitMultiplier, defaulting to 48×).
  • Sessions with backgroundLiveness of 'monitoring' or null continue to be reaped at the normal inactivity threshold.
  • Sessions where background work outlives the ceiling are reaped regardless.
  • A new debug log provider.session.reaper.skipped-background-work is emitted when a session is deferred.

Macroscope summarized 23f9b8b.

Root cause: the reaper treats a session as idle from
`ProviderSessionDirectory.lastSeenAt`, which only advances on turns
(`ProviderService.sendTurn` — "a turn is the clearest sign a session is
still alive"). But a session keeps working between turns: subagent
fleets, workflow runs and background shells outlive the turn that
spawned them. A session with no user turn for 30 minutes was
`stopSession`'d while its agents were still running, silently killing
30+ minutes of in-flight work.

The projection already answers this question. `ThreadBackgroundLiveness`
is fed by the same task lifecycle stream for every provider, drops
terminal/idle/inert tasks, and is cleared on `session.exited`; its
`backgroundLiveness` verdict already rides on the `OrchestrationThreadShell`
that the reaper fetches for its `activeTurnId` check. So this is a
read of an existing field, not new plumbing.

Only "working" earns a reprieve. "monitoring" is watch loops (monitor
tasks, background shells) that tick forever by design — honouring it
would make those sessions immortal and turn the sweep into a no-op as a
leak backstop. Losing a watch loop to the reaper is recoverable; losing
a running agent is not.

The reprieve is bounded by `backgroundExtensionLimitMultiplier`
(48 x the inactivity threshold = 24h by default), so an agent that is
wedged yet still emitting progress cannot pin its session open forever.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1693a9f0-0000-4db9-add2-1e487704a51c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 8, 2026
@macroscopeapp

macroscopeapp Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR changes session reaper behavior to preserve sessions with active background work for up to 24 hours. While the fix intent is clear and tests are comprehensive, this fundamentally alters when sessions get terminated, affecting resource management and production runtime behavior. Human review recommended for this behavioral change.

You can customize Macroscope's approvability policy. Learn more.

@shivamhwp

Copy link
Copy Markdown
Collaborator

gpt-5.6-sol on behalf of shivamhwp.

Thank you for working on the session reaper. We are closing this PR because #5677 has already landed the guard that prevents the reaper from killing sessions with live background agents. It updates the same server path and covers the behavior proposed here.

The fix is present on main, so keeping this branch open would duplicate completed work.

@shivamhwp shivamhwp closed this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants