fix(cli): a running run is not a protocol error (#179) - #180
Conversation
`flows run` intermittently failed a healthy run with:
FAILED [protocol_error] relayflowd could not complete the run request:
relayflowd returned status parked without a classifiable completion
Reproduced locally: 1 failure in 12 runs, and 1 in 4 on another pass.
Instrumenting the fall-through branch named the state exactly:
PROBE_D unclassifiable inspection={"status":"running","needsHuman":false}
`classifyOutcome` loops while the run is parked, asking
`inspectOutOfBandStep` what to do. That helper reports a step only when a
non-deterministic step is `needs_human`, `runnable` or `running`. When a
worker has just completed the step the run parked on, there is a window
where none of those hold while the snapshot's own status is still
`running` -- the daemon has not yet finished driving what follows.
Every branch missed that window, so the loop broke with
`status === 'parked'` and no `parkedStep`, and the tail reported an
unclassifiable completion. The run was healthy and about to succeed.
A run that reports `running` is progressing, so it is now polled rather
than abandoned: bounded at 40 x 50ms = 2s, which is far longer than the
sub-second window observed and short enough that a genuinely stuck run
still reports instead of hanging. The unchanged `break` below keeps it
failing closed if it never resolves.
Evidence:
* before: 1 fail / 12, then 1 fail / 4
* after: 0 fail / 40
* full SDK suite: 31 files passed, 649 tests passed, 3 skipped
* typecheck clean
This is #179's `live-kernel > follows a live worker dispatch` flake, and
also explains the earlier `cli-hn-monitor` failure shape: both are the
CLI treating a transient, healthy state as terminal.
Reproduction environment, since it is not obvious: surface must be built
with `bun install --frozen-lockfile --ignore-scripts && bun run build`
BEFORE the sdk (npm there fails -- surface has no package-lock.json),
then `npm ci --prefix sdk --ignore-scripts`, a release `relayflowd` for
RELAYFLOWD_BIN, and RELAYFLOWS_ALLOW_ANALYZER_SKIP=1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
|
Warning Review limit reachedNext included review available in 18 minutes. View limit detailsLimit details: You’ve used the included review currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (2)
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing. Comment |
The maintainability lens caught a regression I introduced: the poll used a bare `setTimeout`, while every other wait in this file goes through the cancel-aware `delay(ms, signal)`. A Ctrl-C or lifecycle abort during the 2-second window would have been ignored and the CLI would have kept polling the daemon after being told to stop. Now `throwIfCanceled(options.signal, ...)` then `delay(..., options.signal)`, matching `waitForRunningStep` directly above it. Also stated the contract the loop leans on -- `run.resume` is idempotent on a run already progressing, returning current state rather than re-dispatching -- because up to 41 resumes in 2 seconds is a load-bearing assumption that was written nowhere. Style: the constants used `///`, a Rust doc comment, in a file that uses `//`. Re-verified after the change: 20/20 on the previously-flaking test, and the full SDK suite still 31 files / 649 tests passed, 3 skipped. Still outstanding from that review, and NOT done here: the new branch has no unit test. `classifyOutcome` is unexported with no test file, so pinning it means making it injectable or exported first -- a design change that deserves its own pass rather than a rushed one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
The maintainability lens wanted the new branch tested, and was right that without one a refactor of `inspectOutOfBandStep` -- or a tightening of the outer `while (current.status === 'parked')` guard -- would kill it silently and regress #179. `classifyOutcome` is now exported for tests. The branch only occurs in a sub-second window against a live daemon, so pinning it needs a stubbed client rather than a real run: the integration test that found the bug reproduced it roughly 1 time in 12, which is not a gate. Two cases, and they state the rule together: * a run reporting `running` with no identifiable step is waited out and then classified normally -- no `protocol_error`, exit 3, the parked step identified * a run that NEVER becomes classifiable still reports rather than polling forever, which is what keeping the original `break` buys Mutation: disabling the branch (`if (false && ...)`) fails the first test with the exact production string -- expected [ 'protocol_error' ] to not include 'protocol_error' "relayflowd returned status parked without a classifiable completion" -- while the bound test still passes. sha256 8124009e -> e80ee063 -> restored 8124009e, no `if (false &&` left in the tree. Full SDK suite at this head: 32 files passed, 651 tests passed, 3 skipped; typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
…lf is a distinct bug Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
…5 measurement Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
Nothing verified main. `cloud-runtime-artifact.yml` triggered only on `pull_request` and `workflow_dispatch`, and `review-swarm.yml` only on `pull_request` — so every PR is checked at its own head and never as merged, and since we squash-merge onto a main that has moved since that CI ran, the composed result went unverified. Not theoretical: twelve PRs merged on 2026-09-05 across the exactly-once claim path (#171, #182), resume adoption (#177, #186), the authored-flow executor (#184, #187) and the CLI run loop (#180) — each green on its own branch, the composed tree never run until dispatched by hand: ``` run 33987924703 workflow_dispatch main completed/success ed917bf ``` Main is fine. But nobody knew that, and finding out required knowing to ask. Without this, a bad compose surfaces as an unrelated PR going red — the most expensive way to find it, since the author debugs their own change first. Tonight already produced three failures on PRs that belonged to something else (#179, #185, the #174 chain), and each cost a tick to attribute. Paths are deliberately not filtered on the push trigger: on a PR the question is "does this change affect the runtime", on main it is "is the tree good", and a docs-only merge can land on a tree someone else broke. Evidence at the merged head d1cb32a: - signoff: local 3-lens preswarm, maintainability / history / structure all REVIEW_PASSED - CI: run 33988314276 success on d1cb32a, verified by headSha and event=pull_request - YAML parsed and asserted: triggers are `pull_request`, `push`, `workflow_dispatch`; push is branch-scoped to main; the pull_request path filter is unchanged
Fixes the
live-kernel > follows a live worker dispatch through flows runflake from #179, which failed CI three times tonight and forced a re-run each time.Reproduced first
Built a worktree at main's tip to CI's own spec and ran the test in a loop: 1 failure in 12, then 1 in 4 on a second pass. Instrumenting the fall-through branch named the state exactly:
The bug
classifyOutcomeloops while the run is parked, askinginspectOutOfBandStepwhat to do next. That helper reports a step only when a non-deterministic step isneeds_human,runnableorrunning.When a worker has just completed the step the run parked on, there is a window where none of those hold while the snapshot's own status is still
running— the daemon has not yet finished driving what follows. Every branch missed that window, so the loop broke withstatus === 'parked'and noparkedStep, and the tail reported:The run was healthy and about to succeed. The CLI failed it for being observed mid-stride.
The fix
A run that reports
runningis progressing, so it is polled rather than abandoned — bounded at 40 × 50ms = 2s, far longer than the sub-second window observed and short enough that a genuinely stuck run still reports instead of hanging. The unchangedbreakbelow keeps it failing closed if it never resolves.Evidence
This also explains the
cli-hn-monitorfailure shape from #179's table: both are the CLI treating a transient, healthy state as terminal. Worth re-checking that one once this lands.Note on the
unprovable_effectswarningBoth CI failures quoted
WARNING [unprovable_effects] Step "greet" command "printf"...in their assertion message. That is a red herring — it appears in stderr on every run, passing or failing, and is simply the first line of the captured stderr the assertion prints. The real error was always theprotocol_errorbeneath it.Reproduction environment
Not obvious, and it cost me a tick to get right:
surfacemust be installed and built before the sdk, with bun (bun install --frozen-lockfile --ignore-scripts && bun run build) — npm fails there because surface has nopackage-lock.json. Thennpm ci --prefix sdk --ignore-scripts, a releaserelayflowdforRELAYFLOWD_BIN, andRELAYFLOWS_ALLOW_ANALYZER_SKIP=1.🤖 Generated with Claude Code
https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR