fix(review-swarm): print why the swarm failed, not just that it did - #235
Conversation
The gate polls `agent-relay cloud status --json`, reads `.status` off the
response and throws the rest away. When a swarm fails, the only thing that
reaches the GitHub log is the word:
Review swarm did not complete successfully: failed
The actual reason is already in the payload the poll just fetched. For the
five failing runs on 2026-09-07 it was:
Step "lens-maintainability" failed after 2 retries:
Total CPU limit exceeded. Maximum allowed: 250.
Nothing in the log said "quota". Diagnosing it meant knowing to pull the run
id out of the log and query the run by hand, so the cause went uninvestigated
for days while the failures were attributed to a guess.
Surface `.result.error` in the wait step, to stderr and to the step summary.
Verified against the real 04da7e48 payload: the expression yields the quota
text above; a payload without an error yields an empty string rather than the
literal "null", and an empty response (the first-call-failed path) is safe.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…lows #235 A single run places at current occupancy — the failed swarm got its orchestrator sandbox and only the lens fan-out was refused. The proof is a single run, so the missing preview vehicle was the real blocker all along. Also stopped calling trends on a metric that oscillates +-10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
There was a problem hiding this comment.
All reported issues were addressed across 1 file
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Review swarm: maintainabilityNo fresh transcript was produced for run |
Review swarm: historyNo fresh transcript was produced for run |
Review swarm: structureNo fresh transcript was produced for run |
Review swarm: FAILED
Cloud run: |
…he same cap Had ruled out prod from a variable listing whose visible portion could not have contained the answer; checked all 17 and the conclusion holds. #235's own review is the sixth run to fail on the quota while logging only the word "failed". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
maintainability lens — PASSMaintainability review — PR #235Scope. Six lines of shell added after the poll loop in Concerns
Notes
BlockersNone. The change is additive, degrades to no-output on the schema/state edge cases, and does not affect the exit-code path. REVIEW_PASSED |
history lens — PASSBlockers: none. PR #235’s commit Notes:
Concerns: The commit and PR body describe verification against a real cloud payload, but do not include the literal verification command and its captured output. That falls short of the repository’s evidence-presentation standard. I did not independently retrieve that payload, so the historical execution claim remains uncorroborated; missing corroboration alone does not establish an untruth under this lens. Attaching the original command and output would resolve that concern. This is a static HISTORY review; no runtime test result is claimed. REVIEW_PASSED |
structure lens — MISSING |
|
🎯 review-swarm: FAILED (M:pass H:pass S:missing) Lens transcripts posted as sibling comments above. |
cubic P3 on #235, and it is right. The reason string comes from the swarm's status response, which can carry agent output, which can carry content from the PR under review. Two vectors: - a line starting with `::` is parsed by Actions as a workflow command (`::error::`, `::add-mask::`) — log injection; - a line of three backticks closes the fenced block early in the step summary and the remainder renders as markup. Indent every line by four spaces instead of fencing. That defeats both at once: Actions only parses a command at the start of a line, and an indented block is a Markdown code block with no fence to break. Verified against a payload carrying both vectors: no output line starts with `::` or with a fence, and the real quota text still renders intact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
…235 Log-injection and fence-break vectors in the failure reason I added an hour earlier; one four-space indent closes both. Checked threads rather than the check status, which is the only reason it surfaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
…s full Swarms get further and hit "Exceeded the maximum database size" from the Relayfile mount, with result null and the error only at top level. #235's fallback catches that shape — verified against the real payload. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
Whitespace only. I hand-resolved a conflict between #226, #232 and #235 an hour ago and carried the surrounding block's indentation onto the line I lifted out of it, so this check sat two spaces deeper than the two it belongs with. Shell does not care; a reader does, and misaligned indentation from a merge is the first thing that makes someone wonder what else the merge got wrong. Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Co-authored-by: kjgbot <kjgbot@agentrelay.dev> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gate polls
agent-relay cloud status --json, reads.statusoff the response and discards the rest. When a swarm fails, the only thing reaching the GitHub log is the word:The reason is already in the payload the poll just fetched. For the five runs that failed on 2026-09-07 (34168392594, 34167663112, 34165035497, 34164872298, 34164770687) it was:
Nothing in the log said "quota". Finding it meant pulling the run id out of the log and querying the run by hand — so the cause went uninvestigated for days while the failures were attributed to a guess. This is the observability gap, not the quota itself.
Change: in the wait step, when the terminal status is not
completed, extract.result.error // .errorfrom the response already in hand and print it to stderr and the step summary.Verification — run against the real
04da7e48-87ec-4c7a-a1ee-22fd482e1cd1payload:{"status":"completed","result":{}}→ empty string, not the literalnull$responseis unset) → empty, no errorNot merging this myself. The rule is a passing independent signoff at the exact head plus green CI, and the review swarm is exactly what cannot run right now — the failure this PR makes visible is the same one that blocks its own review. Wants a human or a swarm once capacity is back.
🤖 Generated with Claude Code
https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR