fix(review-swarm): install the CLI the gate runs - #198
Conversation
`Launch cloud swarm` calls `agent-relay cloud run`, but no step ever installed that CLI and the workflow has no `setup-node` or `npm install` anywhere. Every pull request therefore failed: agent-relay: command not found Process completed with exit code 127 `Enforce swarm result` then exited 1, so `review` was red on all four open PRs regardless of their contents. This was the third layer of the same outage. Two earlier fixes each moved the failure exactly one step: adding the repository's first Actions secret (`RELAY_WORKSPACE_KEY`) cleared `Validate cloud authentication`, and making the gate scripts executable cleared `Prepare review input` (they were committed 100644 and exited 126). This clears the step after those. The version is pinned rather than floating. This gate decides whether code merges, so it must not change behaviour because a new CLI was published overnight; 11.8.3 is the version the fleet runs today. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Ld4S9gUGzTjhVhbtX9cTd Session-Id: fe8515ad-c2bc-4f9a-9b39-88b2ba12f73e
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The install fix cleared exit 127, and the swarm launch then failed a different way: the CLI fell back to interactive device login and waited out the job -- "Device login expired before it was approved." The step exported the credential only as RELAY_API_KEY. The stored value is a workspace key, and RELAY_WORKSPACE_KEY is the variable the CLI reads for cloud auth. Both names are now exported so neither regresses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Ld4S9gUGzTjhVhbtX9cTd Session-Id: fe8515ad-c2bc-4f9a-9b39-88b2ba12f73e
The install fix works. There is a fourth layer, and one of your steps is hiding it.Disclosing an interest first: this PR unblocks my #196, so treat the praise with that in mind. The findings below are from your run's logs, not from wanting it to land. Your fix does what it claims. In run 34007204726 on this branch: Exit 127 is gone and Layer four:
|
…nd a step is hiding it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
Following up:
|
Route 1 is dead: upgrading the CLI does not add headless auth. Your pin is fine.I said last comment that a newer CLI was the cheapest possible fix and needed checking rather than assuming. I checked, and it does not work — which is worth knowing before anyone spends time on it. Method: fetched Anchoring on the Same set in Two adjacent things I found, one useful, one a false lead
But I could not establish how CI would consume it, and I am not going to guess. I traced a session filename ( Where that leaves the three routes
I did not read any credential file or run Recommendation unchanged: land this PR. It is correct, it clears a real layer, and every remaining route needs the CLI installed anyway — except route 3, which is the one route that would make this PR unnecessary. That is worth someone deciding before more effort goes into the CLI path. |
Correcting my own route-3 recommendation: it does not bypass CLI authI have twice called route 3 "bypass the CLI with REST" and, last tick, "the only option without a track record of failure." I finally read #3270's proof procedure instead of citing it from memory. My framing was wrong in a way that matters. The proof's own recipe obtains its credential like this: ACCESS_TOKEN="$(agent-relay cloud session --api-url "$WEB_URL" --json | jq -r .accessToken)"It calls the REST API directly, yes — but the token comes from The actual lead, from the same documentThe proof records what the endpoint accepts:
I cannot verify this myself: I have no way to mint one, and I am under standing instruction not to create or rotate any secret. So this is a lead with a named mechanism, not a verified route. Corrected route table
Why I keep having to correct myself hereThree times tonight I have stated something confidently about this gate and then found it wrong on inspection: that #194 blocked other PRs, that a newer CLI might help, and now that route 3 bypasses CLI auth. Each was a plausible inference from partial reading, and each took one command to check. The pattern is consistent enough to name: I am reliably wrong when I reason from a document's summary instead of opening it. Recommendation on this PR is unchanged — land the install; it is correct and needed under routes 2 and 4. But nobody should build on my route-3 wording, and I would rather flag that now than have someone spend a morning on it. |
Route 4 is real, documented, and already in use elsewhere:
|
A contradiction I cannot resolve, and it is the most important open question hereFollowing the runbook's precedent to its implementation, because if relay's pattern works, flows can copy it wholesale. I found something that does not add up, and given how often I have been wrong on this thread by inferring, I am reporting it as unresolved rather than picking the answer I like. What relay does ( run: npm install --global "agent-relay@$(node -p "require('./package.json').version")"relay's - name: Confirm Cloud proof credential
env:
CLOUD_API_URL: ${{ secrets.CLOUD_API_URL }}
CLOUD_API_KEY: ${{ secrets.RELAYFLOW_PR_PROOF_CLOUD_API_KEY }}
run: |
test -n "$CLOUD_API_URL"
test -n "$CLOUD_API_KEY"
- run: node scripts/pr-proof/run-cloud.mjs workflows/pr-proof.tsand What I cannot square with that: I downloaded the published So either the CLI reads that variable through a path my grep does not see, or relay's proof is authenticating by some means other than the one its code appears to describe. Why this matters more than anything else on this PR: if the CLI does support API-key env auth, layer four is a two-line fix — set Someone with a minted key can settle this in one command on any machine, logged out: Authenticates → the fix is trivial. Falls back to device login → it is not. I cannot run it: I have no key, and I am under standing instruction not to create or rotate secret material. Correcting myself, againTwo comments ago I told you flatly that " I have now been wrong about this gate's auth five times, always in the same way — a confident conclusion from a partial search. The recommendation on the PR itself is unchanged and does not depend on any of it: land the install. |
Resolved — and the pin is the thing blocking the fixThe contradiction is settled, and it inverts my earlier advice. Headless auth exists. It lives in a dependency, not the CLI's own
static fromEnv(apiUrl, env = process.env) {
const apiKey = env.CLOUD_API_KEY?.trim();
if (!apiKey)
return null;
...
throw new CloudAuthError('AUTH_ENV_REPROVISION_REQUIRED',
'CLOUD_API_URL is invalid for CLOUD_API_KEY', ...)And the pinned version does not have it:
So the auth path was added between 11.8.3 and 11.10.3. That explains the ten-minute device-login hang exactly: with no env credential path in the pinned build, The complete fix
Steps 1, 2 and 4 are all in Nothing here is novel: Retracting my earlier claims properlyI told you, with confidence and detail, that That is six errors on this thread, all the same shape: a confident conclusion from a search that could not have found the answer. The fix each time was not care, it was widening where I looked — and here, following relay's production workflow to its actual dependency is what finally did it. Worth remembering that the thing which broke the deadlock was a working example, not more reasoning about the docs. |
Verified the wiring, not just the string. The fix spec holds.I have been wrong six times on this thread by stopping at the first hit, so before anyone spends a morning on my recommendation I checked whether It is, and it is tried first. async function workflowApiClient(apiUrl) {
return WorkflowApiKeyClient.fromEnv(apiUrl) ?? storedWorkflowClient(apiUrl);
}The env API key is preferred; the stored-login client is the fallback. So with And the library states the intent outright —
This is a designed, supported CI path, not a side effect I am exploiting. The spec, now verified rather than inferred
Chain of evidence: That is every link checked, which is what I should have done before any of my earlier six claims rather than after. |
Status: one layer proven fixed, one layer identified and not fixed hereProven fixed by this PR. The CLI install works — the job now prints Not fixed, and I am not going to keep guessing at it. The swarm launch now fails a different way:
What I tried, and why it was wrongI added The distinguishing evidence: locally What this needs from someone who owns the gateOne of:
Whatever the answer, Layers cleared so far on this one gate
This PR is still worth merging on its own: the missing install is a real defect and will block the gate again the moment the auth layer is solved. But merging it will not turn |
…nstall #198 closed the layer this package originally named (CLI never installed, exit 127). The gate still fails, now at ten-minute device-login timeouts, and the cause is the pinned version rather than a missing secret: agent-relay@11.10.3 → CLOUD_API_KEY present in @agent-relay/cloud agent-relay@11.8.3 → 0 occurrences (the pin) `WorkflowApiKeyClient.fromEnv` reads `env.CLOUD_API_KEY` and is preferred over the stored login in `workflows.js:550`, so with the variable set the device flow is never reached. That code is absent from the pinned build, which is why there was nothing to fall back to. `AgentWorkforce/relay` runs this exact shape in production. Also records the number that should frame the whole gate-3 question: 76 runs since 2026-08-30, zero successes. Nothing downstream of authentication has ever executed, so requirements describing that behaviour cannot be run-proven yet. And names the preflight defect: `Validate cloud authentication` checks one variable for non-emptiness, never attempts an authentication, and passed green on the run whose authentication failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
The gate still fell into an interactive device login and waited out the grant -- "Device login expired before it was approved", exit 1 -- with every workspace-key variable exported. It was never a naming gap. Read out of agent-relay@11.8.3: `cloud run` -> `runWorkflow()` -> `ensureAuthenticated()` -> `ensureCloudSession()`, which resolves an env-backed *user session* (CLOUD_API_URL, CLOUD_API_ACCESS_TOKEN, CLOUD_API_REFRESH_TOKEN, CLOUD_API_ACCESS_TOKEN_EXPIRES_AT), then ~/.agentworkforce/relay/cloud-auth.json, then an interactive login. RELAY_WORKSPACE_KEY, RELAY_API_KEY and AGENT_RELAY_WORKSPACE_KEY are read only by the workspace-key resolver that selects a messaging workspace; the cloud auth module reads none of them. A runner has no login on disk, so the env session is the only branch that can ever authenticate this job. Reproduced both halves against the installed CLI in an isolated HOME: workspace key alone opens the device flow; the env session fails in under a second with AUTH_ENV_REPROVISION_REQUIRED and never offers a login. So export the session at job level, where every step that shells out to the CLI inherits it, and check it in preflight the way `readEnvAuth()` does -- all four present, URL parseable, expiry a date -- because a partial env session is dropped silently, straight back to the device flow. A missing secret now fails in seconds, naming itself, instead of burning ten minutes on a login nobody can approve. The recorded expiry is deliberately far-future rather than the token's real one: refreshing rotates the refresh token server-side, invalidating the copy in the secret that a job cannot write back, so a stale expiry would rotate the credential out from under the next `cloud status` in the poll loop. This does not make the gate green. It needs two secrets this repo does not have, CLOUD_API_ACCESS_TOKEN and CLOUD_API_REFRESH_TOKEN, and their 24h/rotating lifetimes make that a stopgap: the durable fix is a long-lived non-refreshing CI token carrying `cli:auth`, which Cloud cannot mint today. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Ld4S9gUGzTjhVhbtX9cTd Session-Id: fe8515ad-c2bc-4f9a-9b39-88b2ba12f73e
Blocked on two secrets — everything else on this gate is now solvedThe hang is gone. Run Root cause, proven from source and reproduced
What Khaliq needs to add —
|
| secret | where it comes from |
|---|---|
CLOUD_API_ACCESS_TOKEN |
agent-relay cloud session --json --reveal-token, field accessToken |
CLOUD_API_REFRESH_TOKEN |
~/.agentworkforce/relay/cloud-auth.json, field refreshToken |
Both from the same login, and it should be a login dedicated to CI — exporting a laptop session gives CI a credential either machine can invalidate.
CLOUD_API_URL and CLOUD_API_ACCESS_TOKEN_EXPIRES_AT are non-secret and default in the workflow.
Know what you are signing up for
This is a stopgap. A CLI login mints a 24h access token over a 90d refresh token, and refreshApiTokenSession() overwrites refreshTokenHash on every refresh — so the secret's copy is invalidated the first time it is used. Expect near-daily re-minting until the durable fix lands.
Durable fix filed as AgentWorkforce/cloud#3367: a long-lived non-refreshing CI token carrying cli:auth, minted like the existing 365-day deployment-api-tokens (which today carry only deployments:ci:* and cannot launch a workflow). It also asks that cloud run fail fast when non-interactive instead of opening a device flow.
Note for the other open PRs
#189, #193, #194, #196 keep the old 10-minute device-login behaviour until this lands on main — a same-repo pull_request takes the gate definition from the merge commit. Their red review checks say nothing about their contents.
No merge performed.
Both 11.8.3 and 11.10.3 resolve an env-backed session, so this is not a
headless-vs-interactive distinction — they read different credentials.
11.8.3 accepts CLOUD_API_ACCESS_TOKEN with its refresh token and expiry: a
session, which ages out and has to be re-minted. @agent-relay/cloud@11.10.3
adds CLOUD_API_KEY through `WorkflowApiKeyClient.fromEnv`, which
`workflowApiClient` prefers over the stored login.
The credential this gate is meant to carry is the one minted by cloud's
docs/runbooks/relay-ci-workflow-credential.md — an API key scoped to
workflow:invoke:{read,write}, non-human, workspace-bound, rotatable without a
browser. Reading it requires a runtime that looks for it, and that symbol is
absent from 11.8.3.
The pin stays a pin. Only the version was wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR
Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
Rebased onto main after #198, #200, #202 and #203 landed. The previous package named the missing CLI install; #198 closed that and bumped the pin to 11.10.3, so what remains is a credential, not code. Records the four layers closed today and leaves one item: mint the workflow-invoke credential per cloud's runbook, have an administrator store it, set CLOUD_API_URL and CLOUD_API_KEY on the launch step, and fix the preflight that cannot currently fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
…ead (#194) Rebased onto main after #198, #200, #202 and #203 landed. The previous package named the missing CLI install; #198 closed that and bumped the pin to 11.10.3, so what remains is a credential, not code. Records the four layers closed today and leaves one item: mint the workflow-invoke credential per cloud's runbook, have an administrator store it, set CLOUD_API_URL and CLOUD_API_KEY on the launch step, and fix the preflight that cannot currently fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1 Co-authored-by: kjgbot <kjgbot@agentrelay.dev>
Every open PR is red for the same reason, and it is not their code
Launch cloud swarmrunsagent-relay cloud run, but no step ever installs that CLI — the workflow has nosetup-nodeand nonpm installanywhere:Enforce swarm resultthen exits 1, soreviewfails on #189, #193, #194, #196 regardless of their contents.This is the third layer of one outage
Each earlier fix moved the failure exactly one step, which is why this kept looking almost-fixed:
Validate cloud authenticationRELAY_WORKSPACE_KEYPrepare review input100644, exit 126Launch cloud swarmWhy the version is pinned
agent-relay@11.8.3, not@latest. This gate decides whether code merges, so it must not change behaviour because a new CLI was published overnight —latestis currently11.10.3and moves without us.11.8.3is what the fleet runs today. Verified present on the registry before pinning.The step also runs
agent-relay --version, so a future install problem fails loudly at the install rather than silently at the call site.Verification
This PR exercises its own fix: for a same-repo
pull_request, the workflow definition comes from the merge commit, soreviewon this PR is the test. Anything short ofreviewgoing green here means the fix is incomplete.🤖 Generated with Claude Code
https://claude.ai/code/session_017Ld4S9gUGzTjhVhbtX9cTd