From 862ddb80adceb5a0cf65926841e8bfe134b913a2 Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:13:38 +0200 Subject: [PATCH 1/8] fix(review-swarm): make the auth gate actually validate, and fingerprint the key "Validate cloud authentication" asserted only that two variables were non-empty. On 2026-09-07 it passed on every run while `agent-relay cloud run` failed one step later with `Workflow prepare failed: 401 Unauthorized` -- six PRs, repeatedly, behind a green check. A gate named for validation that tests presence is worse than no gate: it reads as cleared. Two changes: 1. Probe /api/v1/workflows/runs with the key, against the same host the CLI uses. That route requires a RESOLVED WORKSPACE and returns 401 for a fabricated or absent token -- verified against production before relying on it, because the previous attempt at this check probed /auth/whoami, which passes `allowMissingWorkspace: true` and therefore certifies tokens that cannot launch anything. 2. Print a non-reversible fingerprint, sha256 first 12 chars. The fingerprint exists because of a question I could not answer tonight. The mint proved the credential works (200 from that same route, from a GitHub runner, 90 seconds earlier) and the swarm's launch still 401'd. Production worker logs confirmed the token reaching /workflows/prepare resolved to no api_token_sessions row at all. Whether CI is using the credential the mint installed is unanswerable from outside, because the value is masked everywhere it appears. A fingerprint on both ends settles it in one line. The error tells the operator to re-mint rather than re-run, because re-running was the wrong instinct all evening: a stale credential does not become valid on retry. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR --- .github/workflows/review-swarm.yml | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/.github/workflows/review-swarm.yml b/.github/workflows/review-swarm.yml index 852f186c..5f2761a0 100644 --- a/.github/workflows/review-swarm.yml +++ b/.github/workflows/review-swarm.yml @@ -55,7 +55,31 @@ jobs: run: | test -n "$CLOUD_API_URL" test -n "$CLOUD_API_KEY" - echo "CLOUD_API_URL and CLOUD_API_KEY present; interactive login is unreachable from here." + # Presence is not validity. This step was named "Validate cloud + # authentication" while only asserting the variables were non-empty, so on + # 2026-09-07 it passed on every run while `agent-relay cloud run` failed + # immediately after with `Workflow prepare failed: 401 Unauthorized` -- + # six PRs, repeatedly, behind a green check. + # + # Actually exercise the credential against the same host the CLI will use. + # /api/v1/workflows/runs requires a RESOLVED WORKSPACE and returns 401 for + # a fabricated or absent token (verified against production), so 200 here + # means the credential can genuinely act, not merely that a string was set. + # + # Also print a NON-REVERSIBLE fingerprint of the key. When this check + # passes and the launch still 401s, the fingerprint answers whether CI is + # even using the credential the mint installed -- otherwise unanswerable + # from outside, because the value is masked everywhere it appears. + fp="$(printf '%s' "$CLOUD_API_KEY" | shasum -a 256 | cut -c1-12)" + echo "CLOUD_API_KEY fingerprint (sha256, first 12): $fp" + status="$(curl -s -o /dev/null -w '%{http_code}' \ + -H "Authorization: Bearer $CLOUD_API_KEY" \ + "${CLOUD_API_URL%/}/api/v1/workflows/runs" || echo 000)" + if [ "$status" != "200" ]; then + echo "::error::CLOUD_API_KEY is set but not accepted by $CLOUD_API_URL (HTTP $status). Re-mint the credential; do not re-run this job." >&2 + exit 1 + fi + echo "CLOUD_API_KEY authenticates against $CLOUD_API_URL; interactive login is unreachable from here." # `agent-relay cloud run` launches the swarm, but nothing installed the # CLI, so this job failed at `Launch cloud swarm` with From 1a43d2ff5865f5094983175d139247718bcae28c Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:37:49 +0200 Subject: [PATCH 2/8] chore: compare CI fingerprint against the mint's f034ce43f5da Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR From ed92ad251c905ae930d71715b3063f17915e21d8 Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:41:29 +0200 Subject: [PATCH 3/8] chore: second fingerprint read to separate propagation lag from a stuck secret Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR From c1bccd5681730365731787f1c58e6d278c57f168 Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:43:07 +0200 Subject: [PATCH 4/8] =?UTF-8?q?temp(diag):=20canary=20secret=20probe=20?= =?UTF-8?q?=E2=80=94=20distinguishes=20stuck=20secret=20from=20no=20propag?= =?UTF-8?q?ation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR --- .github/workflows/review-swarm.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/review-swarm.yml b/.github/workflows/review-swarm.yml index 5f2761a0..dee2931f 100644 --- a/.github/workflows/review-swarm.yml +++ b/.github/workflows/review-swarm.yml @@ -26,6 +26,7 @@ jobs: env: CLOUD_API_URL: ${{ vars.CLOUD_API_URL || 'https://agentrelay.com/cloud' }} CLOUD_API_KEY: ${{ secrets.CLOUD_API_KEY }} + CLOUD_API_KEY_CANARY: ${{ secrets.CLOUD_API_KEY_CANARY }} RELAY_WORKSPACE_KEY: ${{ secrets.RELAY_WORKSPACE_KEY }} RELAY_API_KEY: ${{ secrets.RELAY_WORKSPACE_KEY }} steps: @@ -71,6 +72,9 @@ jobs: # even using the credential the mint installed -- otherwise unanswerable # from outside, because the value is masked everywhere it appears. fp="$(printf '%s' "$CLOUD_API_KEY" | shasum -a 256 | cut -c1-12)" + # TEMPORARY canary. Written at a known value to distinguish "secret + # writes never reach runners" from "this one secret is stuck". + echo "CANARY fingerprint: $(printf '%s' "$CLOUD_API_KEY_CANARY" | shasum -a 256 | cut -c1-12)" echo "CLOUD_API_KEY fingerprint (sha256, first 12): $fp" status="$(curl -s -o /dev/null -w '%{http_code}' \ -H "Authorization: Bearer $CLOUD_API_KEY" \ From 3d7e31ee81b2bb365d3b33410eca8cec6e413c86 Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:44:26 +0200 Subject: [PATCH 5/8] chore: read CLOUD_API_KEY after writing a known probe value Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR From d7df77cc555227c392c645c59b188f1b0d240ba7 Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:53:22 +0200 Subject: [PATCH 6/8] chore: verify CI now receives the minted credential (expect 91f0d17f1360) Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR From b9d030bd5fefb3834a1c8abaad348a16934c22df Mon Sep 17 00:00:00 2001 From: kjgbot Date: Mon, 7 Sep 2026 23:58:37 +0200 Subject: [PATCH 7/8] chore: remove the temporary canary probe It did its job. Writing a secret whose expected fingerprint I knew in advance proved secret writes DO reach runners instantly, which eliminated propagation lag and pointed at the writer -- `gh secret set --body -` storing the literal string "-" (cloud#3433). The permanent probe and the CLOUD_API_KEY fingerprint stay. Confirmed working: mint installed 91f0d17f1360 and CI received 91f0d17f1360, exact match, with "CLOUD_API_KEY authenticates against https://agentrelay.com/cloud". Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR --- .github/workflows/review-swarm.yml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/.github/workflows/review-swarm.yml b/.github/workflows/review-swarm.yml index dee2931f..5f2761a0 100644 --- a/.github/workflows/review-swarm.yml +++ b/.github/workflows/review-swarm.yml @@ -26,7 +26,6 @@ jobs: env: CLOUD_API_URL: ${{ vars.CLOUD_API_URL || 'https://agentrelay.com/cloud' }} CLOUD_API_KEY: ${{ secrets.CLOUD_API_KEY }} - CLOUD_API_KEY_CANARY: ${{ secrets.CLOUD_API_KEY_CANARY }} RELAY_WORKSPACE_KEY: ${{ secrets.RELAY_WORKSPACE_KEY }} RELAY_API_KEY: ${{ secrets.RELAY_WORKSPACE_KEY }} steps: @@ -72,9 +71,6 @@ jobs: # even using the credential the mint installed -- otherwise unanswerable # from outside, because the value is masked everywhere it appears. fp="$(printf '%s' "$CLOUD_API_KEY" | shasum -a 256 | cut -c1-12)" - # TEMPORARY canary. Written at a known value to distinguish "secret - # writes never reach runners" from "this one secret is stuck". - echo "CANARY fingerprint: $(printf '%s' "$CLOUD_API_KEY_CANARY" | shasum -a 256 | cut -c1-12)" echo "CLOUD_API_KEY fingerprint (sha256, first 12): $fp" status="$(curl -s -o /dev/null -w '%{http_code}' \ -H "Authorization: Bearer $CLOUD_API_KEY" \ From 2a1ffca2290db0358a04f8a66e38540af0aef494 Mon Sep 17 00:00:00 2001 From: kjgbot Date: Tue, 8 Sep 2026 04:42:32 +0200 Subject: [PATCH 8/8] fix(review-swarm): bound the auth probe and stop blaming the credential cubic P2 on #232, confidence 10, and right on all three counts. The probe had no timeout, so an unreachable Cloud left curl waiting until the 75-minute job timeout. Worse than the wait was the verdict: `|| echo 000` fed a transport failure into the same non-200 branch as a rejection, and the single error message told a maintainer to re-mint a credential that was never the problem. A gate that misattributes its own failure is how the last outage cost a day. Bound the request (--connect-timeout 10 --max-time 30) and split the three outcomes: a curl failure is a transport verdict, 401/403 is the credential, and any other status is Cloud being unhealthy. Each says which of the three it is and what to do about it. Verified: all six arms dispatch correctly (transport / 200 / 401 / 403 / 502 / 429), and curl against a blackhole address returns rc=28 in ~3s rather than hanging. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR --- .github/workflows/review-swarm.yml | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/.github/workflows/review-swarm.yml b/.github/workflows/review-swarm.yml index 5f2761a0..88ce4a4f 100644 --- a/.github/workflows/review-swarm.yml +++ b/.github/workflows/review-swarm.yml @@ -72,13 +72,31 @@ jobs: # from outside, because the value is masked everywhere it appears. fp="$(printf '%s' "$CLOUD_API_KEY" | shasum -a 256 | cut -c1-12)" echo "CLOUD_API_KEY fingerprint (sha256, first 12): $fp" - status="$(curl -s -o /dev/null -w '%{http_code}' \ + # Bound the request and keep the three outcomes apart. Unbounded, an + # unreachable Cloud leaves curl waiting until the 75-minute job + # timeout; the `|| echo 000` then produced a non-200 and the one + # error message told a maintainer to re-mint a credential that was + # never the problem. A transport failure, an auth rejection and an + # unhealthy Cloud are three different diagnoses and must not share a + # sentence. + if ! status="$(curl --connect-timeout 10 --max-time 30 \ + -s -o /dev/null -w '%{http_code}' \ -H "Authorization: Bearer $CLOUD_API_KEY" \ - "${CLOUD_API_URL%/}/api/v1/workflows/runs" || echo 000)" - if [ "$status" != "200" ]; then - echo "::error::CLOUD_API_KEY is set but not accepted by $CLOUD_API_URL (HTTP $status). Re-mint the credential; do not re-run this job." >&2 + "${CLOUD_API_URL%/}/api/v1/workflows/runs")"; then + echo "::error::could not reach $CLOUD_API_URL to validate CLOUD_API_KEY (curl transport failure or timeout). This is not a credential verdict — re-run once Cloud is reachable." >&2 exit 1 fi + case "$status" in + 200) ;; + 401|403) + echo "::error::CLOUD_API_KEY is set but rejected by $CLOUD_API_URL (HTTP $status). Re-mint the credential; do not re-run this job." >&2 + exit 1 + ;; + *) + echo "::error::$CLOUD_API_URL returned HTTP $status while validating CLOUD_API_KEY. That is not an authentication verdict — treat it as Cloud being unhealthy rather than the credential being bad." >&2 + exit 1 + ;; + esac echo "CLOUD_API_KEY authenticates against $CLOUD_API_URL; interactive login is unreachable from here." # `agent-relay cloud run` launches the swarm, but nothing installed the