Skip to content

fix(workflows): build-1 preps kernel + fixtures inside its own sandbox (replaces #64) - #65

Closed
kjgbot wants to merge 1 commit into
mainfrom
fix/build-1-preps-kernel-and-fixtures
Closed

fix(workflows): build-1 preps kernel + fixtures inside its own sandbox (replaces #64)#65
kjgbot wants to merge 1 commit into
mainfrom
fix/build-1-preps-kernel-and-fixtures

Conversation

@kjgbot

@kjgbot kjgbot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Codex previously rejected #64 as history lens for exactly the right reason: cloud steps don't share state. This fix moves the prep INTO the build-1 agent's task prompt — same sandbox as npm test.

Ref: ops/BACKLOG.md:686-707

Prep prepended to build-1's task:

  1. chmod +x ops/cargo.sh
  2. cd kernel && sh ../ops/cargo.sh build
  3. chmod +x testdata/preflight/*-cli

verify-1 already uses this same in-sandbox pattern (ops/cargo.sh build inside its own step).

PR #64 tried to solve this via a pre-build step but was correctly
rejected — cloud workflow steps run in isolated sandboxes (ops/BACKLOG.md:686-707).
This puts the prep inside build-1's task prompt so it runs in the same
sandbox as npm test.

The prep: chmod +x ops/cargo.sh, build kernel, restore +x on
testdata/preflight/*-cli fixtures. Same pattern verify-1 already uses
inside its own sandbox.
@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 35 minutes.

View limit details

Limit details: You’ve used the included review currently available.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 399808a2-db3e-4da7-9e07-b82c9832ed34

📥 Commits

Reviewing files that changed from the base of the PR and between 7369f55 and fde9287.

📒 Files selected for processing (2)
  • workflows/drive-cloud.yaml
  • workflows/drive.yaml

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

@kjgbot

kjgbot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

maintainability lens — FAIL

Maintainability review — PR #65

Blocker — the prompt comment asserts a fact ops/cargo.sh actively contradicts.
drive-cloud.yaml:183 and drive.yaml:277 add the justification "SDK live-kernel tests exec kernel/target/debug/relayflowd". But ops/cargo.sh:42-52 explicitly redirects builds to $RELAYFLOWS_TOOLCHAIN_HOME/target/<worktree-hash>/debug/relayflowd via CARGO_TARGET_DIR, and the comment at cargo.sh:42 says the opposite rule in force: "Anything that hardcodes kernel/target/debug must read RELAYFLOWD_BIN instead." The prompt teaches the agent the wrong invariant. If a future maintainer trusts the comment and hard-codes the path anywhere downstream, they'll regress the worktree isolation PR #38 fixed. Either drop the parenthetical entirely or restate it as "the SDK tests run $RELAYFLOWD_BIN, which is populated by ops/cargo.sh."

Concern — duplicated setup logic with no forcing function to keep it in sync.
The three preamble steps (chmod cargo.sh, sh ../ops/cargo.sh build, chmod *-cli) are already implemented and thoroughly justified in verify-1 (drive-cloud.yaml:210-236 and drive.yaml:322-340). This PR copies the what into the prompt but strips the why (run 909e18f6, 143→150 pass counts, materialization-drops-exec-bit context). A six-month reader editing verify's chmod step won't know a prompt mirrors it; a reader editing the prompt won't know verify already does the same thing. This is the classic "two places to change, one obvious." Better shape: extract a single ops/preflight.sh (or reuse a subset of the verify block) and reference it from both the builder task and verify.

Concern — silent failure swallow in a step whose failure the summary is supposed to prove.
chmod +x ops/cargo.sh || true and find … 2>/dev/null || true hide the exact failure the "paste the last 5 lines" instruction is meant to expose. If chmod fails for a reason other than "already set", or if testdata/preflight is missing, the agent prints nothing distinguishable from success. Drop || true, or write an explicit echo PREFLIGHT_OK/PREFLIGHT_SKIP so the pasted output is falsifiable.

Concern — instruction lives only in prompt text; no gate enforces it.
Nothing verifies the agent actually ran the preamble or pasted the output. The output_contains: BUILD_DONE verification (drive-cloud.yaml:187, drive.yaml:284) accepts any transcript ending with the token. A test that would fail if the behavior broke does not exist here — the whole change is a soft "please do this."

Note — the two workflow files must be edited in lockstep by hand.
This PR touches both, correctly. But there is no check that drive.yaml and drive-cloud.yaml stay aligned; ops/gen-drive-cloud.py exists (per ls) and presumably generates one from the other. Was drive-cloud.yaml hand-edited here, or regenerated? If hand-edited, next gen-drive-cloud.py run will silently undo this change.

REVIEW_FAILED

@kjgbot

kjgbot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

history lens — FAIL

Blocker

  • workflows/drive.yaml:277-281 and workflows/drive-cloud.yaml:182-186 make fixture repair depend on shell-directory persistence. If the numbered commands run as separate agent tool calls, command 2’s cd kernel does not persist; command 3 therefore begins at the repository root, executes cd .., and searches the wrong tree. Its || true then hides the miss. This can reproduce the recorded EACCES failures instead of fixing them. Use root-stable commands such as (cd kernel && sh ../ops/cargo.sh build) followed by find testdata/preflight ..., with an explicit executable-bit assertion. This ordering is known to be load-bearing (ops/DRIVE-LOG.md:2993-3008).

Concerns

  • The same ranges require “the last 5 lines” from chmod and find, although successful executions normally emit nothing. Combined with 2>/dev/null || true, the summary cannot distinguish successful repair from a silent no-op. That repeats the repository’s recorded evidence problem: claims must carry reproducible captured output. Emit explicit status and fail if expected files remain non-executable.
  • The commit message is accurate about moving preparation into build-1’s own sandbox, but overstates that the diff “restore[s] +x” because the fixture command can target the wrong directory and suppress failure.

Notes

REVIEW_FAILED

@kjgbot

kjgbot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

structure lens — PASS

→ Read docs/RFC-0001-everything-is-a-relayflow.md
→ Read AGENTS.md
→ Read workflows/drive.yaml
→ Read workflows/drive-cloud.yaml

Structure review — PR #65

The change touches only two orchestration YAML files, adding a three-command "preflight" preamble to the build step's agent task in both workflows/drive.yaml (lines 276–286) and workflows/drive-cloud.yaml (lines 182–185). No kernel code, no new primitive, no RFC vocabulary drift — so no constitutional blocker exists here.

Blockers

None.

Concerns

1. Product/ordering logic duplicated between prose and a check (drive.yaml:276-282 vs :335,341,365-371).
The three new commands (chmod +x ops/cargo.sh, cd kernel && sh ../ops/cargo.sh build, chmod the testdata/preflight clis) re-encode, as agent prose, the exact build-before-test ordering that the deterministic verify step already enforces as shell — sh ../ops/cargo.sh build at line 335, and the node_modules exec-bit repair at lines 365–371. The repo's own argument is on file: "prose does not hold and a check does" (drive.yaml:388). This PR adds back a prose copy of an invariant the check already holds, so the fix now lives in two places and can drift. If the goal is to make the builder's DoD run in its own sandbox, that's a separate environment from verify — and the divergence ("run npm test yourself") means the builder now also self-judges its own pass, which the independent verify gate is supposed to be. At minimum, the preamble should point at, not duplicate, the truth.

2. drive-cloud.yaml is generated (drive-cloud.yaml:1-2).
The header says "Do not hand-edit: change drive.yaml, then regenerate." Divergence of the two task strings (the cloud variant carries chmod wrapped in || true/2>/dev/null, testdata/preflight referenced but absent from the sync required-paths list) suggests regeneration, but this should be confirmed against ops/gen-drive-cloud.py, not hand-edited in parallel.

Notes

  • File size: drive.yaml is 598 lines and grows here; AGENTS.md standard gate1: kernel + sdk skeletons (bootstrap relayflow output) #1 flags ~500 lines as a smell. A workflow DAG is arguably not a "module," but the file is a candidate for splitting steps into workflows/gates/ per RFC §2 rule 1.
  • Fail-closed posture: || true here is acceptable — it's best-effort fixture prep, not a journal write — but it should stay out of any completion-critical path.

Verdict: the diff is structurally sound as a prompt edit, but it reinstates the prose-over-check anti-pattern the repo has already paid to eliminate.

REVIEW_PASSED

@kjgbot

kjgbot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

🎯 review-swarm: FAILED (M:fail H:fail S:pass)

Lens transcripts posted as sibling comments above.

@kjgbot

kjgbot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Superseding — swarm found two real issues: (1) prompt taught hardcoded kernel/target/debug/ path that ops/cargo.sh:42-52 explicitly redirects via CARGO_TARGET_DIR (PR #38 fixed); (2) cd's don't persist across agent tool calls, so 'cd .. && find testdata/preflight' hits wrong dir. Retrying with subshells + correct RELAYFLOWD_BIN reference + no failure-hiding || true.

@kjgbot kjgbot closed this Aug 30, 2026
@kjgbot
kjgbot deleted the fix/build-1-preps-kernel-and-fixtures branch August 30, 2026 21:07
kjgbot added a commit that referenced this pull request Aug 30, 2026
… drive loop, replaces #64/#65/#68) (#69)

* fix(sdk): build kernel + restore fixture +x before npm test (unblocks drive loop)

Root cause the drive loop has been stalling on since PR #63 (2026-08-30):
sdk/tests/live-kernel.test.ts requires a built relayflowd binary
(ops/cargo.sh's toolchain-external target), and the cloud sandbox
does not build it before running npm test. Result: the assessor observes
red tests, writes ops/NEEDS_HUMAN.md, assess-gate parks the run, and no
build/verify step ever fires.

Fix moves the prep INSIDE npm test itself so it's correct for every
caller — the drive assessor, drive builder, verify step, and humans on
a laptop.

Superseded PRs (all closed):
- #64: separate pre-build step — steps have per-step sandboxes, prep
  invisible to build (BACKLOG.md:688-707)
- #65, #68: prep in build's task prompt — assess parks first, so build
  never runs

This lands the prep at a lower layer that all three stages hit.

* fix(sdk): fail-closed on fixture chmod (address swarm blocker on #69)

Both lenses on #69 correctly flagged `2>/dev/null || true` as
failure-hiding: fixtures are tracked, chmod errors should surface.

Replaced with `[ ! -d ../testdata/preflight ] || find ...` — tolerant
of an absent dir, but any real error (missing find, permission denied,
typo in flag) fails test:prep loudly. Maintainability lens's exact
recommendation.

---------

Co-authored-by: kjgbot <kjgbot@agentrelay.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant