Skip to content

ci: stop losing releases to things that are allowed to fail - #121

Merged
filipeforattini merged 2 commits into
mainfrom
fix/pipeline-hermetic-and-retries
Sep 5, 2026
Merged

ci: stop losing releases to things that are allowed to fail#121
filipeforattini merged 2 commits into
mainfrom
fix/pipeline-hermetic-and-retries

Conversation

@filipeforattini

@filipeforattini filipeforattini commented Sep 5, 2026

Copy link
Copy Markdown

The number

Nine of the last twenty-three test.yml runs failed — 39% — and 0.15.0 has been sitting unreleased
through five of them.

That sounds like chaos and is not. Most of today's failures were real defects CI had never been
able to see
until #108 turned on the suites it had been skipping since the rename: Windows path
and CRLF assumptions, a component that mishandled mutation-record ordering, a test racing its own
request. Those are fixed. What is left is a short list of external dependencies that fail a few
percent of the time and take a release with them each time.

What this changes

bun install is retried on every platform, not only Windows. The action already retried there
for a known bun issue; Linux and macOS got one attempt. That asymmetry is exactly where the 0.14.0
publish died — Failed to install 1 package on the darwin-arm64 sidecar. This composite action runs
in every job of every workflow, so one transient refusal anywhere costs a build.

playwright install-deps chromium is apt against Ubuntu's mirrors. It ran on every Linux e2e
run, gated only on the OS, and when the mirrors refused (Connection failed [IP: 91.189.92.22 80])
the job died before a test executed. It now runs only when the browser cache missed — the deps come
with the cache — and retries when it does run.

Spawned CLI processes get the file-watcher kill switch. The unit job sets
REDCODE_EXPERIMENTAL_DISABLE_FILEWATCHER on Windows, but cli-process.ts builds a replacement
environment, so every cliIt.live test started a redcode that loaded @parcel/watcher anyway —
the module Bun segfaulted in. A test pins it, because the failure mode is silent.

packages/core tests stop writing to the real home directory. global.ts resolves the app home
at module scope from REDCODE_TEST_HOME ?? os.homedir(), and core's preload — five lines — never set
it. So every core run created ~/.red/redcode on the runner, shared one binary cache across
concurrent jobs, and let an ambient ~/.npmrc reach the installer under test. The preload now
isolates home and XDG and clears provider keys, the way packages/redcode's already does.

One deliberate wrinkle: the isolated home is shared across core test processes rather than per-pid.
Global.Path.bin hangs off it and is where ripgrep lands when the machine has none on PATH; per-pid
would isolate perfectly and make every suite download it again.

packages/core runs at 30 s per test. It was on bun's 5 s default — the only heavyweight package
still there — while spawning git, spawning node, and resolving ripgrep across 149 files on a
shared 4-vCPU runner.

Deliberately not done

No blanket retry on the test jobs. Retrying setup steps is right: they are I/O against someone
else's server. Retrying bun turbo test would have hidden every real defect found today.

Verification

packages/core 1143 pass with the isolated home; packages/redcode/test/cli 378 pass with the
watcher off in subprocesses. The workflow changes prove themselves over the next runs — that is what
a flake fix is.

https://claude.ai/code/session_014XwJPDhq1ahcm3rd454WzQ


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

filipeforattini and others added 2 commits September 5, 2026 02:45
Nine of the last twenty-three test runs failed. Most were real defects CI had
never been able to see until #108 turned the suites on, and those are fixed.
These are the rest — each one tied to a failure that took a release with it.

- `bun install` retried on every platform, not only Windows. A registry that
  answered "Failed to install 1 package" once is why 0.14.0 did not publish, and
  this action runs in every job of every workflow.
- `playwright install-deps` is apt against Ubuntu's mirrors. It now runs only
  when the browser cache missed, and retries when it does run.
- Spawned CLI processes get the file-watcher kill switch. The environment there
  replaces the parent's, so the flag CI sets never arrived, and Bun segfaulted
  inside that watcher on a Windows runner.
- `packages/core` tests no longer write to the real home directory. They were
  creating `~/.red/redcode`, sharing a binary cache across concurrent jobs, and
  letting an ambient `~/.npmrc` reach the installer under test. The binary cache
  stays shared across processes on purpose: per-pid would isolate correctly and
  make every suite re-download ripgrep.
- `packages/core` runs at 30s per test like the other heavy packages, instead of
  bun's 5s default while it spawns git and node.

Claude-Session: https://claude.ai/code/session_014XwJPDhq1ahcm3rd454WzQ
@filipeforattini
filipeforattini merged commit d9538c9 into main Sep 5, 2026
7 checks passed
@filipeforattini
filipeforattini deleted the fix/pipeline-hermetic-and-retries branch September 5, 2026 06:20
filipeforattini added a commit that referenced this pull request Sep 5, 2026
Ported from anomalyco/opencode 03cb632.

Port note: our preload had been rewritten for hermetic homes (#121), so the one line went
in by hand beside the other network switches; the new test came across as written.

Upstream: 03cb632
filipeforattini added a commit that referenced this pull request Sep 5, 2026
Ported from anomalyco/opencode 03cb632.

Port note: our preload had been rewritten for hermetic homes (#121), so the one line went
in by hand beside the other network switches; the new test came across as written.

Upstream: 03cb632
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant