test(sdk): capture the child stderr these tests deliberately provoke (#156) - #190
Conversation
#156 reported an intermittent `ENOENT ... .relayflow/backlog-picker-entry .json` and read it as a flake caused by a relative path racing under vitest parallelism. I could not reproduce a failure, and the stated cause does not hold: every caller of the flow's steps already runs in its own `mkdtempSync` directory with an explicit `cwd`. What is actually happening is simpler and does not fail anything. Several tests here assert a step FAILS -- `select-entry` refusing an unactionable backlog, `emit-package` with no entry to read. `execFileSync` echoes a failing child's stderr into the parent's, so a completely green run prints a full ENOENT stack trace and looks broken. Measured over six consecutive full suites on main before changing anything: 662 passed, 3 skipped, every time -- and the ENOENT text present every time. Never a failure, always the noise. So the defect is the noise, because it is what got misdiagnosed. Child stderr is now captured rather than echoed. It remains on the thrown error for any test that wants to assert on it. with the change, three consecutive runs: Tests 662 passed | 3 skipped (665) | ENOENT lines: 0 Tests 662 passed | 3 skipped (665) | ENOENT lines: 0 Tests 662 passed | 3 skipped (665) | ENOENT lines: 0 with the `stdio` line reverted: Tests 662 passed | 3 skipped (665) | ENOENT lines: 2 Tests pass either way, which is exactly why this was read as a flake: the only signal was output that looked like a failure and was not. I have NOT closed #156 as invalid. Its first run genuinely showed `Test Files 1 failed`, so something failed then; the suite has changed substantially since (#153 widened it from four files to all of them, and several fixes have landed). I cannot reproduce it, and I would rather leave that stated than declare a bug I never saw to be absent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
|
Warning Review limit reachedNext included review available in 23 minutes. View limit detailsLimit details: You’ve used the included review currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (2)
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing. Comment |
…d but left open Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1
Addresses #156 — but not as filed.
The reported cause does not hold
#156 read the intermittent
ENOENT ... .relayflow/backlog-picker-entry.jsonas a relative path racing under vitest parallelism. Every caller of the flow's steps already runs in its ownmkdtempSyncdirectory with an explicitcwd—backlog-picker.test.ts,backlog-picker-flow.test.ts,work-package-consumer.test.ts. There is no shared working directory to race over.What is actually happening
Several tests here assert a step fails —
select-entryrefusing an unactionable backlog,emit-packagewith no entry to read.execFileSyncechoes a failing child's stderr into the parent's, so a completely green run prints a full ENOENT stack trace and looks broken.Measured over six consecutive full suites on main before changing anything:
662 passed, 3 skippedevery time, and the ENOENT text present every time. Never a failure, always the noise.So the defect is the noise
It is what got misdiagnosed, and it will misdiagnose the next person reading a CI log. Child stderr is now captured rather than echoed; the text remains on the thrown error for any test that wants to assert on it.
662 passed | 3 skipped— ENOENT lines: 0 each timestdioline reverted662 passed | 3 skipped— ENOENT lines: 2Tests pass either way. That is exactly why this was read as a flake: the only signal was output that looked like a failure and was not.
What I am not claiming
I have not closed #156 as invalid. Its first run genuinely showed
Test Files 1 failed, so something failed then. The suite has changed substantially since (#153 widened it from four files to all of them, and several fixes have landed). I could not reproduce it in six runs, and I would rather say that than declare a bug I never observed to be absent.Signoff: local 3-lens preswarm, maintainability / history / structure all REVIEW_PASSED.
🤖 Generated with Claude Code
https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR