Skip to content

fix(tests): isolate native Claude home, settle IPC contenders, select pack entries by name - #432

Merged
ScriptedAlchemy merged 2 commits into
mainfrom
fix/sweep-test-isolation
Sep 3, 2026
Merged

fix(tests): isolate native Claude home, settle IPC contenders, select pack entries by name#432
ScriptedAlchemy merged 2 commits into
mainfrom
fix/sweep-test-isolation

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Summary

Three test-isolation fixes, each fixed at source (no retries, no padding, no longer timeouts):

  1. native-claude-contract.test.ts — every enabled runNativeClaudeSmoke now receives an isolated homeDirectory (withIsolatedHome, mkdtemp + cleanup). Five tests previously ran with environment: {} / no homeDirectory, so resolveClaudeNormalHome fell back to os.homedir() and digested the developer's real ~/.claude (2.1 GB / 7,479 files on this machine) before and after every stubbed run. The "fails closed when the candidate plugin…" case runs three smokes (six digests) and timed out at 5 s wherever the claude CLI is installed.
  2. event-ipc.test.ts — the three contender-race tests (does not unlink a concurrent winner…, serializes concurrent reclamation…, excludes a second orphan-claim remover…) now attach the settlement handler to the losing server promise the moment it is created (settleContender). Root cause: the loser rejects Event runtime endpoint already has a live server. as soon as the winner listens (≤ 10 ms retry loop), while the test only reached its await secondServer after the winner had fully resolved (listen + chmod + stat + Effect layer plumbing). On a loaded machine the rejection landed first and surfaced as an unhandled, file-level failure ("fullName": "", stack in claimEndpoint) — the exact signature seen in prior pnpm test:unit runs under concurrent lanes. Endpoint ids are crypto.randomUUID() so there is no cross-run socket-path collision to isolate; the shared /tmp was only the load source. A copy of the file with a forced 200 ms gap between winner resolution and the loser await reproduced the failure deterministically before the fix and passed after it.
  3. npm pack --json entry selectionpackOutputFromJson (src/build/pack-inventory.ts + the scripts/npm-pack-json.mjs mirror) accepts a packageName and selects the entry by npm name instead of by position; without a name it still requires exactly one entry. scripts/run-packed-tests.mjs, scripts/audit-packed-release.mjs, tests/support/shared-pack.ts, tests/support/packed-native-smoke.ts, packed-consumer.test.ts, and cli.test.ts pass agent-bundle / @agent-bundle/runtime / create-agent-bundle, so a workspace-aware pack that lists sibling packages still resolves the intended tarball. Unit coverage added in prepack.test.ts (array + package-keyed shapes, name-less npm 12 objects, 0/2 match errors).

Evidence

Machine: claude on PATH (/home/zack/.local/bin/claude), other lanes running tests concurrently.

Item Before After
native-claude-contract 1 failed: fails closed…test timed out in 5000ms; file wall 16.5 s 17 passed / 1 skipped, tests 681 ms, file wall 1.8 s; 10/10 loop runs green (wall 1.82–2.07 s each)
event-ipc prior test:unit logs: file-level EventRuntimeTransportError: Event runtime endpoint already has a live server. with "fullName": "" (5 separate runs on record); deterministic repro via 200 ms gap copy → same signature 200 ms-gap copy passes; 10/10 sequential loop green; two concurrent runs of the file both green (16/16 each); 8 concurrent runs green
packed-native-smoke pnpm build && pnpm test:packed:native on this machine: 6 passed / 1 skipped (35.8 s) — the 4-entry npm pack --json output did not reproduce with npm 12.0.2 here, so the fix is by construction + unit-tested 6 passed / 1 skipped (27.4 s); 10/10 pnpm test:packed:native loop green (wall 23.7–28.0 s each, opt-in test skipped, Claude plugin proof passing); prepack.test.ts 10/10 incl. new name-selection test

The isolated "fails closed…" case alone now runs in 179 ms (-t filtered run) versus the former 5 s timeout.

Gates: pnpm typecheck ✅ (6.8 s), pnpm lint ✅ (0 errors / 0 warnings, 1161 files), pnpm test:unit ✅ 204 files, 3003 passed / 5 skipped / 0 failed (44.8 s).

Test plan

  • pnpm exec rstest --config rstest.unit.config.ts packages/agent-bundle/tests/native-claude-contract.test.ts (with claude installed) — < 2 s, green.
  • Launch packages/agent-bundle/tests/event-ipc.test.ts twice in parallel — both green.
  • pnpm build && pnpm test:packed:native — reaches harness setup, opt-in test skipped, Claude plugin proof passes.
  • AGENT_BUNDLE_WORKBENCH_PREBUILT=1 AGENT_BUNDLE_PACKAGE_PREBUILT=1 pnpm exec rstest --config rstest.integration.config.ts packages/agent-bundle/tests/prepack.test.ts — green.

Review status

  • CI: all required checks green on head c511d01cd6bbabed1bf96df3db21af27b7a90239 (Changeset present, Dependency review, Docs site, Examples check, Host install proofs, RSC runtime micro-eval, Release gates, Verify (Node 24), pkg.pr.new preview).
  • chatgpt-codex-connector: three requests (automatic on open at 16:51Z, @codex review at 16:52Z and 17:11Z) all answered "You have reached your Codex usage limits for code reviews". No review threads were opened.
  • Last codex-reviewed SHA: none. Unreviewed SHAs: 93bd86ea7 (fix), c511d01cd (changeset PR reference; head). Merged on CI green per the standing rule; a follow-up PR will answer any threads codex raises later.

… pack entries by name

- native-claude-contract: every enabled runNativeClaudeSmoke now gets an
  isolated homeDirectory, so the stubbed smoke never digests the developer's
  real ~/.claude (2 GB / 7k files here) and the "fails closed" case no longer
  times out at 5 s when the claude CLI is installed.
- event-ipc: attach the settlement handler to the losing contender the moment
  it is created. The loser rejects "already has a live server" as soon as the
  winner listens; on a loaded machine that happened before the test reached
  its await and surfaced as an unhandled file-level failure.
- packOutputFromJson (src + scripts mirror) accepts a package name and selects
  the entry by it instead of by position; run-packed-tests, the release audit,
  shared-pack, and the packed native smoke harness pass their package names so
  a workspace-aware `npm pack --json` listing sibling packages still resolves
  the intended tarball.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@changeset-bot

changeset-bot Bot commented Sep 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c511d01

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
agent-bundle Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@pkg-pr-new

pkg-pr-new Bot commented Sep 3, 2026

Copy link
Copy Markdown
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle@432
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/create-agent-bundle@432
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/@agent-bundle/runtime@432

commit: c511d01

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@ScriptedAlchemy
ScriptedAlchemy merged commit 721a0e0 into main Sep 3, 2026
13 checks passed
@ScriptedAlchemy
ScriptedAlchemy deleted the fix/sweep-test-isolation branch September 3, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant