CI speed: PR runner-minutes cut ~47% (trimmed Verify matrix + docs-only skip) — before/after and remaining ideas

[ScriptedAlchemy]

Landed in #232 (merge 82c44352b). Measured baseline from 8 sampled runs on 2026-09-01 (per-job/per-step timings via the Actions API):

Before (median compute per job)

Job	Median	Dominant step
Verify (Node 22.19.0)	6.8 min	pnpm test ~6.2 min
Verify (Node 24)	6.6 min	pnpm test ~5.9 min
Verify (Node 26)	6.8 min	pnpm test ~6.0 min
Examples check	3.9 min	pnpm examples:check ~3.5 min
Release gates	3.0 min	pnpm check:release:ci ~2.7 min
RSC runtime micro-eval	38 s	pnpm eval:spot ~18 s
Dependency review	8 s	—

PR total: ~28 runner-minutes; wall 8.1–15.3 min (median 12.7). Main pushes: same jobs but 20–27 min runner-pool queue before the first job (28–47 min wall for ~7 min of work) — the queue is starved by total fleet runner-minutes, and PRs' 3-wide Verify matrix was ~75% of each PR's cost.

After (from #232's own hosted runs)

Job	Measured
Detect changed paths (new)	5 s
Verify (Node 24) — only PR leg	6.8–6.9 min
Examples check	3.5–3.9 min
Release gates	3.3 min
RSC runtime micro-eval	42 s
Dependency review	8 s

PR total: ~15 runner-minutes (−47%). Pushes to main and workflow_dispatch still run the full 22.19/24/26 matrix — main coverage unchanged. Docs-only PRs (docs/, agent-patterns/, .changeset/*.md, top-level *.md; renames count both paths; fail-open on any API doubt or truncated listing) additionally skip Verify/examples/release-gates/micro-eval, dropping a docs-only PR to well under 1 runner-minute.

Deliberately not done (data said no)

• pnpm store / Playwright-Chrome / build-output caching: already solved (pnpm/setup cache ~3s installs, runner-image Chrome <5s, pnpm build ~10s).
• Test sharding: pnpm test is ~6 min but splitting adds job-count overhead to an already starved runner pool.
• Main-push concurrency cancellation: deliberately SHA-keyed no-cancel; every main commit stays verified.

Remaining ideas

• Main queue starvation is the dominant wall-clock cost on main; the PR trim frees ~13 runner-min/PR of shared supply, but if starvation persists, the next lever is examining runner concurrency limits or a larger runner pool — not workflow logic.
• packages/workbench/tests/examples-real.e2e.test.ts stale-diagnostic flake keeps biting under hosted load (bit ci: trim PR Verify matrix to Node 24 and skip heavy jobs on docs-only PRs #232's first run); related to Flake: overview.e2e owned-reload generation budget exceeded under hosted-runner load #200/flake: track remaining dev-watcher and MCP App gate failures #122.
• main is red: #231 Agent Document stage broke runtime-inspector and public-api declaration tests #240 tracks the (unrelated) deterministic main breakage from feat(workbench): Agent Document stage over server-decoded render events (#105 stage 2) #231 that was red during this landing.

[ScriptedAlchemy]

Follow-ups landed in #259 (9a046d963)

Implemented

• Docs-only classifier extracted + fixture-tested (scripts/classify-docs-only.mjs, packages/agent-bundle/tests/classify-docs-only.test.ts). Expected savings: 0 runner-minutes (correctness). This is the remaining quality gap from ci: trim PR Verify matrix to Node 24 and skip heavy jobs on docs-only PRs #232: the allowlist, rename-source classification, and fail-open listing checks are no longer untested inline bash, so a classifier bug cannot skip Verify. The changes job sparse-checkouts only that script (filter: blob:none) so the vendored Effect subtree stays off the classify critical path.
• examples-real.e2e stale-diagnostic journeys now use { retry: 2 } (capability repair, hooks/diagnostics repair, audiobook catalog stale/repair) — same interim as Flake: overview.e2e owned-reload generation budget exceeded under hosted-runner load #200 / the flake: track remaining dev-watcher and MCP App gate failures #122 family. Expected savings: avoid a wasted ~7 min Verify rerun when the hosted-load flake hits. Root-cause coalescing stays on flake: track remaining dev-watcher and MCP App gate failures #122 / Flake: overview.e2e owned-reload generation budget exceeded under hosted-runner load #200.

Skipped (per this issue's cost/benefit notes)

• pnpm store / Playwright-Chrome / build-output caching — already solved (pnpm/setup cache ~3s installs, runner-image Chrome <5s, pnpm build ~10s). Not worth more workflow complexity.
• Test sharding — would add job-count overhead on an already starved runner pool.
• Main-push concurrency cancellation — deliberately SHA-keyed no-cancel so every main commit stays verified.
• Setup dedup / artifact reuse across jobs — same cost/benefit as caching: setup and build are already seconds, not the dominant cost (pnpm test ~6 min). Serializing jobs behind a shared build artifact would raise wall time for ~10s of savings.
• Main queue starvation via workflow changes — this issue already says the next lever is runner concurrency limits / a larger pool, not workflow logic. The ci: trim PR Verify matrix to Node 24 and skip heavy jobs on docs-only PRs #232 PR trim is the workflow-side supply reduction.
• main is red: #231 Agent Document stage broke runtime-inspector and public-api declaration tests #240 deterministic main breakage — tracked separately and already closed; ci: trim PR Verify matrix to Node 24 and skip heavy jobs on docs-only PRs #232's first red run was that, not the e2e flake.

Validation (local only; hosted not waited)

• actionlint 1.7.12 on .github/workflows/ci.yml
• rstest unit: classify-docs-only.test.ts 4/4 (allowlist, fail-open, renames, GITHUB_OUTPUT)
• rslint clean on the touched files

Coverage / names

Main's 22.19/24/26 matrix and hosted job names are unchanged. Docs-only skip still fails open.

Closing this issue: the remaining in-repo list is done. Queue-starvation ops and the e2e root cause stay on infra / #122 / #200.