Skip to content

feat(ios): usbmux runner route override + #1403 transport experiment evidence - #1510

Merged
thymikee merged 2 commits into
mainfrom
claude/usbmux-expansion-discussion-ba9eac
Jul 31, 2026
Merged

feat(ios): usbmux runner route override + #1403 transport experiment evidence#1510
thymikee merged 2 commits into
mainfrom
claude/usbmux-expansion-discussion-ba9eac

Conversation

@thymikee

Copy link
Copy Markdown
Member

Summary

Groundwork + live evidence for #1403 (usbmux as the primary physical-iOS runner transport). This PR does not implement the transport switch — it lands the experiment override, its tests, and the measured results that shape the implementation.

  • Adds AGENT_DEVICE_IOS_RUNNER_ROUTE=usbmux: forces coredevice-backend physical devices' runner commands through the existing usbmux transport (read per-resolve inside the route resolver, the single choke point). Also simplifies the xctest branch that awaited a no-op resolveRunnerTransport.
  • Adds docs/usbmux-runner-transport-experiment.md with the full live results and re-measurement pitfalls.

Key evidence (iPhone 17 Pro, iOS 26.5.2, same build, verified output)

Scenario network (tunnel-IP) usbmux
Steady-state snapshot (cable in) 350–450 ms 440 ms
Snapshot after 40 s idle 4.5–4.6 s 440 ms
Runner killed → next command ~4.0 s 4.7 s
Wi-Fi only (cable out) works fails (by design)
open/snapshot/tap/screenshot/close
  • The idle tax is the win: past the 30 s tunnel-IP cache TTL the network route pays ~4.5 s (devicectl tunnel re-probe + runner re-establish) on the next command; over usbmux the session stays hot. Agent workflows hit this shape constantly.
  • Full tunnel-code deletion is off the table: CoreDevice Wi-Fi devices (remoted) never appear in usbmuxd — verified live including unplug. Verdict: usbmux-primary + CoreDevice-network fallback.
  • Failure gap (pre-existing, also hits the xctest backend): with no cable, runner commands burn 2×45 s of retries and surface a generic "Runner did not accept connection", swallowing the usbmux DEVICE_NOT_FOUND cable/trust hint. Implementation should make it non-retryable or trigger immediate network fallback.

Follow-up (implementation PR)

One private route resolver: try usbmux first, fall back to the CoreDevice tunnel on usbmux DEVICE_NOT_FOUND; tunnel cache code survives but only runs on the fallback path. Cabled devices then never pay the probe/TTL tax.

Testing

  • 2 new unit tests covering the override in sendRunnerCommandOnce and waitForRunner; runner-transport.test.ts 9/9.
  • pnpm check:affected --run: green except 5 timeout-only failures in android-test-suite.test.ts (untouched by this change) which pass 7/7 in isolation — known contention flake signature.

Closes nothing; informs #1403.

…evidence

Adds AGENT_DEVICE_IOS_RUNNER_ROUTE=usbmux, an experimental override that
routes coredevice-backend physical devices' runner commands through usbmux,
and simplifies the xctest branch that awaited a no-op resolveRunnerTransport.

Documents the live #1403 experiment (iPhone 17 Pro, USB + Wi-Fi legs):
steady-state is a wash, but the >30s-idle tax drops from ~4.5s (tunnel
re-probe + session re-establish) to ~440ms because the usbmux session stays
hot; CoreDevice Wi-Fi devices never appear in usbmuxd, so the verdict is
usbmux-primary with network fallback rather than tunnel-code deletion.
Also records the cable-out failure gap (2x45s retry hang swallowing the
usbmux DEVICE_NOT_FOUND hint), which affects today's xctest backend too.
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.91 MB 1.90 MB -7.7 kB
JS gzip 612.2 kB 609.9 kB -2.4 kB
npm tarball 729.7 kB 727.6 kB -2.1 kB
npm unpacked 2.56 MB 2.55 MB -9.9 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.0 ms 27.2 ms -0.8 ms
CLI --help 59.1 ms 58.4 ms -0.7 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/device2.js +62.5 kB +18.5 kB
dist/src/internal/daemon.js -27.0 kB -8.0 kB
dist/src/session.js +21.7 kB +6.0 kB
dist/src/cli.js +18 B +134 B
dist/src/sdk-selectors.js +90 B +30 B

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed exact head b7baccd2c020209fa1fa094a4496cc7ea4c0ab5b against #1403. One actionable experiment-integrity issue remains.

[P2] AGENT_DEVICE_IOS_RUNNER_ROUTE is daemon-scoped, not per CLI command. The client reuses an existing long-lived daemon, and startDaemon copies process.env only when that daemon launches. Changing the variable for a later CLI invocation can therefore silently keep the previous route, contradicting the source comment that per-resolve reads avoid daemon restarts and risking another misattributed A/B. For this experiment, the cheap fix is enough: correct the comment/docs (including “uncommitted experiment patch”) to require a fresh isolated state-dir/daemon for each route leg, and add a lifecycle regression that locks this behavior. Making it request-scoped can wait unless you want that contract.

The centralized route seam, direct/wait coverage, and live lifecycle/idle/Wi-Fi evidence otherwise support this groundwork. Multi-device/unpaired matrices remain follow-up risk for the implementation PR. Exact-head Coverage is also red from an unrelated corpus-replay.test.ts fork-worker termination after 615/616 files passed; rerun it before merge. Do not apply ready-for-human yet.

Residual risk: no separately authorized cross-vendor review was performed.

The env is re-read per resolve but from the daemon's environment, which is
captured at daemon launch — a later CLI invocation cannot flip the route on
a running daemon. Correct the source comment and experiment doc, and lock
the read-point semantics with a regression test.
@thymikee

Copy link
Copy Markdown
Member Author

Clean review at 034a99fe20e0572f42842db06eccd164e5ed453b. The prior experiment-integrity finding is resolved: source and docs now correctly describe the override as daemon-scoped and require a fresh isolated daemon/state-dir per route leg. Both command-send and readiness paths still route through the same private resolver, and the new test pins its environment read point.

The live USB/Wi-Fi evidence is credible and candid about remaining multi-device, failure-matrix, and immediate-fallback work for the later implementation PR. Exact-head CI is fully green, including Coverage and all platform smokes. Ready for human review.

Residual risk: no separately authorized cross-vendor review was performed.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 31, 2026
@thymikee
thymikee merged commit 6b0b9cb into main Jul 31, 2026
30 of 45 checks passed
@thymikee
thymikee deleted the claude/usbmux-expansion-discussion-ba9eac branch July 31, 2026 06:36
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-31 06:36 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant