fix: make record app-scoped by default - #1163
Conversation
Size Report
Startup median (7 runs, lower is better):
Top changed chunks:
|
Review: does this fix #1162 correctly?TL;DR — yes for the reported repro, and it's well-tested and green, but it's a scoped fix that closes the exact reproduction rather than the general "valid MP4 of the wrong surface" class the issue describes. The gate is built on session state ( What it fixes, and why it worksThe repro hinges on a fact that's easy to miss: The two new guards both land on this:
Tests cover both, plus the still-supported record-only path and the metadata. CI is green. Residual gaps (the part I'd flag)
Closing (1)–(2) really needs one Minor
Net: good, safe, well-tested fix for the filed repro. I'd merge it, but I'd either re-scope #1162 to "explicit-session fallback" (and open a follow-up for real foreground verification) or note in the PR that the foreground-proof guarantee and the remaining JSON fields are deferred. |
5dfc715 to
ecfe909
Compare
|
Review pass on ecfe909: checked the record start/stop route against issue #1162. Explicit missing sessions now fail before record-only capture, existing iOS simulator sessions require active app context, Agent Device Runner is rejected as the active app, and record-only simulator capture remains available without an existing explicit session. Metadata is added to start/stop responses, tests cover the reported path, and the PR includes live iPhone 17 simulator evidence. Checks are 21/21 green. No actionable blockers found. |
|
|
Reconciled the earlier scope concern in 6288138.
I also verified the revised behavior live on an iPhone 17 simulator:
The remaining caveat is intentional for this PR: app scope is enforced from tracked session state, not a new iOS foreground-bundle probe. |
6288138 to
5364abc
Compare
Android --settle diffs were flooded by system chrome: status-bar churn (clock/signal/battery) appeared as +/- lines in every settled diff, and summoning the keyboard added dozens of IME ref-bearing nodes that suppressed the unchanged-interactive tail (it fired zero times across the 2026-07-10 Android benchmark, regressing haiku to 2/3 success). Every Android accessibility node carries its owning package (bundleId), so settle chrome is now classified per node: IME-owned nodes collapse to one surviving line per contiguous run (the iOS [Keyboard] analog), and with a session appBundleId all other foreign-package nodes are dropped from both diff sides, mirroring record --scope app (#1163). Classification deliberately never walks parentIndex chains: Android's interactive-only pruning can chain an app node's parent through another window's node (locked in by a real-capture regression test). Inert on iOS/macOS, whose nodes never set bundleId. Benchmark (haiku, settle arm, 3 runs): 2/3 -> 3/3 success, snapshots per run 21/5/6 -> 7/1/3, settle uses 3/9/8 -> 10/11/11.
#1200) * fix: scope Android settle diffs to app content and collapse IME chrome Android --settle diffs were flooded by system chrome: status-bar churn (clock/signal/battery) appeared as +/- lines in every settled diff, and summoning the keyboard added dozens of IME ref-bearing nodes that suppressed the unchanged-interactive tail (it fired zero times across the 2026-07-10 Android benchmark, regressing haiku to 2/3 success). Every Android accessibility node carries its owning package (bundleId), so settle chrome is now classified per node: IME-owned nodes collapse to one surviving line per contiguous run (the iOS [Keyboard] analog), and with a session appBundleId all other foreign-package nodes are dropped from both diff sides, mirroring record --scope app (#1163). Classification deliberately never walks parentIndex chains: Android's interactive-only pruning can chain an app node's parent through another window's node (locked in by a real-capture regression test). Inert on iOS/macOS, whose nodes never set bundleId. Benchmark (haiku, settle arm, 3 runs): 2/3 -> 3/3 success, snapshots per run 21/5/6 -> 7/1/3, settle uses 3/9/8 -> 10/11/11. * fix: keep unknown-foreign Android packages in settle output PR #1200 review blocker (live-reproduced with a real Sharesheet): the initial blanket foreign-package drop made settle blind to blocking system dialogs — a ResolverActivity over the app produced an empty diff and a tail pointing at covered buttons, relocating the death-spiral failure mode from IME noise to system dialogs. Invert the default to three per-node tiers: IME packages still collapse to one line per contiguous run; persistent system chrome (exactly com.android.systemui, the status-bar churn that motivated the fix) drops from both diff sides; every other foreign package is kept in full — a permission prompt's or share sheet's buttons stay actionable. appBundleId is no longer a classification input, only the never-drop-the-app-under-test guard, which also defuses the pre-action staleness noted in review. Regression tests use the real captured Sharesheet (raw 41-node + interactive 10-node shapes, package "android" throughout). * fix: drop only status/nav-bar systemui runs from settle output Second review round on PR #1200: com.android.systemui hosts actionable overlays (volume panel, media/output pickers, some system prompts), so package-level dropping reintroduced dialog blindness for that class. Key the drop on window-runs instead, live-verified on the emulator: the status-bar window carries com.android.systemui:id/status_bar* markers throughout while the VolumeDialog window carries only volume_dialog* ids. A systemui run (contiguous same-package parent chain) drops only when a member matches the status_bar*/navigation_bar* marker prefixes; every other systemui surface is kept with actionable refs. Run grouping never crosses packages, preserving the per-node cross-window guarantee. Also corrects the tracking-issue references (#1178 was an unrelated registry issue; the Android settle-noise issue is #1198) and adds a real-capture VolumeDialog regression test: status bar drops, volume dialog survives in both diff and tail.
Summary
Makes
record startapp-scoped by default: it now requires an active app session fromopen <app>instead of silently creating a record-only whole-screen capture when the session is missing.Adds explicit
--scope app|device|system;device/systemare the opt-in whole-screen intent values for recordings that intentionally span the full screen, multiple apps, Settings, Home, or app transitions. iOS simulator record-only capture now requires one of those whole-screen scopes.Persists and returns
recordingScopealongside the existing backend, record-only state, active app, and duration metadata, and updates CLI help/docs plus provider scenarios for the new default.Scope note: this resolves #1162 by making app proof videos the default and whole-screen capture explicit. It does not add a new iOS foreground-bundle probe; app scope is enforced from session state today.
Validation
pnpm check:quick./node_modules/.bin/vitest run --project unit-core src/commands/recording/index.test.ts src/cli/parser/__tests__/args-parse-interaction.test.ts src/cli/parser/__tests__/cli-help-command-usage.test.ts./node_modules/.bin/vitest run --project unit-core src/commands/recording/index.test.ts src/cli/parser/__tests__/args-parse-interaction.test.ts src/daemon/handlers/__tests__/record-trace.test.ts./node_modules/.bin/vitest run --project unit-core src/commands/__tests__/command-flags.test.ts src/cli/parser/__tests__/cli-help-command-usage.test.ts./node_modules/.bin/vitest run --project provider-integration./node_modules/.bin/vitest run --project unit-core --project android-adb./node_modules/.bin/vitest run --coveragenode --test test/integration/smoke-*.test.ts./node_modules/.bin/tsdownoxfmt --writeover the repo becausepnpm formatwas blocked by pnpm version verification in this environment.--sessionwithINVALID_ARGSand creates no MP4.--scope devicerecord-only capture succeeds and produces/private/tmp/agent-device-record-device-scope.mp4(52K), then cleans up the session.open settingsfollowed by defaultrecord startsucceeds withrecordingScope: "app"andactiveSessionApp.bundleId: "com.apple.Preferences", produces/private/tmp/agent-device-record-app-scope.mp4(52K), then cleans up the session.