diff --git a/CHANGELOG.md b/CHANGELOG.md index 92a0cc4f58..06c87ba2ba 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ ## Unreleased +- `agent-device help workflow` is now a compact ~8KB card instead of a ~41KB dump; the same depth still exists, split into `help scripting` (save-script, secret-safe fills, batch JSON, replay divergence/repair, recording) and `help gestures` (multi-touch shapes and platform quirks), plus a few paragraphs folded into the topics that already owned the subject (`help debugging`, `help physical-device`, `help validate`). Every `help ` first line is now `agent-device ` so an agent can read the installed version from its mandatory first help read instead of a separate `agent-device --version` call. + - `scroll` and `back` now accept `--settle` (with `--settle-quiet` and `--timeout`), collapsing scroll-then-observe and back-then-observe into one call (#1638). The response carries the same settled payload the touch commands return — verdict, changed-lines diff with fresh refs on added lines, the unchanged-interactive tail, and `refsGeneration` when the settled tree was stored — and is best-effort: it never fails the action. One difference is deliberate: `scroll`/`back` resolve no element, so the diff baseline is the session's stored pre-action tree ("the last tree you observed") rather than a freshly resolved pre-action capture. Both commands now also preserve the daemon on timeout, like the other settle-capable commands. - Security: repository `./agent-device.json` now accepts only project-safe automation defaults. It rejects daemon endpoint/auth/transport/server settings, tenant/run/lease selectors, provider/cloud and Metro connection fields, headers, executable reporter modules, local write destinations, and other operator-controlled values before local module loading or any daemon health/RPC request. Put remote endpoint and token together in protected CI environment variables, user config, an explicit `--config` file, or the existing `connect`/`--remote-config` workflow. Daemon auth tokens no longer travel in serialized command flags. - `viewport` is now rejected during capability admission on Apple targets instead of reaching the device and failing inside dispatch. No Apple backend can resize a screen — simulator and device geometry is fixed by the selected device type — so `viewport` on iOS/iPadOS/tvOS/macOS now fails with `UNSUPPORTED_OPERATION`, `viewport is not supported on this device`, and a hint pointing at `--platform web` and at picking a different simulator. `capabilities` no longer advertises `viewport` on Apple targets. Web viewport resizing (`agent-device viewport 1280 900 --platform web`) is unchanged, and Android was already denied. diff --git a/scripts/__tests__/help-conformance-bench.test.ts b/scripts/__tests__/help-conformance-bench.test.ts index 3bccd5fd88..aba7588f40 100644 --- a/scripts/__tests__/help-conformance-bench.test.ts +++ b/scripts/__tests__/help-conformance-bench.test.ts @@ -379,6 +379,80 @@ test('plan validator rejects shell projection and non-permitted executables', as assert.ok(placeholder.issues.some(({ kind }) => kind === 'shell-projection')); }); +// The compact workflow card teaches chaining confident consecutive steps +// with an unquoted `&&`. This is the validator side of that contract: split +// on `&&` and validate each chained segment as its own agent-device command, +// instead of failing the whole line as one shell-projection violation. +test('plan validator splits an unquoted && chain into independently valid segments', async () => { + const [press, fill] = await validatePlanCommands([ + 'agent-device press \'label="Search"\' --settle && agent-device fill \'label="Search"\' "query" --settle', + ]); + assert.equal(press.issues.length, 0); + assert.deepEqual(press.tokens, ['agent-device', 'press', 'label="Search"', '--settle']); + assert.equal(fill.issues.length, 0); + assert.deepEqual(fill.tokens, ['agent-device', 'fill', 'label="Search"', 'query', '--settle']); +}); + +test('plan validator fails only the offending segment of a chained plan', async () => { + const [goodFirst, badSecond] = await validatePlanCommands([ + 'agent-device snapshot -i && agent-device press @ --settle', + ]); + assert.equal(goodFirst.issues.length, 0); + assert.ok(badSecond.issues.some(({ kind }) => kind === 'pseudo-ref')); +}); + +test('plan validator does not split && inside a quoted selector value', async () => { + const [single] = await validatePlanCommands([ + 'agent-device fill \'label="A && B"\' "value" --settle', + ]); + assert.equal(single.issues.length, 0); + assert.deepEqual(single.tokens, ['agent-device', 'fill', 'label="A && B"', 'value', '--settle']); +}); + +test('plan validator still rejects an unquoted lone & as a shell operator', async () => { + const [lone] = await validatePlanCommands(['agent-device open foo & agent-device close']); + assert.equal(lone.issues[0]?.kind, 'shell-projection'); +}); + +test('plan validator keeps single-command results identical when no chain is present', async () => { + const [single] = await validatePlanCommands(['agent-device snapshot -i']); + assert.equal(single.issues.length, 0); + assert.deepEqual(single.tokens, ['agent-device', 'snapshot', '-i']); + assert.equal(single.command, 'agent-device snapshot -i'); +}); + +// A real shell rejects && with an empty operand on either side. A validator +// that silently dropped the empty segment (instead of failing it) would +// bless a plan that fails at execution — exactly the gap review found. +test('plan validator rejects a leading && as an empty chain operand', async () => { + const [empty, closeSegment] = await validatePlanCommands(['&& agent-device close']); + assert.equal(empty.issues[0]?.kind, 'empty-chain-operand'); + assert.equal(closeSegment.issues.length, 0); +}); + +test('plan validator rejects a trailing && as an empty chain operand', async () => { + const [pressSegment, empty] = await validatePlanCommands(['agent-device press @e1 --settle &&']); + assert.equal(pressSegment.issues.length, 0); + assert.equal(empty.issues[0]?.kind, 'empty-chain-operand'); +}); + +test('plan validator rejects a doubled && as an empty chain operand', async () => { + const [openSegment, empty, closeSegment] = await validatePlanCommands([ + 'agent-device open foo && && agent-device close', + ]); + assert.equal(openSegment.issues.length, 0); + assert.equal(empty.issues[0]?.kind, 'empty-chain-operand'); + assert.equal(closeSegment.issues.length, 0); +}); + +test('plan validator still allows a quoted && to pass through a single segment unsplit', async () => { + const [single] = await validatePlanCommands([ + 'agent-device fill \'label="A && B"\' "value" --settle', + ]); + assert.equal(single.issues.length, 0); + assert.deepEqual(single.tokens, ['agent-device', 'fill', 'label="A && B"', 'value', '--settle']); +}); + test('case matchers score parsed tokens so shell quoting does not change results', async () => { const commands = [ 'agent-device open "com.example.shop"', diff --git a/scripts/help-conformance-case-checks.mjs b/scripts/help-conformance-case-checks.mjs index f50e03f5f2..3f7e8b4c1b 100644 --- a/scripts/help-conformance-case-checks.mjs +++ b/scripts/help-conformance-case-checks.mjs @@ -10,6 +10,12 @@ const EXPECTATION_SCORERS = { ), usesSnapshotI: ({ commands }) => commands.some((command) => /\bsnapshot\b.*\s-i\b/.test(command)), usesSettleOnMutations: ({ commands }) => allMutationsUseSettle(commands), + // help workflow teaches chaining confident consecutive steps with an + // unquoted &&. This reads the raw (pre-split) command lines, not `joined` + // (canonicalPlan flattens a chain into separate lines once the plan + // validator splits and validates each segment), so it is the only place + // that can tell whether the model actually chained. + usesConfidentChaining: ({ commands }) => commands.some((command) => /&&/.test(command)), noWaitStable: ({ joined }) => !joined.includes('wait stable'), verifiesNamedExpectation: ({ joined }) => /\b(wait|is|get|find)\b/.test(joined), usesDogfoodEvidence: ({ joined }) => diff --git a/scripts/help-conformance-cases.mjs b/scripts/help-conformance-cases.mjs index 456ff0c8b5..573a1d59df 100644 --- a/scripts/help-conformance-cases.mjs +++ b/scripts/help-conformance-cases.mjs @@ -315,6 +315,86 @@ export const CASES = [ { id: 'noOpenArtifactPath', pattern: /(?:^|\n)agent-device\s+open\s+[^\n]*\.apk\b/i }, ], }, + { + // Review of the compact workflow card's && guidance found the plan + // validator failed a plan that followed it (unquoted && classified as + // shell-projection). Both target elements are already named/unambiguous + // here, which is exactly the "confident consecutive steps" case the card + // describes, so usesConfidentChaining is a real (not just possible) + // expectation, and validPlanCommands proves the fixed validator accepts + // the chained shape end to end. + id: 'chains-confident-consecutive-settle-steps', + docs: ['--help:first30', 'workflow'], + task: 'The Search tab is visible, labeled "Search", and known to reveal a search field also labeled "Search" with no other candidate on screen. Plan commands to press the Search tab and fill that field with "react native", settling after each step, then close. Plan commands only.', + expectations: [ + 'validPlanCommands', + 'fullPrefix', + 'usesSettleOnMutations', + 'usesConfidentChaining', + ], + matchers: [ + { + id: 'pressesSearchTab', + pattern: /\bagent-device\s+press\b[^\n]*label="?search"?[^\n]*--settle\b/i, + }, + { + id: 'fillsSearchField', + pattern: + /\bagent-device\s+fill\b[^\n]*label="?search"?[^\n]*(?:"react native"|'react native')[^\n]*--settle\b/i, + }, + ], + }, + { + // help scripting owns --record-as secret-safe fills and save-script + // authoring now that this content left the mandatory workflow card; + // this proves an agent can actually plan the loop from the topic alone. + id: 'scripting-secret-safe-recorded-login', + docs: ['--help:first30', 'scripting'], + task: 'Author a reusable login script for the installed app com.example.app that never records the literal password. The AD_VAR_PASSWORD environment variable is already set in your shell, so do not plan a shell export line. Arm recording on open with --save-script=login.ad, fill the password field (id="password") from AD_VAR_PASSWORD using --record-as, verify the login succeeded, then publish the script without closing the session. Plan agent-device commands only.', + expectations: ['validPlanCommands', 'fullPrefix'], + matchers: [ + { + id: 'armsSaveScriptOnOpen', + pattern: /\bagent-device\s+open\s+com\.example\.app\b[^\n]*--save-script[=\s]*login\.ad/i, + }, + { + id: 'recordsSecretSafeFill', + pattern: + /\bagent-device\s+fill\s+(?:'|")?id="?password"?(?:'|")?\s+"?\$AD_VAR_PASSWORD"?[^\n]*--record-as\s+PASSWORD\b/i, + }, + { + id: 'verifiesLoginSucceeded', + pattern: /\b(?:wait|is|get|find)\b/i, + }, + { id: 'publishesWithoutClosing', pattern: /\bagent-device\s+session\s+save-script\b/i }, + ], + forbidden: [ + { id: 'noBareClose', pattern: /(?:^|\n)agent-device\s+close\b/i }, + { id: 'noNoRecordOnSecretFill', pattern: /--no-record/i }, + ], + }, + { + // help gestures owns multi-touch shapes now that this content left the + // mandatory workflow card. The exact verification text ("pan changed + // yes") only appears in the gestures topic's own example, so a correct + // plan proves the model actually read it rather than guessing a shape. + id: 'gestures-android-transform-then-verify', + docs: ['--help:first30', 'gestures'], + task: 'On the already-open Android app, plan a combined pan/scale/rotate transform gesture centered at (200, 420) with dx=80, dy=-40, scale=2, rotate=35 degrees over 700ms, then verify the app-reported pan change using the exact confirmation text shown in the gesture reference. Plan commands only.', + expectations: ['validPlanCommands', 'fullPrefix'], + matchers: [ + { + id: 'runsAndroidTransform', + pattern: + /\bagent-device\s+gesture\s+transform\s+200\s+420\s+80\s+-40\s+2\s+35\s+700\b[^\n]*--platform\s+android\b/i, + }, + { + id: 'verifiesSemanticPanChange', + pattern: /\bagent-device\s+wait\s+text\s+"pan changed yes"[^\n]*--platform\s+android\b/i, + }, + ], + forbidden: [{ id: 'noRawCoordinateTarget', pattern: RAW_COORDINATE_TARGET }], + }, // Next-command quiz cases: captured output (pinned to the real renderer by // scripts/__tests__/help-conformance-sample-outputs.test.ts) plus a task, // scored by regex instead of the named expectation scorers above. diff --git a/scripts/help-conformance-plan-validator.mjs b/scripts/help-conformance-plan-validator.mjs index 6a2181dae2..327ee3d2a8 100644 --- a/scripts/help-conformance-plan-validator.mjs +++ b/scripts/help-conformance-plan-validator.mjs @@ -9,7 +9,7 @@ const COMMAND_VALIDATOR = join(ROOT, 'scripts', 'help-conformance-command-valida const ALLOWED_PNPM_SCRIPTS = new Set(['build', 'build:android', 'build:xcuitest', 'clean:daemon']); export async function validatePlanCommands(commands, options = {}) { - const parsedCommands = commands.map((command) => parsePlanCommand(command)); + const parsedCommands = commands.flatMap((command) => parseCommandLine(command)); const agentCommands = parsedCommands.filter( ({ tokens, issues }) => issues.length === 0 && tokens[0] === 'agent-device', ); @@ -38,6 +38,75 @@ function applyCommandPolicy(parsed, agentResultState, allowedExternalCommands) { ); } +// The compact workflow card teaches chaining confident consecutive steps with +// an unquoted `&&` (`press ... --settle && fill ... --settle`). Split on it +// before tokenizing a line so each chained segment is validated as its own +// full agent-device invocation, rather than the whole line failing as one +// shell-projection violation. A `&&` inside a quoted selector value (for +// example label="A && B") is not a chain boundary and must not split. +// +// A real shell rejects `&&` with an empty operand on either side (leading +// `&& foo`, trailing `foo &&`, or doubled `foo && && bar`): each is a syntax +// error, not two commands. The splitter below produces an empty segment for +// exactly those shapes, so parseChainSegment turns an empty (post-trim) +// segment into a validation issue instead of silently dropping it — a plan +// with one of these shapes must not be blessed by validPlanCommands when it +// would fail at execution. +function parseCommandLine(command) { + return splitOnUnquotedAnd(command).map((segment) => parseChainSegment(segment)); +} + +function parseChainSegment(segment) { + if (segment.trim().length > 0) return parsePlanCommand(segment.trim()); + return { + command: segment, + tokens: [], + issues: [ + { + kind: 'empty-chain-operand', + error: + 'A && chain must have a non-empty command on both sides (no leading, trailing, or doubled &&).', + }, + ], + }; +} + +function splitOnUnquotedAnd(command) { + const state = { segments: [], current: '', quote: undefined }; + for (let index = 0; index < command.length; index += 1) { + index = consumeSplitCharacter(command, index, state); + } + state.segments.push(state.current); + return state.segments; +} + +function consumeSplitCharacter(command, index, state) { + const character = command[index]; + if (state.quote) return consumeQuotedSplitCharacter(command, index, character, state); + if (character === "'" || character === '"') { + state.quote = character; + state.current += character; + return index; + } + if (character === '&' && command[index + 1] === '&') { + state.segments.push(state.current); + state.current = ''; + return index + 1; + } + state.current += character; + return index; +} + +function consumeQuotedSplitCharacter(command, index, character, state) { + if (character === '\\' && state.quote === '"' && index + 1 < command.length) { + state.current += character + command[index + 1]; + return index + 1; + } + state.current += character; + if (character === state.quote) state.quote = undefined; + return index; +} + function parsePlanCommand(command) { const tokenized = tokenize(command); const issues = []; diff --git a/skills/agent-device/SKILL.md b/skills/agent-device/SKILL.md index fe93725bce..4454413728 100644 --- a/skills/agent-device/SKILL.md +++ b/skills/agent-device/SKILL.md @@ -5,17 +5,7 @@ description: Automates Apple-platform apps (iOS, tvOS, macOS), Android devices, # agent-device -Router only. Private setup before using this skill: - -```bash -agent-device --version -``` - -If that fails but the user may have installed `agent-device` globally, check the user's configured login/interactive shell and environment before using `npx`. Resolve the command the same way the user would from a normal terminal session, then run the absolute binary path if found. This may require inspecting shell startup behavior or package-manager/global bin locations; do not assume the Codex process `PATH` is the user's `PATH`. - -Require `agent-device >= 0.20.0`; older CLIs lack the current help topics and Vega OS routing. If older, stop and tell the user to upgrade the trusted install or approve an exact-version npm command. Do not run `npm install -g agent-device@latest` or `npx -y agent-device@latest` autonomously, and do not include version/upgrade commands in final plans. - -Before your first agent-device command or plan, read the smallest version-matched CLI guide that fits the task: +Router only. Before your first agent-device command or plan, read the smallest version-matched CLI guide that fits the task — this single read also replaces a separate `agent-device --version` check: ```bash agent-device help manual-qa # scripted/manual QA, acceptance checks, checklist execution @@ -24,10 +14,16 @@ agent-device help dogfood # exploratory app dogfooding and evidence collecti agent-device help workflow # fallback reference for general app driving or mixed tasks ``` +That topic's first line is `agent-device ` (for example `agent-device 0.21.0 — workflow`). Read the version from it instead of running `agent-device --version` separately. If the first line instead reads `agent-device help ` with no version — or the command fails, or the topic is unrecognized — the installed CLI predates this header and its current help topics/Vega OS routing. Stop and tell the user to upgrade the trusted install or approve an exact-version npm command. Do not run `npm install -g agent-device@latest` or `npx -y agent-device@latest` autonomously, and do not include version/upgrade commands in final plans. + +If `agent-device` fails outright but the user may have installed it globally, check the user's configured login/interactive shell and environment before using `npx`. Resolve the command the same way the user would from a normal terminal session, then run the absolute binary path if found. This may require inspecting shell startup behavior or package-manager/global bin locations; do not assume the Codex process `PATH` is the user's `PATH`. + Read additional topics only when relevant: ```bash agent-device help debugging +agent-device help scripting # save-script, secret-safe fills, batch JSON, replay repair +agent-device help gestures # multi-touch gesture shapes and platform quirks agent-device help react-native agent-device help react-devtools agent-device help cdp diff --git a/src/__tests__/cli-help.test.ts b/src/__tests__/cli-help.test.ts index fafea5d358..81c78f4965 100644 --- a/src/__tests__/cli-help.test.ts +++ b/src/__tests__/cli-help.test.ts @@ -64,87 +64,80 @@ test('help react-devtools prints agent workflow topic and skips daemon dispatch' const result = await runCliCapture(['help', 'react-devtools']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match(result.stdout, /agent-device help react-devtools/); + assert.match(result.stdout, /^agent-device \S+ — react-devtools/); assert.match(result.stdout, /React Native performance\/profiling/); assert.match(result.stdout, /agent-device react-devtools status/); }); -test('help workflow prints agent workflow topic and skips daemon dispatch', async () => { +test('help workflow prints the compact workflow card with a version header and skips daemon dispatch', async () => { const result = await runCliCapture(['help', 'workflow']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match(result.stdout, /agent-device help workflow/); - assert.match(result.stdout, /Core loop:/); - assert.match(result.stdout, /Do not use CSS selectors/); - assert.match(result.stdout, /Native \.ad interpolation is late-bound after planning/); + assert.match(result.stdout, /^agent-device \S+ — workflow/); + assert.ok( + Buffer.byteLength(result.stdout, 'utf8') < 9000, + `help workflow should stay close to the compact-card size target, was ${Buffer.byteLength(result.stdout, 'utf8')} bytes`, + ); + assert.match(result.stdout, /open -> snapshot -i -> settle -> verify -> close loop/); + assert.match(result.stdout, /no CSS selectors/); + assert.match(result.stdout, /help scripting/); + assert.match(result.stdout, /help gestures/); +}); + +test('help workflow encourages chaining confident steps and requires the end state to be on screen', async () => { + const result = await runCliCapture(['help', 'workflow']); + assert.equal(result.code, 0); + assert.equal(result.calls.length, 0); + assert.match(result.stdout, /Chain confident consecutive steps with &&/); + assert.match(result.stdout, /Fall back to one command at a time when a step is uncertain/); assert.match( result.stdout, - /Maestro environment substitution occurs during compatibility parsing/, + /confirm the requested end state is actually visible on the current screen, scrolling it into view if needed/, ); + assert.match(result.stdout, /get text alone, or stopping one screen early, is not enough/); }); test('help workflow preserves known device workaround guidance', async () => { const result = await runCliCapture(['help', 'workflow']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match(result.stdout, /disabled\/hittable:false/); - assert.match(result.stdout, /snapshot -i --json/); + assert.match(result.stdout, /snapshot -i/); assert.match(result.stdout, /@Label_Name/); assert.match(result.stdout, /press @e12/); - assert.match(result.stdout, /Snapshot legend:/); - assert.match(result.stdout, /preview="Leave at side\.\.\." truncated/); - assert.match(result.stdout, /wait text/); - assert.match(result.stdout, /Never use args/); - assert.match(result.stdout, /Never use args, step/); - assert.match(result.stdout, /scroll bottom\/top/); + assert.match(result.stdout, /Legend:/); assert.match(result.stdout, /--delay-ms/); - assert.match(result.stdout, /Discovery is not enough when the task asks to open\/start/); - assert.match(result.stdout, /If the task says install, use install/); - assert.match(result.stdout, /do not inspect project files to find one/); - assert.match(result.stdout, /do not split clear\/restart/); - assert.match(result.stdout, /do not write network log headers/); - assert.match(result.stdout, /iOS Allow Paste prompt cannot be exercised under XCUITest/); - assert.match(result.stdout, /agent-device clipboard write "some text"/); - assert.match(result.stdout, /provider-native text injection when available/); - assert.match(result.stdout, /Do not switch to raw adb, clipboard, or paste as an agent fallback/); - assert.match(result.stdout, /exact key that includes the agent-device package and Xcode version/); - assert.match(result.stdout, /Avoid broad restore-key fallbacks/); + assert.match(result.stdout, /Never open artifact paths or invent package ids/); + assert.match( + result.stdout, + /iOS paste-prompt limits and Android IME\/handwriting capture quirks/, + ); }); test('help workflow documents the selector disambiguation policy (#1037)', async () => { const result = await runCliCapture(['help', 'workflow']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match(result.stdout, /does not fail by default/); + assert.match(result.stdout, /do not fail by default/); assert.match(result.stdout, /deepest node first/); assert.match(result.stdout, /then smallest on-screen area/); assert.match(result.stdout, /Selector did not resolve uniquely/); - assert.match( - result.stdout, - /replay's suggestion re-resolution.*applies the same depth-then-area policy/, - ); assert.match(result.stdout, /targetHittable: false/); }); -test('help workflow documents selector and hittability guarantees (#1051)', async () => { +test('help workflow documents open/close/relaunch runner guarantees as lifecycle facts (#1051)', async () => { const result = await runCliCapture(['help', 'workflow']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match(result.stdout, /Guarantees:/); - assert.match( - result.stdout, - /auto-disambiguates deepest node first, then smallest on-screen area/, - ); - assert.match(result.stdout, /Selector did not resolve uniquely/); - assert.match(result.stdout, /non-hittable resolution is allowed by design/); - assert.match(result.stdout, /targetHittable: false plus a hint/); + assert.match(result.stdout, /Lifecycle facts \(trust these instead of probing\)/); + assert.match(result.stdout, /idempotent-foreground/); + assert.match(result.stdout, /already owned by another agent-device daemon/); + assert.match(result.stdout, /Env vars: help physical-device/); }); -test('help workflow documents open/close/relaunch runner guarantees (#1051)', async () => { - const result = await runCliCapture(['help', 'workflow']); +test('help physical-device documents the runner/daemon lifecycle detail moved out of workflow (#1051)', async () => { + const result = await runCliCapture(['help', 'physical-device']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match(result.stdout, /idempotent-foreground for an already-running app/); assert.match( result.stdout, /one simctl launch --terminate-running-process call instead of a separate terminate-then-launch/, @@ -155,13 +148,6 @@ test('help workflow documents open/close/relaunch runner guarantees (#1051)', as ); assert.match(result.stdout, /the session held a device lease/); assert.match(result.stdout, /AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS/); - assert.match(result.stdout, /default 5 minutes/); -}); - -test('help workflow documents daemon idle reap and stale lease takeover guarantees', async () => { - const result = await runCliCapture(['help', 'workflow']); - assert.equal(result.code, 0); - assert.equal(result.calls.length, 0); assert.match( result.stdout, /self-exits after an idle window \(default 5 minutes, matching the runner idle-stop default\)/, @@ -174,24 +160,29 @@ test('help workflow documents daemon idle reap and stale lease takeover guarante assert.match(result.stdout, /genuinely live owner whose state dir still exists still rejects/); }); -test('help workflow documents ref lifetime, snapshot diff, and wait guarantees (#1051)', async () => { +test('help workflow documents ref lifetime and snapshot diff guarantees (#1051)', async () => { const result = await runCliCapture(['help', 'workflow']); assert.equal(result.code, 0); assert.equal(result.calls.length, 0); - assert.match( - result.stdout, - /open and open --relaunch clear the session's stored snapshot outright/, - ); - assert.match( - result.stdout, - /diff snapshot compares the current capture against the session's last stored snapshot/, - ); - assert.match( - result.stdout, - /initializes the baseline and reports zero additions\/removals instead of failing/, - ); - assert.match(result.stdout, /polls on a fixed interval \(300ms\)/); - assert.match(result.stdout, /Timing out raises a command failure/); + assert.match(result.stdout, /open\/--relaunch clears the stored snapshot outright/); + assert.match(result.stdout, /initializes the baseline \(zero changes\) instead of failing/); +}); + +test('help scripting documents replay divergence/resume and wait polling guarantees (#1051)', async () => { + const result = await runCliCapture(['help', 'scripting']); + assert.equal(result.code, 0); + assert.equal(result.calls.length, 0); + assert.match(result.stdout, /REPLAY_DIVERGENCE with a bounded report/); + assert.match(result.stdout, /replay --from --plan-digest /); + assert.match(result.stdout, /resume never re-executes skipped steps/); +}); + +test('help gestures prints the multi-touch topic and skips daemon dispatch', async () => { + const result = await runCliCapture(['help', 'gestures']); + assert.equal(result.code, 0); + assert.equal(result.calls.length, 0); + assert.match(result.stdout, /^agent-device \S+ — gestures/); + assert.match(result.stdout, /agent-device gesture transform 200 420 80 -40 2 35 700/); }); test('help unknown command prints error plus global usage and skips daemon dispatch', async () => { diff --git a/src/cli/parser/__tests__/cli-help-command-usage.test.ts b/src/cli/parser/__tests__/cli-help-command-usage.test.ts index 0b1fe9077e..b0700e93ee 100644 --- a/src/cli/parser/__tests__/cli-help-command-usage.test.ts +++ b/src/cli/parser/__tests__/cli-help-command-usage.test.ts @@ -136,13 +136,17 @@ test('usageForCommand documents prepare ios-runner', async () => { test('workflow help keeps common copyable command forms', async () => { const help = await usageForCommand('workflow'); if (help === null) throw new Error('Expected workflow help text'); - assert.match(help, /network dump --include headers/); - assert.match(help, /settings animations off/); - assert.match(help, /connect --remote-config/); assert.match(help, /metro reload/); assert.match(help, /screenshot --overlay-refs/); - assert.match(help, /snapshot -s @e7/); - assert.match(help, /clipboard write "some text"/); + // Concrete ref shape, not the @ref placeholder the same card forbids using + // as a target (#1663 review). + assert.match(help, /snapshot -s @e12 \(the current concrete ref\)/); + assert.doesNotMatch(help, /snapshot -s @ref\b/); + // Moved out of the compact card (not deleted) into their owning sub-topics: + // network dump/settings animations -> help debugging; connect --remote-config + // -> help remote; clipboard write -> help debugging (text-entry quirks). + assert.match(help, /help debugging/); + assert.match(help, /help remote/); }); test('debug command help stays scoped to symbolication', async () => { @@ -198,7 +202,7 @@ test('session command help includes daemon state directory discovery', async () test('web command help includes managed backend setup', async () => { const help = await usageForCommand('web'); if (help === null) throw new Error('Expected command help text'); - assert.match(help, /agent-device help web/); + assert.match(help, /^agent-device \S+ — web/); assert.match(help, /managed, pinned agent-browser backend/); assert.match( help, diff --git a/src/cli/parser/__tests__/cli-help-topics.test.ts b/src/cli/parser/__tests__/cli-help-topics.test.ts index 79edd59163..b5ef7d7971 100644 --- a/src/cli/parser/__tests__/cli-help-topics.test.ts +++ b/src/cli/parser/__tests__/cli-help-topics.test.ts @@ -234,127 +234,149 @@ test('usageForCommand resolves Maestro compatibility help topic', async () => { test('usageForCommand resolves workflow help topic', async () => { const help = await usageForCommand('workflow'); if (help === null) throw new Error('Expected workflow help text'); - assert.match(help, /agent-device help workflow/); - assert.match(help, /type never accepts --settle/); - assert.match(help, /explicit success confirmation is visible, stop/); - assert.match(help, /Use selectors as positional targets/); - assert.match(help, /Do not use CSS selectors/); - assert.match(help, /Snapshot legend:/); - assert.match(help, /@e12 \[button\] label="Add to cart"/); - assert.match(help, /Truncated text\/input previews: do not use get text first/); - assert.match(help, /snapshot -s @e7/); - assert.match(help, /Use plain fill\/type first for ordinary login and form fields/); - assert.match(help, /--delay-ms intentionally paces character entry/); - assert.match(help, /agent-device fill 'id="password"' "\$AD_VAR_PASSWORD" --record-as PASSWORD/); - assert.match(help, /published script contain only \$\{PASSWORD\}/); - assert.match(help, /Do not record passwords, tokens, or other secrets without --record-as/); - assert.match(help, /Read-only visible\/state question: use snapshot\/get\/is\/find/); - assert.match(help, /wait_target_absent means at least one readable capture/); - assert.match(help, /wait_capture_stalled means no readable capture/); - assert.match(help, /wait_deadline_exceeded means a later capture/); - assert.match(help, /wait_landmark_identity_mismatch means a replay destination guard/); - assert.match(help, /wait_stable_timeout means wait stable/); - assert.match(help, /Use snapshot -i only when refs are needed/); - assert.match(help, /install-from-source --github-actions-artifact org\/repo:app-debug/); - assert.match(help, /Discovery is not enough when the task asks to open\/start/); - assert.match(help, /If the task says install, use install/); - assert.match(help, /Do not open artifact paths or invent package ids/); - assert.match(help, /agent-device get attrs @e4/); - assert.match(help, /Ambiguous find: add --first or --last/); - assert.match(help, /report that gap instead of typing\/searching\/navigating/); - assert.match(help, /App-owned action sheets, menus, and camera\/scan screens are normal UI/); - assert.match(help, /wait for a concrete result before returning to chat\/form state/); - assert.match(help, /choose a point near the center of the intended app-owned target/); - assert.match(help, /Avoid screen edges, tab bars, navigation bars, and home indicators/); - assert.match(help, /Android transform injects a geometric two-finger path/); - assert.match(help, /verify semantic app state or coarse per-component effects/); - assert.match(help, /instead of exact numeric deltas/); - assert.match(help, /prefer isolated gesture pan --pointer-count 2, gesture pinch/); - assert.match(help, /gesture pan is one finger by default/); - assert.match(help, /--pointer-count 2 for a parallel two-finger pan/); - assert.match(help, /falls back to the visible snapshot union/); - assert.match(help, /tvOS coordinate pan and fling preserve only the dominant direction/); - assert.match(help, /longpress accepts coordinates, @refs, or selectors/); - assert.match(help, /use help react-native for Metro\/Re\.Pack Fast Refresh/); - assert.match(help, /iOS Allow Paste prompt cannot be exercised under XCUITest/); - assert.match(help, /Empty replacement is not a supported clear-field command/); - assert.match(help, /do not plan fill ""/); - assert.match(help, /To hide the keyboard, use keyboard dismiss/); - assert.match(help, /reports UNSUPPORTED_OPERATION rather than tapping elsewhere/); - assert.match(help, /no tap outside the keyboard can be proven side-effect-free/); + assert.match(help, /^agent-device \S+ — workflow/); + assert.ok( + Buffer.byteLength(help, 'utf8') < 9000, + `workflow help topic should stay close to the compact-card size target, was ${Buffer.byteLength(help, 'utf8')} bytes`, + ); + assert.match(help, /open -> snapshot -i -> settle -> verify -> close loop/); + assert.match(help, /type never takes --settle/); assert.match( help, - /On iOS, if it still returns UNSUPPORTED_OPERATION, both mechanisms were exhausted/, + /Chain confident consecutive steps with &&: press 'label="Search"' --settle && fill 'label="Search"' "query" --settle/, ); - assert.match(help, /On Android, keyboard dismiss first avoids navigation/); - assert.match(help, /use back only when normal back behavior is acceptable/); - assert.match(help, /UNSUPPORTED_OPERATION/); - assert.match(help, /Stateful commands within one session must run serially/); + assert.match(help, /Fall back to one command at a time when a step is uncertain/); + assert.match(help, /never a placeholder \(@ref, @eN, @Label_Name\)/); assert.match( help, - /Do not run open\/press\/fill\/type\/scroll\/back\/alert\/replay\/batch\/close commands in parallel/, + /iOS rejects a stale pinned ref -- refresh with snapshot -i or use a selector/, ); - assert.match(help, /agent-device clipboard write "some text"/); - assert.match(help, /For gesture-heavy iOS simulator proof videos, prefer --hide-touches/); - assert.match(help, /only a means to reveal or reach an expected target/); - assert.match(help, /using the id, selector, or text named by the task/); + assert.match(help, /Known flow: batch \.\/steps\.json \(help scripting\)/); + assert.match(help, /Shapes and platform quirks: help gestures/); + assert.match(help, /Never open artifact paths or invent package ids/); assert.match( help, - /iOS simulator transform uses private XCTest synthesis for a continuous two-finger pan\/scale\/rotation path/, + /Apple CI: prepare ios-runner after boot\/install, before replay\/test \(help prepare\)/, ); - assert.match(help, /Android Gboard handwriting\/stylus UI can capture text/); - assert.match(help, /targetInput\/actualInput details/); - assert.match(help, /Do not keep retrying fill\/type against the same field/); - assert.match(help, /provider-native text injection when available/); - assert.match(help, /Do not switch to raw adb, clipboard, or paste as an agent fallback/); - assert.match(help, /if no URL is provided but a target\/app name is provided, open that target/); - assert.match(help, /localhost\/127\.0\.0\.1\/\[::1\] with a port auto-configure/); - assert.match(help, /Manual adb reverse tcp: tcp: is only needed/); - assert.match(help, /do not stop at the action itself/); - assert.match(help, /do not split clear\/restart/); - assert.match(help, /do not write network log headers/); - assert.match(help, /Web: agent-device uses a managed, pinned agent-browser backend/); + assert.match(help, /Reusable scripts, secret-safe fills, replay repair: help scripting/); + assert.match(help, /snapshot -i gets current interactive refs only/); + assert.match(help, /Legend: @e12 \[button\] label="Add to cart"/); + assert.match(help, /open\/--relaunch clears the stored snapshot outright/); assert.match( help, - /Use --platform web when a browser step belongs inside an agent-device session/, + /A known selector\/label after a mutation is often enough, since interaction commands refresh state internally/, ); - assert.match(help, /use agent-browser directly for standalone web automation/); - assert.match(help, /agent-device web setup/); - assert.match(help, /agent-device web doctor/); - assert.match(help, /agent-device open https:\/\/example\.com --platform web/); - assert.match(help, /agent-device get text @e2 --platform web/); - assert.match(help, /agent-device is visible 'label="Welcome"' --platform web/); - assert.match(help, /agent-device find text "Welcome" exists --platform web/); - assert.match(help, /agent-device close --platform web/); - assert.match(help, /Use agent-browser directly for browser-specific features/); - assert.match(help, /agent-device open exp:\/\/127\.0\.0\.1:8081 --platform ios/); - assert.match(help, /agent-device open "Expo Go" exp:\/\/127\.0\.0\.1:8081 --platform ios/); - assert.match(help, /There is no open-url command/); - assert.match(help, /direct URL open can report success while leaving the runner\/shell focused/); - assert.match(help, /verify with snapshot -i after opening/); - assert.match(help, /snapshot returns a sparse\/AX-unavailable state/); - assert.match(help, /Use plain screenshot, not screenshot --overlay-refs/); - assert.match(help, /retry snapshot -i after reaching another screen/); - assert.match(help, /test \.\/e2e\/maestro --maestro --device udid1,emulator-5554 --shard-all 2/); - assert.match(help, /agent-device open exp:\/\/127\.0\.0\.1:8081 --platform android/); - assert.match(help, /apps lookup misses the project but shows Expo Go\/dev-client/); - assert.match(help, /metro prepare --kind expo/); - assert.match(help, /agent-device prepare ios-runner --platform ios --timeout 240000/); - assert.match(help, /prepare ios-runner builds\/reuses the XCTest runner/); + assert.match(help, /TV\/D-pad focus: help tv/); + assert.match(help, /not bare role keys \(button="Search"\)/); + assert.match(help, /"Selector did not resolve uniquely"/); + assert.match(help, /iOS AX flags are unreliable on deep RN trees/); + assert.match(help, /targetHittable: false plus a hint -- verify or re-target, not a failure/); assert.match( help, - /not a recovery step for "runner already owned by another agent-device daemon"/, + /Empty replacement is not a clear-field command \(do not plan fill ""\)/, ); - assert.match(help, /prepared runner does not keep a live lease/); + assert.match(help, /retry with --delay-ms before clipboard paste/); + assert.match( + help, + /keyboard dismiss taps its own dismiss key when one exists, else UNSUPPORTED_OPERATION/, + ); + assert.match(help, /prefer type "\\n" to submit/); + assert.match( + help, + /iOS paste-prompt limits and Android IME\/handwriting capture quirks: help debugging/, + ); + assert.match(help, /run serially within one session/); + assert.match(help, /Wait failure contract:/); + assert.match(help, /wait_target_absent: a readable capture ran and found no match/); + assert.match(help, /wait_capture_stalled: no readable capture finished before the deadline/); + assert.match(help, /wait_deadline_exceeded: a later capture used the remaining budget/); + assert.match(help, /wait_landmark_identity_mismatch: a replay destination guard/); + assert.match(help, /wait_stable_timeout: wait stable never saw a stable UI/); + assert.match(help, /Ambiguous find: add --first or --last/); + assert.match(help, /macOS context menus are secondary clicks \(help macos\)/); + assert.match(help, /Nearby mutation diff: diff snapshot -i/); + assert.match(help, /initializes the baseline \(zero changes\) instead of failing/); + assert.match( + help, + /confirm the requested end state is actually visible on the current screen, scrolling it into view if needed/, + ); + assert.match(help, /get text alone, or stopping one screen early, is not enough/); + assert.match(help, /Perf\/memory\/log\/network\/trace\/crash: help debugging/); + assert.match(help, /Recording, save-script, batch, replay repair: help scripting/); + assert.match(help, /help react-native for Metro\/Re\.Pack reload/); + assert.match(help, /Lifecycle facts \(trust these instead of probing\)/); + assert.match(help, /open without --relaunch is idempotent-foreground/); + assert.match(help, /already owned by another agent-device daemon/); + assert.match(help, /Env vars: help physical-device/); + assert.match(help, /Escalate:/); + assert.match(help, /help scripting recording, save-script, batch, replay repair/); + assert.match(help, /help gestures multi-touch gesture shapes\/quirks/); assert.match(help, /help react-devtools/); assert.match(help, /help react-native/); assert.doesNotMatch(help, /agent-device react-devtools profile/); + // Deep content moved out of the compact card, not deleted: it now lives in the + // owning sub-topic (see the corresponding topic tests below). + assert.doesNotMatch(help, /prepare ios-runner builds\/reuses the XCTest runner/); + assert.doesNotMatch( + help, + /agent-device fill 'id="password"' "\$AD_VAR_PASSWORD" --record-as PASSWORD/, + ); + assert.doesNotMatch(help, /REPLAY_DIVERGENCE/); + assert.doesNotMatch(help, /gesture transform 200 420 80 -40 2 35 700/); +}); + +test('usageForCommand resolves scripting help topic', async () => { + const help = await usageForCommand('scripting'); + if (help === null) throw new Error('Expected scripting help text'); + assert.match(help, /^agent-device \S+ — scripting/); + assert.match(help, /agent-device open com\.example\.app --relaunch --save-script=screen-x\.ad/); + assert.match(help, /agent-device session save-script/); + assert.match(help, /publishes the sole recorded open through the destination guard/); + assert.match(help, /A second successful open aborts publication/); + assert.match(help, /export AD_VAR_PASSWORD=''/); + assert.match(help, /agent-device fill 'id="password"' "\$AD_VAR_PASSWORD" --record-as PASSWORD/); + assert.match(help, /published script contain only \$\{PASSWORD\}/); + assert.match(help, /Do not record passwords\/tokens without --record-as/); + assert.match(help, /REPLAY_DIVERGENCE with a bounded report/); + assert.match(help, /replay --from --plan-digest /); + assert.match(help, /resume never re-executes skipped steps/); + assert.match(help, /replay \.ad --keep-session/); + assert.match(help, /--update\/-u is a no-op \(ADR 0012\)/); + assert.match(help, /record-and-heal means press the correct control via a blessed @ref/); + assert.match(help, /state-repair means the script is correct but app state is not/); + assert.match(help, /close --save-script\[=\] \(default \.healed\.ad\)/); + assert.match(help, /agent-device batch --steps '\[\{"command":"open"/); + assert.match(help, /Never use args, step positionals, or flags in new batch JSON/); + assert.match(help, /test \.\/e2e\/maestro --maestro --device udid1,emulator-5554 --shard-all 2/); + assert.match(help, /Android adb screenrecord has a 180s limit/); + assert.match(help, /--hide-touches skips that for the fastest raw recording/); + assert.match(help, /trace start \.\/trace\.log, trace stop \.\/trace\.log/); +}); + +test('usageForCommand resolves gestures help topic', async () => { + const help = await usageForCommand('gestures'); + if (help === null) throw new Error('Expected gestures help text'); + assert.match(help, /^agent-device \S+ — gestures/); + assert.match(help, /agent-device gesture pan 200 420 80 -40 700 --pointer-count 2/); + assert.match(help, /agent-device gesture transform 200 420 80 -40 2 35 700/); + assert.match(help, /press --count --jitter-px for tap series/); + assert.match( + help, + /iOS simulator transform\/pinch\/rotate use private XCTest synthesis for a continuous two-finger/, + ); + assert.match(help, /Android transform injects a geometric two-finger path/); + assert.match(help, /verify semantic app state or coarse per-component effects/); + assert.match(help, /tvOS coordinate pan and fling preserve only the dominant direction/); + assert.match(help, /falls back to the visible snapshot union/); + assert.match(help, /Rare iOS accessibility gap/); + assert.match(help, /agent-device click @e66 --button secondary --platform macos/); + assert.match(help, /fixed pixel wheel steps/); }); test('usageForCommand resolves tv help topic', async () => { const help = await usageForCommand('tv'); if (help === null) throw new Error('Expected tv help text'); - assert.match(help, /agent-device help tv/); + assert.match(help, /^agent-device \S+ — tv/); assert.match(help, /agent-device tv-remote press down/); assert.match(help, /agent-device screenshot \.\/tv-focus\.png --overlay-refs/); assert.match(help, /tv-remote longpress select/); @@ -374,7 +396,7 @@ test('usageForCommand resolves tv help topic', async () => { test('usageForCommand resolves web help topic', async () => { const help = await usageForCommand('web'); if (help === null) throw new Error('Expected web help text'); - assert.match(help, /agent-device help web/); + assert.match(help, /^agent-device \S+ — web/); assert.match(help, /agent-device uses a managed, pinned agent-browser backend/); assert.match(help, /agent-device owns command\/session\/replay integration/); assert.match(help, /agent-browser owns browser launch, page control, screenshots/); @@ -412,7 +434,7 @@ test('usageForCommand resolves web help topic', async () => { test('usageForCommand resolves debugging help topic', async () => { const help = await usageForCommand('debugging'); if (help === null) throw new Error('Expected debugging help text'); - assert.match(help, /agent-device help debugging/); + assert.match(help, /^agent-device \S+ — debugging/); assert.match(help, /Use logs when you need the lead-up timeline/); assert.match(help, /relaunches the session app through devicectl process launch --console/); assert.match(help, /Use debug symbols when you have crash\.ips\/crash\.log/); @@ -448,6 +470,11 @@ test('usageForCommand resolves debugging help topic', async () => { assert.match(help, /Treat native perf output as the agent evidence/); assert.match(help, /sizeBytes=5392410/); assert.match(help, /5\.3 MB raw trace stays in the artifact/); + assert.match(help, /iOS Allow Paste cannot be exercised under XCUITest/); + assert.match(help, /prefill with clipboard write "some text"/); + assert.match(help, /Android Gboard handwriting\/stylus UI can capture text/); + assert.match(help, /targetInput\/actualInput details/); + assert.match(help, /Do not keep retrying fill\/type against the same field/); }); test('usageForCommand resolves remote help topic', async () => { @@ -507,7 +534,7 @@ test('usageForCommand resolves remote help topic', async () => { test('usageForCommand resolves physical-device help topic', async () => { const help = await usageForCommand('physical-device'); if (help === null) throw new Error('Expected physical-device help text'); - assert.match(help, /agent-device help physical-device/); + assert.match(help, /^agent-device \S+ — physical-device/); assert.match(help, /Start with Automatic Signing and only these env vars/); assert.match(help, /AGENT_DEVICE_IOS_TEAM_ID=ABCDE12345/); assert.match(help, /AGENT_DEVICE_IOS_BUNDLE_ID=com\.yourname\.agentdevice\.runner/); @@ -518,12 +545,24 @@ test('usageForCommand resolves physical-device help topic', async () => { help, /app inventory, install\/reinstall, logs, performance sampling, recording, deep links, and launch arguments/, ); + assert.match(help, /idempotent-foreground for an already-running app/); + assert.match( + help, + /one simctl launch --terminate-running-process call instead of a separate terminate-then-launch/, + ); + assert.match(help, /AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS/); + assert.match(help, /AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS/); + assert.match( + help, + /a stale iOS runner lease — its owner process dead, or its AGENT_DEVICE_STATE_DIR deleted — is reclaimed automatically/i, + ); + assert.match(help, /genuinely live owner whose state dir still exists still rejects/); }); test('usageForCommand resolves ios-system-ui help topic', async () => { const help = await usageForCommand('ios-system-ui'); if (help === null) throw new Error('Expected ios-system-ui help text'); - assert.match(help, /agent-device help ios-system-ui/); + assert.match(help, /^agent-device \S+ — ios-system-ui/); assert.match(help, /agent-device open com\.apple\.springboard --platform ios/); assert.match(help, /longpress on an empty area of the home screen/); assert.match(help, /discover them from the current snapshot/); @@ -538,7 +577,7 @@ test('usageForCommand resolves ios-system-ui help topic', async () => { test('usageForCommand resolves manual QA help topic', async () => { const help = await usageForCommand('manual-qa'); if (help === null) throw new Error('Expected manual QA help text'); - assert.match(help, /agent-device help manual-qa/); + assert.match(help, /^agent-device \S+ — manual-qa/); assert.match(help, /Execute the script/); assert.match(help, /Run snapshot -i to get current refs/); assert.match(help, /press\/fill\/click\/longpress --settle/); @@ -546,14 +585,14 @@ test('usageForCommand resolves manual QA help topic', async () => { assert.match(help, /use fill --settle to replace/); assert.match(help, /use type only to append to an already-focused field/); assert.match(help, /Do not use placeholders such as @ref/); - assert.match(help, /wait_target_absent means at least one readable capture/); - assert.match(help, /wait_capture_stalled means no readable capture/); + assert.match(help, /wait_target_absent: a readable capture ran and found no match/); + assert.match(help, /wait_capture_stalled: no readable capture finished before the deadline/); }); test('usageForCommand resolves validate help topic', async () => { const help = await usageForCommand('validate'); if (help === null) throw new Error('Expected validate help text'); - assert.match(help, /agent-device help validate/); + assert.match(help, /^agent-device \S+ — validate/); assert.match(help, /validating a code change/); assert.match(help, /Required freshness gate before device verification/); assert.match(help, /For a TypeScript runtime or CLI output change, start with pnpm build/); @@ -563,6 +602,8 @@ test('usageForCommand resolves validate help topic', async () => { assert.match(help, /Do not build the Apple runner for TypeScript-only changes/); assert.match(help, /Use the settled diff as evidence/); assert.match(help, /Close sessions and release leases/); + assert.match(help, /exact key that includes the agent-device package and Xcode version/); + assert.match(help, /Avoid broad restore-key fallbacks/); }); test('usageForCommand resolves macos help topic', async () => { @@ -576,7 +617,7 @@ test('usageForCommand resolves macos help topic', async () => { test('usageForCommand resolves dogfood help topic', async () => { const help = await usageForCommand('dogfood'); if (help === null) throw new Error('Expected dogfood help text'); - assert.match(help, /agent-device help dogfood/); + assert.match(help, /^agent-device \S+ — dogfood/); assert.match(help, /Find user-visible issues from runtime behavior/); assert.match(help, /Severity: critical blocks a core flow\/data\/crashes/); assert.match(help, /Interactive\/behavioral issues need step screenshots/); @@ -648,7 +689,7 @@ test('usageForCommand resolves cdp help topic', async () => { test('usageForCommand resolves react-native help topic', async () => { const help = await usageForCommand('react-native'); if (help === null) throw new Error('Expected react-native help text'); - assert.match(help, /agent-device help react-native/); + assert.match(help, /^agent-device \S+ — react-native/); assert.match(help, /React Native-specific automation hazards/); assert.match(help, /Choose the next help topic/); assert.match(help, /help workflow/); diff --git a/src/cli/parser/cli-help.ts b/src/cli/parser/cli-help.ts index ed24aebfc2..ea16de55fe 100644 --- a/src/cli/parser/cli-help.ts +++ b/src/cli/parser/cli-help.ts @@ -15,6 +15,7 @@ import { type FlagKey, } from '../../cli-schema/command-schema.ts'; import { buildCommandUsage } from '../../cli-schema/usage.ts'; +import { readVersion } from '../../utils/version.ts'; const AGENT_WORKFLOWS = [ { @@ -145,12 +146,12 @@ const EXAMPLE_LINES = [ ] as const; const WAIT_FAILURE_CONTRACT = `Wait failure contract: - Read wait failures from error.details.reason in --json output; do not infer the verdict from the message. - wait_target_absent means at least one readable capture saw no matching target. It includes readableCaptures and waitedMs, and may include currentSurface details. - wait_capture_stalled means no readable capture established an observation before the deadline. It is retriable; retry or use screenshot to inspect the surface. - wait_deadline_exceeded means a later capture consumed the remaining budget after an earlier readable capture; it includes captureTruncated and readableCaptures. - wait_landmark_identity_mismatch means a replay destination guard found the selector but not the recorded target identity. - wait_stable_timeout means wait stable did not observe a stable UI; it is not an element-absence verdict. + Read the verdict from error.details.reason in --json, not the message text. + wait_target_absent: a readable capture ran and found no match. + wait_capture_stalled: no readable capture finished before the deadline -- retriable. + wait_deadline_exceeded: a later capture used the remaining budget after an earlier readable one. + wait_landmark_identity_mismatch: a replay destination guard found the selector but not the recorded identity. + wait_stable_timeout: wait stable never saw a stable UI -- not an absence verdict. `; const HELP_TOPICS = { @@ -213,252 +214,143 @@ Focused compatibility request: ${MAESTRO_COMPATIBILITY_ISSUE_URL}`, summary: 'Normal agent-device bootstrap, exploration, and validation loop', body: `agent-device help workflow -Version-matched operating guide for normal agent-device work. - -Core loop: - Start with the top-level Agent Starting Point for the default settle-first loop. This topic is the full reference for command shapes, refs, selectors, waits, recovery, and platform limits. - If you intentionally skip --settle or use a command that does not support it, verify a mutation with diff snapshot (or diff snapshot -i) instead of a full snapshot: it diffs the rendered snapshot lines against the previous one in this session and prints only what changed. - Once the task's requested end state or an explicit success confirmation is visible, stop; do not tap transient follow-up controls or navigate away only to re-verify. +Command shapes, refs, selectors, waits, recovery, and platform limits for the default open -> snapshot -i -> settle -> verify -> close loop. Command shape: - Plans should use agent-device commands, not raw platform tools, pseudo commands, package-manager aliases, or helper prose. - If the user asks for a command plan, final output should be command lines only: no intro sentence, numbered list, Markdown fence, shell pipe, grep/head/tail helper, or explanatory bullets. - While exploring, do not pipe or redirect agent-device output through jq/grep/head/tail or 2>/dev/null; raw output carries refs, warnings, hints, and diagnostics needed for the next step. - Put subcommand first, then positionals, then flags: - agent-device open com.example.app --session checkout --platform android --relaunch - agent-device record start ./checkout.mp4 --session checkout - Snapshot refs look like @e12. After snapshot -i, use the exact @eN ref from that output. - If the exact ref is not known yet, first output snapshot -i, then use a concrete example shape like press @e12 in the next command; do not write @, @ref, @Label_Name, or @eN placeholders. - Close means agent-device close. App-owned back means back; system back means back --system. - type never accepts --settle: run agent-device type "text", then diff snapshot if verification is needed. - Taps are press or click; tap is an alias for press. On Android TV, tvOS, and Vega OS, read help tv and use tv-remote press up|down|left|right|select to move D-pad/remote focus before activating controls; use tv-remote longpress