Skip to content

refactor: isolate maestro engine package - #1506

Merged
thymikee merged 2 commits into
mainfrom
agent/1478-p2-maestro
Jul 30, 2026
Merged

refactor: isolate maestro engine package#1506
thymikee merged 2 commits into
mainfrom
agent/1478-p2-maestro

Conversation

@thymikee

@thymikee thymikee commented Jul 30, 2026

Copy link
Copy Markdown
Member

Summary

Package Maestro behind the single private @agent-device/maestro root facade.
Keep the root declaration/re-export-only, split implementation by capability, move daemon authority adapters under src/daemon/adapters/maestro, and keep ranking/conformance internals private.
Centralize Maestro-compatible discovery/format routing, require explicit --maestro for YAML, and remove superseded routing.

The follow-up also keeps yaml external as the baseline runtime dependency, removes the broad shared chunk, and consolidates fixture sealing and YAML-path recognition to one implementation each.

Part of #1478

Validation

  • Maestro conformance: 46/46
  • Focused facade/routing/discovery/failure/artifact tests: 49/49
  • pnpm check:tooling
  • Packed CLI smoke: clean install, --help, and --version
  • Size versus base: +5.3 kB raw, +1.8 kB gzip, +1.7 kB tarball
  • 31-run startup: +1.1 ms --version, +0.8 ms --help
  • pnpm check:affected --run: format/lint/typecheck/layering/Fallow/MCP/build and 5,177 tests passed; five unrelated Android subprocess timeouts in two files then passed isolated 25/25

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.90 MB 1.91 MB +5.2 kB
JS gzip 609.8 kB 611.6 kB +1.7 kB
npm tarball 727.6 kB 729.3 kB +1.7 kB
npm unpacked 2.55 MB 2.55 MB +5.3 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.3 ms 28.1 ms +0.8 ms
CLI --help 58.3 ms 58.1 ms -0.2 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/internal/daemon.js +27.0 kB +8.0 kB
dist/src/session.js -21.7 kB -6.0 kB
dist/src/cli.js -18 B -134 B
dist/src/sdk-selectors.js -90 B -30 B
dist/src/viewport-dimension.js -7 B -15 B

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-30 18:59 UTC

Copy link
Copy Markdown
Member Author

Reviewed at 4e90f3dd. The extraction is solid and the hard boundary work is done correctly. One substantive concern about façade width, plus two smaller notes.

Verified against the P2 brief

I checked each "Done" criterion mechanically rather than by reading the summary:

  • packages/maestro imports no root src/, daemon, platform, provider, or native replay — grep returns nothing. package.json depends only on @agent-device/contracts, @agent-device/kernel, and yaml, and exports only .. ✅
  • .ad never enters Maestro — no .ad reference anywhere under packages/maestro/src. ✅
  • runTypedReplayIfNeeded deleted — gone from the tree, not shimmed. ✅
  • Format routing is a pure resolversrc/replay/format.ts is 17 lines, extension + explicit backend only, no content probing and no cross-engine fallback. The doc comment states both non-goals. ✅
  • Discovery unifiedsrc/daemon/replay-source-discovery.ts is the single expansion implementation, and session-test-discovery.ts drops 182 lines to consume it plus resolveReplayFormat at :53 and :142. ✅
  • Typed failure identity from outcomessession-replay-maestro-failure.ts:1 now imports MaestroFailedAction from the package instead of rebuilding it from observer state, and the file sheds most of its former weight. ✅

Also worth saying: splitting the 314-line runtime port into daemon-runtime-port.ts + daemon-runtime-tap.ts is a good call, not scope creep — the old src/compat/maestro/daemon-runtime-port.ts carried 17 tap-related references in one file.

Main concern: the façade exports four of the six things the brief says stay private

The brief is specific: "Add one Maestro façade and route non-test consumers through it. Keep engine IR, parser, plan, ranking, expression state, and compatibility policy private."

packages/maestro/src/index.ts currently exports, with the consumers I traced:

export brief category consumer
rankMaestroFailureCandidates ranking src/daemon/handlers/session-replay-maestro-failure.ts — a daemon handler, not an adapter
MAESTRO_COMPATIBILITY_PRESETS, MAESTRO_DEFAULT_SETTLE_TIMEOUT_MS compatibility policy 4 files under src/daemon/adapters/maestro/
SUPPORTED_MAESTRO_COMMAND_NAMES, parseMaestroConformanceSource parser scripts/maestro-conformance/verify.ts only
MaestroPlatform, MaestroSelector (from internal/program-ir.ts) engine IR type-only

These aren't equivalent and I'd treat them differently:

  • rankMaestroFailureCandidates is the one I'd push back on. Ranking is named explicitly as private, and its consumer is a daemon handler rather than an adapter — so the export exists to serve daemon-side failure presentation. Either the ranking call belongs inside executeMaestroFlow's outcome (so the daemon receives already-ranked candidates and never invokes the ranker), or session-replay-maestro-failure.ts moves under src/daemon/adapters/maestro/ where engine-adjacent knowledge is expected.
  • The conformance exports are the weakest justification. SUPPORTED_MAESTRO_COMMAND_NAMES and parseMaestroConformanceSource exist solely for a script under scripts/. That's #1478's "expose implementation files merely to make imports compile", and a subpath isn't the escape hatch either — the issue reserves focused subpaths for leaf vocabulary packages and says the deep Maestro/replay-test/ad-replay packages export only their façade at .. Moving the conformance harness inside the package would keep the parser private and cost nothing externally.
  • The policy constants and IR types are the most defensible, since the adapters genuinely need settle timeouts and selector/platform vocabulary to implement MaestroRuntimePort. But if they stay, it's worth a comment in index.ts recording why those specific ones are public, so the next extraction doesn't read the current surface as licence to widen further.

None of this is a STOP condition — there's no shared replay VM, no parser probing, no fallback engine, and DaemonRequest hasn't been promoted into contracts. It's a "does the façade match its stated contract" question, and right now it doesn't.

Two smaller notes

1. Bare .yaml routes to the native engine. resolveReplayFormat('flow.yaml', undefined) returns 'ad', which matches the brief exactly (".yaml/.yml means Maestro only when the Maestro backend was explicitly requested"). But it means a user running replay flow.yaml without --maestro hands a YAML file to the .ad parser. Could you confirm the resulting error names the missing --maestro flag rather than surfacing a raw parse failure? The routing is right; I'm only asking about the diagnostic.

2. Discovery ownership now overlaps P3. session-test-discovery.ts:5 imports inspectMaestroFlow from the package for Maestro titles. That's correct here, but #1478 also says replay-test owns discovery policy — so P3 will need a split rather than a move. The workable line looks like: host owns path expansion and per-file manifest inspection (this PR's replay-source-discovery.ts + resolveReplayFormat), replay-test owns filtering, ordering, skip decisions and skip-message text. Flagging so P3 doesn't try to move what you've just correctly placed in the daemon.

Testing

Test topology mirrors the moves, the conformance oracle is preserved, and android-test-suite.test.ts being substantially rewritten (669 lines) is expected given routing changed underneath it. Nothing here looks like coverage lost to the move.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Correcting my earlier review: I raised the façade width as an architecture-contract question and missed that it has already caused a measured bundle and startup regression. The Bundle Size check is informational, so it stayed green.

The regression

From the size report on 4e90f3dd:

Metric Base Current Diff
JS raw 1.90 MB 2.02 MB +116.1 kB
JS gzip 609.8 kB 645.0 kB +35.2 kB
npm tarball 727.6 kB 762.6 kB +35.0 kB
CLI --help 61.0 ms 68.1 ms +7.1 ms (~12%)

Chunk movement tells the story:

Chunk Raw diff
dist/src/src.js +181.7 kB
dist/src/session.js −35.2 kB
dist/src/internal/daemon.js −18.9 kB
dist/src/cli.js −8.4 kB
dist/src/cli-help.js −1.8 kB

Roughly 64 kB left chunks that were loaded lazily and ~182 kB landed in the shared chunk every entrypoint evaluates. --help now pays for the Maestro engine and yaml, which it has no reason to touch. --version is unchanged (−1.1 ms), consistent with it short-circuiting before the shared chunk matters.

This is a regression against a stated non-negotiable, not just a size preference — #1478 lists "lazy platform loading" in both the compatibility rules and the success metrics.

Why it happens

packages/maestro/src/index.ts is an implementation module rather than a re-export barrel:

  • 17 static module-scope imports covering the parser, IR, runtime port, execution, conformance normalization, and compatibility policy;
  • three runtime definitions in the file itself — MAESTRO_CONFORMANCE_CONSTANTS (:134), parseMaestroConformanceSource (:143, which calls parseMaestroProgram), and createMaestroRuntimePort (:155, a real function body).

So any import from @agent-device/maestro — including a type-only one that should erase — evaluates the entire engine graph. yaml is statically imported by six internal modules (program-ir-parser.ts, program-ir-values.ts, program-ir-command-parser.ts, program-ir-flow-parser.ts, program-ir-gesture-parser.ts, export-yaml.ts), all reachable from that graph, so it comes too.

This is the same root cause as my façade-width note, which I under-called. The conformance exports I described as "the weakest justification" are exactly what pins the parser into the static graph.

Suggested direction

  1. Make index.ts a pure re-export barrel. Move createMaestroRuntimePort's body to internal/runtime-port-factory.ts and re-export it; the façade should contain no logic of its own.
  2. Get the conformance surface out of the production entry. MAESTRO_CONFORMANCE_CONSTANTS, parseMaestroConformanceSource, and SUPPORTED_MAESTRO_COMMAND_NAMES serve only scripts/maestro-conformance/. Moving that harness inside the package removes the parser from the public graph entirely, which is also what my earlier note recommended on contract grounds.
  3. Keep the engine behind a dynamic import at the daemon adapters, so it loads when a Maestro replay actually runs rather than at CLI start. refactor: isolate Limrun construction dependencies #1502 (W1c) has a good pattern for this, including a module-load regression test that counts mocked-module factory invocations to prove construction doesn't pull the helpers. The same test shape would pin --help against re-acquiring the engine.
  4. Consider making the size check gate rather than inform, at least on a raw-bytes or startup-ms threshold. This regression was visible in CI the whole time and passed, and the remaining extractions (P3, P5, W1b) will each move code across chunk boundaries in the same way. A ratchet here would catch the next one automatically.

Verification for whichever fix lands: dist/src/src.js should return to roughly its base size, --help to ~61 ms, and yaml should not appear in the static import header of the shared chunk.

Apologies for the miss on the first pass — the façade observation was right there and I did not follow it to its runtime cost.


Generated by Claude Code

@thymikee
thymikee force-pushed the agent/1478-p2-maestro branch from 4e90f3d to 901b7aa Compare July 30, 2026 17:58
@thymikee

Copy link
Copy Markdown
Member Author

Blocking follow-up resolved at 901b7aa80b (rebased onto current origin/main).

Review resolutions:

  • packages/maestro/src/index.ts is now declaration/re-export-only. Execution/inspection, export, runtime-port construction, and support vocabulary live in focused internal facade modules; no public subpaths were added.
  • Conformance-only parser/canonicalization/constants moved under packages/maestro/test/conformance and no longer widen the production facade.
  • Ranking policy and types remain private. The daemon consumes a failure-only package capability through src/daemon/adapters/maestro/failure-snapshot.ts; normal target resolution carries no candidate/reporting data.
  • The exported runtime adapter policy contains only values used by real daemon adapters. Parser, plan, expression, ranking, and broad compatibility-policy state stay private.
  • Bare .yaml/.yml replay without explicit Maestro routing now returns the clear --maestro diagnostic; there is no content probing or fallback.
  • yaml stays a baseline-compatible root runtime dependency and is external in emitted production chunks. The narrow Fallow exception documents that package source owns the import while the published root artifact retains it.
  • sideEffects: false is safe because the private package has no registration/bootstrap side effects or observable top-level mutation; its root-facade test enforces the declaration/re-export-only shape.
  • Fixture sealing now has one package-owned implementation used by verification and regeneration. The duplicate script helper and stale Fallow export exception were deleted.
  • Test artifact handling now reuses central isMaestroYamlPath; the adjacent .yaml/.yml predicate was deleted.

Bundle/package A/B:

Metric Delta
JS raw +5.3 kB
JS gzip +1.8 kB
npm tarball +1.7 kB
CLI --version median, 31 runs +1.1 ms
CLI --help median, 31 runs +0.8 ms

dist/src/src.js is gone, dist/src/script.js is narrow, CLI/help do not import the Maestro parser chunk, and the daemon chunk retains the external yaml import. A clean packed install runs --help and --version.

Validation:

  • Maestro conformance: 46/46
  • Focused facade/routing/discovery/failure/artifact tests: 49/49
  • Tooling gate: green
  • Affected gate: format/lint/typecheck/layering/Fallow/MCP/build and 5,177 tests passed before five unrelated Android subprocess timeouts in two files
  • Both timeout-affected files passed isolated: 25/25, confirming contention

The PR remains draft for independent review.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed 901b7aa: clean P2 boundary/routing cutover. Maestro now exposes only its root facade; engine/parser/plan/ranking remain private, daemon authority stays in src/daemon/adapters/maestro, .ad remains native under --maestro, and YAML requires explicit --maestro. The central discovery resolver, failure-outcome provenance, conformance, layering, replay-compat, coverage, and bundle-size gates are green. Keep this draft/unlabeled until the exact-head package/device checks finish and Android+iOS Maestro-flow evidence for the moved adapter is attached. Residual risk: no separately authorized cross-vendor pass.

@thymikee

Copy link
Copy Markdown
Member Author

Exact-head follow-up for 901b7aa80b5681e9587cc98901b3430aa142f2ba.

Live Maestro adapter evidence

iOS — iPhone 17 Pro simulator (C25DBB5B-9254-4293-A8D5-2785C78DE03A)

node bin/agent-device.mjs \
  --session pr1506-maestro-ios \
  --state-dir /private/tmp/agent-device-pr1506-ios \
  test examples/test-app/maestro/checkout-form.yaml \
  --maestro \
  --platform ios \
  --udid C25DBB5B-9254-4293-A8D5-2785C78DE03A \
  --metro-port 8081 \
  --artifacts-dir /private/tmp/pr1506-maestro-ios-artifacts \
  --timeout 180000

Result:

✓ Checkout form 45.0s
Test summary: 1 passed (1) in 45.0s

This is the unmodified repository flow. It exercised Maestro YAML discovery/parsing, launchApp, assertions, file runFlow, platform conditions, scrolling, selectors, text entry, swipe, and repeat control flow through the moved daemon adapter. Preflight snapshot showed the expected Agent Device Tester surface.

Android — Pixel 9 Pro XL API 37 emulator (emulator-5554)

The repository release test app was installed so launchApp(clearState: true) exercised the app directly rather than Expo Dev Launcher state. The focused evidence flow exercised launchApp with clear state, text/id assertions, scrollUntilVisible, tapOn, and swipe:

appId: com.callstack.agentdevicelab
name: Android Maestro adapter evidence
onFlowStart:
  - launchApp:
      clearState: true
  - assertVisible: Agent Device Tester
onFlowComplete:
  - assertVisible: Delivery choices
---
- scrollUntilVisible:
    element:
      id: home-open-form
- tapOn:
    id: home-open-form
- assertVisible: Checkout form
- swipe:
    start: 50%, 75%
    end: 50%, 35%
    duration: 300
- scrollUntilVisible:
    element:
      text: Delivery choices
- assertVisible: Delivery choices
node bin/agent-device.mjs \
  --session pr1506-maestro-android-evidence \
  --state-dir /private/tmp/agent-device-pr1506-android-release \
  test /private/tmp/pr1506-maestro-android-evidence.yaml \
  --maestro \
  --platform android \
  --artifacts-dir /private/tmp/pr1506-maestro-android-evidence-final-artifacts \
  --timeout 180000

Result:

✓ Android Maestro adapter evidence 5.77s
Test summary: 1 passed (1) in 5.77s

Preflight snapshot showed Agent Device Tester and confirmed androidSnapshot.backend=android-helper, helperVersion=0.20.3, API version 2, persistent-session transport.

All verification sessions auto-finalized or were explicitly closed; session lists were empty afterward. Metro was stopped and the emulator booted for this check was shut down.

CI

All exact-head checks are now green. The initial iOS fixture E2E failure reported structured wait_capture_stalled while waiting for Agent Device Tester; it was unrelated to the Maestro extraction and passed on the exact-head rerun. Passing iOS smoke rerun.

No source changes or push were needed. The PR remains draft at the same head.

@thymikee
thymikee marked this pull request as ready for review July 30, 2026 18:58
@thymikee
thymikee merged commit 0e51007 into main Jul 30, 2026
33 of 34 checks passed
@thymikee
thymikee deleted the agent/1478-p2-maestro branch July 30, 2026 18:58
thymikee pushed a commit that referenced this pull request Jul 31, 2026
The scheduler called `isRequestCanceled(requestId)` in five places. That both
reaches a request-global registry and forces the scheduler to name a daemon
request id as the cancellation key — neither survives the move into
`packages/replay-test`.

The host now binds the predicate to its own request and passes
`isCanceled: () => boolean`. The scheduler asks a question it is entitled to
ask and learns nothing about how cancellation is tracked. `shouldStopReplayTestExecution`
takes the capability rather than a request id, so no scheduler function threads
a daemon identifier for this purpose any more.

`session-test-attempt.ts` and `session-test.ts` no longer import
`request/cancel.ts` at all. It remains in `session-test-runtime.ts`, which does
something different — `registerRequestAbort`, `markRequestCanceled` and the
parent-abort relay are cancellation *binding*, which the brief assigns to the
daemon adapter, so that split is its own step.

Behavior preserved: both pinned reporter characterizations pass unmodified,
32/33 across the five scheduler suites. The one failure is the pre-existing
P2/#1506 discovery-ordering regression, unrelated and untouched here.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8
thymikee pushed a commit that referenced this pull request Jul 31, 2026
The scheduler called `isRequestCanceled(requestId)` in five places. That both
reaches a request-global registry and forces the scheduler to name a daemon
request id as the cancellation key — neither survives the move into
`packages/replay-test`.

The host now binds the predicate to its own request and passes
`isCanceled: () => boolean`. The scheduler asks a question it is entitled to
ask and learns nothing about how cancellation is tracked. `shouldStopReplayTestExecution`
takes the capability rather than a request id, so no scheduler function threads
a daemon identifier for this purpose any more.

`session-test-attempt.ts` and `session-test.ts` no longer import
`request/cancel.ts` at all. It remains in `session-test-runtime.ts`, which does
something different — `registerRequestAbort`, `markRequestCanceled` and the
parent-abort relay are cancellation *binding*, which the brief assigns to the
daemon adapter, so that split is its own step.

Behavior preserved: both pinned reporter characterizations pass unmodified,
32/33 across the five scheduler suites. The one failure is the pre-existing
P2/#1506 discovery-ordering regression, unrelated and untouched here.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8
thymikee pushed a commit that referenced this pull request Jul 31, 2026
The scheduler called `isRequestCanceled(requestId)` in five places. That both
reaches a request-global registry and forces the scheduler to name a daemon
request id as the cancellation key — neither survives the move into
`packages/replay-test`.

The host now binds the predicate to its own request and passes
`isCanceled: () => boolean`. The scheduler asks a question it is entitled to
ask and learns nothing about how cancellation is tracked. `shouldStopReplayTestExecution`
takes the capability rather than a request id, so no scheduler function threads
a daemon identifier for this purpose any more.

`session-test-attempt.ts` and `session-test.ts` no longer import
`request/cancel.ts` at all. It remains in `session-test-runtime.ts`, which does
something different — `registerRequestAbort`, `markRequestCanceled` and the
parent-abort relay are cancellation *binding*, which the brief assigns to the
daemon adapter, so that split is its own step.

Behavior preserved: both pinned reporter characterizations pass unmodified,
32/33 across the five scheduler suites. The one failure is the pre-existing
P2/#1506 discovery-ordering regression, unrelated and untouched here.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8
thymikee added a commit that referenced this pull request Jul 31, 2026
* refactor(replay-test): source the manifest device vocabulary from the kernel

`session-test-types.ts` reached `ReplayScriptMetadata['platform']` and
`['target']` through `replay/script.ts` — the native `.ad` engine. A
format-neutral scheduler must not name an engine module, and P5 relocates that
engine into `packages/ad-replay` regardless, so the import had to go before the
scheduler can move.

Both members already resolve to neutral kernel types
(`Exclude<PlatformSelector, 'web'>` and `DeviceTarget` from
`@agent-device/kernel/device`), so this re-sources them directly and the
manifest shape is unchanged. Only the import direction differs.

First increment of P3b; the scheduler still has request-global, engine and
daemon imports to port before the physical move.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): inject the progress sink instead of reading a request global

The scheduler called `emitRequestProgress` in eight places, which reads a sink
out of a request-global `AsyncLocalStorage`. That is ambient authority a
format-neutral scheduler cannot hold once it lives in `packages/replay-test`,
and #1505 recorded it as a shrink-only R10 entry.

The host now injects the capability through the existing
`ReplayTestRuntimeDependencies` seam established in P3a, so no new seam is
invented. `session-replay.ts` supplies `emitProgress: emitRequestProgress`;
`src/request/progress.ts` keeps the sink and its AsyncLocalStorage binding for
every other caller.

The port is deliberately narrower than `RequestProgressSink`: it accepts only
`ReplayTestSuiteProgressEvent | ReplayTestProgressEvent`, so the scheduler is
not handed the ability to emit `CommandProgressEvent`.

Authority narrows again one hop down: `runReplayTestAttempt` spread the whole
dependency bag but uses three of its members and never publishes progress, so
it now takes `Pick<..., 'runReplay' | 'cleanupSession' | 'finalizeAttempt'>`.
That is why no runtime test fixture needed changing — the attempt runtime never
gained the capability in the first place.

Also drops the last two `replay/script.ts` type references from
`session-test-runtime.ts`, so the engine import is gone from that file too.

Reporter contract preserved: `session-test-reporter-values.test.ts` and
`session-test-reporter-values-maestro.test.ts` both pass unmodified (27 tests
green across the five scheduler suites). Typecheck clean.

Remaining scheduler boundary for P3b: `request/cancel.ts`, `replay/format.ts`,
`replay/script.ts` in discovery, `session-store.ts`, `daemon/types.ts`,
`replay-source-discovery.ts`, `core/dispatch*`, `utils/diagnostics.ts`.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): ask the host whether the suite is canceled

The scheduler called `isRequestCanceled(requestId)` in five places. That both
reaches a request-global registry and forces the scheduler to name a daemon
request id as the cancellation key — neither survives the move into
`packages/replay-test`.

The host now binds the predicate to its own request and passes
`isCanceled: () => boolean`. The scheduler asks a question it is entitled to
ask and learns nothing about how cancellation is tracked. `shouldStopReplayTestExecution`
takes the capability rather than a request id, so no scheduler function threads
a daemon identifier for this purpose any more.

`session-test-attempt.ts` and `session-test.ts` no longer import
`request/cancel.ts` at all. It remains in `session-test-runtime.ts`, which does
something different — `registerRequestAbort`, `markRequestCanceled` and the
parent-abort relay are cancellation *binding*, which the brief assigns to the
daemon adapter, so that split is its own step.

Behavior preserved: both pinned reporter characterizations pass unmodified,
32/33 across the five scheduler suites. The one failure is the pre-existing
P2/#1506 discovery-ordering regression, unrelated and untouched here.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): drop the dead request-tracking call from attempt ids

`buildReplayTestAttemptRequestId` wrapped its template in
`resolveRequestTrackingId`, pulling `request/cancel.ts` into the scheduler.

That wrapper substitutes a generated id only when its first argument is an
empty string. The template here always contains `:test:`, so it is never empty
and the wrapper always returned it unchanged — the call is unreachable in this
path. Probed all three shapes (explicit request id, suite-id fallback with a
shard, and degenerate empty inputs); every one returns the template verbatim.

Removing it takes `request/cancel.ts` out of discovery without altering a
single produced id. The scheduler mints attempt identity itself, which is what
the brief asks for.

Evidence the ids are byte-identical: the pinned reporter characterizations
assert exact session strings such as
`default:test:suite-reporter:1-02-retry:attempt-1` and pass unmodified —
30 tests green across the reporter, suite and discovery suites.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): move cancellation binding and diagnostics to the host

`session-test-runtime.ts` held the last two request-globals in the scheduler:
`request/cancel.ts` (registerRequestAbort, markRequestCanceled,
clearRequestCanceled, plus the parent-abort relay) and `utils/diagnostics.ts`.

These are different in kind from the earlier ports. The brief gives the daemon
adapter the job of mapping an attempt id to daemon request identifiers and
*binding cancellation*, while timeout policy stays scheduler-owned. So the
scheduler now receives a per-attempt capability with exactly two verbs —
`cancel()` on timeout and `release()` when the attempt settles — and every
registry interaction, including `relayReplayTestAbortFromParent`, moved to
`session-replay.ts` next to the rest of the adapter.

Diagnostics became a narrow publish capability for the same reason:
`emitDiagnostic` reads a request-global scope. The level vocabulary is spelled
out at the seam rather than imported, so nothing engine- or daemon-shaped
crosses it.

The runtime fixtures drive the real exported host binding rather than a stub.
They assert cancellation through `isRequestCanceled`, and a stubbed binding
would have kept those assertions passing while proving nothing.

24 tests green across the runtime, suite and both reporter characterizations,
which pass unmodified. Typecheck, lint and oxfmt clean.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): split discovery into host inspection and scheduler policy

discoverReplayTestEntries expanded paths, read every file, and called both
engines — readReplayScriptMetadata for .ad, inspectMaestroFlow for Maestro —
plus resolveReplayFormat to choose between them. Four imports a format-neutral
scheduler cannot hold.

Inspection is now the host's discoverSources capability. What stays in the
scheduler is the genuinely neutral half: which sources a --platform filter
runs, which it skips and with what message, and the empty-suite error.

The manifest carries exactly the four fields the scheduler consumes (platform,
target, retries, timeoutMs) plus the reporter's title, per the brief's
instruction not to add more without a demonstrated call site.

The platform tag is what removes the last format leak. The filter used to ask
resolveReplayFormat(...) === 'maestro' to decide whether a missing platform was
disqualifying. It now reads a tag: caller-bound means the invocation supplies
the platform, unspecified means the source declared none. Maestro is what
caller-bound looks like from the scheduler's side, and the format cannot be
recovered from it.

Discovery tests drive the real inspection capability, writing actual .ad and
Maestro sources — a stubbed host half would have kept them green while proving
nothing about the composition they exist to pin.

35 tests green across discovery, suite, runtime and both reporter
characterizations, which pass unmodified. The Maestro one is the direct check
that titles still flow, since they now arrive via the manifest.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): build attempt ids from named segments; trim comments

Review feedback on the attempt-id builder: the comment explained a deletion
that git already records, and it sat above an opaque template literal.

The id is now a segment list joined on ':', so its shape is readable without
prose. Output is byte-identical — the reporter characterizations assert exact
session and attempt strings and pass unmodified.

Applied the same standard to four other docblocks in this PR that narrated
what the code used to do rather than what it does. The durable 'why' stays:
which side of the seam owns what, and why the vocabulary is neutral. The
migration history goes, since git carries it and these docblocks will outlive
the migration.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): move shard device binding to the host

buildReplayTestShardPlan called listDeviceInventory to discover what to shard
across, and buildReplayTestShardFlags constructed daemon CommandFlags for the
nested request. Inventory enumeration, allowlists, simulator set paths,
explicit --device selectors and the too-few-devices error are host concerns;
what is scheduler-owned is deciding how many shards exist and which entries
each one runs.

The scheduler now receives resolved shard targets through a capability. The
target is neutral: id and name for session labels and progress metadata, plus
platform and target, which are already kernel vocabulary. DeviceInfo no longer
crosses into scheduling.

One behavior note: an explicit --device selector could in principle name a web
target, which is not a shardable device. That is now rejected with INVALID_ARGS
rather than widening the neutral platform vocabulary to carry something the
scheduler can never run. Implicit selection already filtered to mobile.

919 of 920 handler tests pass. The one failure, session-test-runner.test.ts
'binds each replay script to its declared platform metadata', fails identically
on clean origin/main in this container and is unrelated: directory discovery
walks with opendirSync/readSync and directory results are deduped but not
sorted, while glob results are sorted, so suite order is filesystem-dependent.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(replay-test): extract packages/replay-test behind a façade

Completes the P3b extraction. The scheduler, attempt runtime, discovery
policy, sharding distribution, artifacts and neutral types now live in
packages/replay-test/src/internal/, with one package-root export.

The façade takes a neutral ReplayTestSuiteRequest and returns a tagged
ReplayTestSuiteOutcome. DaemonRequest, DaemonResponse and CommandFlags no
longer reach the scheduler; the adapter translates flags and meta in, and the
outcome back to a daemon response. Eight flags were read by the scheduler and
each became a field it owns.

Host work moved to daemon adapters: source inspection (both engines and format
routing), shard device binding and shard-flag parsing, and artifacts-dir home
expansion, which is why the package can resolve paths without SessionStore.

The one remaining shared concern was the timing trace: the host writes video
lifecycle events into the same trace the scheduler owns. Rather than export a
writer from the façade, each attempt hands the host an appendTimingEvent
closure, so the trace format stays private and the authority is scoped to that
attempt.

Tests mirror the topology. Discovery tests split along the seam they now
cross: ordering, traversal and routing are pinned host-side against real files,
filtering policy is pinned in the package against fake sources. The runtime
tests assert the scheduler's cancellation obligation (cancel once on timeout,
always release) against a recording binding, and a new daemon test pins the
adapter's half — registry entries, the parent-abort relay, and detach on
release — so that coverage moved rather than disappeared.

R10 retargeted to packages/replay-test/src/ and the zone ranked alongside
maestro. R11 confirms zero root-src imports from the package.

914 of 915 handler and package tests pass. The one failure,
session-test-runner 'binds each replay script to its declared platform
metadata', fails identically on clean main here: directory discovery walks with
opendirSync and dedupes without sorting, while globs sort, so suite order is
filesystem-dependent.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* refactor(daemon): simplify replay-test request translation

Fallow flagged toReplayTestSuiteRequest at 14 cyclomatic in 18 lines. The
branches were self-inflicted: every req.flags?.x is one, and each optional
field was written as a conditional spread to avoid setting an undefined key.

exactOptionalPropertyTypes is not enabled, so assigning undefined to an
optional field is equivalent and the spreads bought nothing. Destructuring
flags once and extracting two flag readers removes most of the rest.

One correctness note on the simplification itself: the first version used
`artifactsDir && expandHome(...)`, which returns '' for an empty-string flag
where the previous code called expandHome(''). Replaced with an explicit
undefined check so the empty-string path is unchanged.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* test(live): share the replay test-suite harness across iOS and Android

Both live journeys invoked the public test command and then re-derived the
same value-contract assertions by hand — suite totals, per-script status,
replay counts, non-empty JUnit. Those are claims about the published suite
result and are identical on every platform, and they had already drifted: iOS
iterated with readReplayCommands inline, Android cast data.tests at the call
site.

The shared helper owns exactly that boundary. It takes the caller's runStep
rather than binding a context type, so it is not a platform-configured runner
and cannot template a platform's journey.

Everything a platform genuinely differs on stays with the caller: which
scripts run, the retry policy (iOS 2, Android none — itself a claim worth
keeping), which commands each script exercises, and the behavioral evidence.
Both callers keep every verify* call they had.

67 lines removed, 15 added.

Residual risk: this container has no iOS or Android devices, so the live suites
could not be executed here. Typecheck and lint pass; the harness needs a run on
real targets before the claim that behavior is unchanged is evidence rather
than inference.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* test: pin directory enumeration in the platform-binding suite test

The test wrote two scripts into a temp directory and assumed discovery would
return them in creation order. Directory expansion deliberately preserves
filesystem order to match Maestro — only glob expansion sorts, and 'preserves
Maestro directory filesystem order' pins that with a mocked opendirSync. So the
ordering contract is correct; this test's assumption about enumeration was not.

It passes on CI, where small directories usually enumerate in creation order,
and fails on filesystems that do not — identically on clean main, where the
platform-to-script binding appears reversed.

Pinning enumeration the way the discovery tests already do keeps the subject
intact (each script binds to ITS declared platform, and session numbering
follows discovery order) without depending on the host filesystem. The fs
import became a default import because vi.spyOn cannot redefine an ESM
namespace export.

915 of 915 handler and package tests now pass here.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* test: scope the enumeration spy and restore it in a finally

The spy I added restored only on the happy path and asserted on its argument
inside the mock implementation. Either would misfire for anything else sharing
the worker: an assertion thrown from inside fs, or a leaked global opendirSync,
surfaces as a worker crash with no failed test rather than a readable failure.

It now delegates to the real implementation for any directory but this suite's
own, and restores in a finally.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* fix(replay-test): put package tests where they are actually run

Review found the moved package tests were neither executed nor typechecked.
They sat under packages/replay-test/test/, but vitest's unit-core lane includes
packages/*/src/**/*.test.ts, and neither the root nor the package tsconfig
covers a top-level test directory. A plain unit-core run discovered zero files
under the package.

That is why they looked green: my earlier runs passed those paths explicitly on
the command line, which masked that the default run skipped them. The count is
the proof — 550 files/4741 tests before, 553/4753 now, and the delta is exactly
the three files and twelve tests that were being skipped.

The runtime test also imported runReplayTestAttempt from the package specifier,
which the facade does not export. It would have failed the moment it was
discovered. It now imports internally, like the rest of the internal tests.

Also removed replayTestAttemptFailure from the facade: zero consumers outside
the package, so exporting it widened the boundary for nothing. P3 asks for a
one-function facade.

553 test files and 4753 tests pass; lint and the layering guard are clean.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* style: format the facade after removing the unused export

A scripted edit removed the export line but left a stray blank line; oxfmt was
not re-run on that file afterward, so Lint & Format caught what pnpm lint
alone does not.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

* fix(replay-test): typecheck the package and fix a type-only import

Review found the moved package tests were transpiled by vitest but never
typechecked: the root typecheck script builds six packages via tsc -b and
packages/replay-test was not among them, so its tsconfig was never used.

That hid a real TS2459. session-test-runtime.test.ts imported
ReplayTestAttemptOutcome from ../session-test-runtime.ts, which imports that
type but does not re-export it. It now imports from ../session-test-types.ts,
where the type is defined.

Adding the package to the tsc -b list closes the gap. Verified empirically
rather than assumed: planting a string-to-number error in a package test makes
typecheck fail, and removing it makes it pass. This is the second finding of
the same shape on this PR — first the tests were not discovered by vitest, now
they were not covered by typecheck — so the gate was confirmed to reach the
files rather than trusted to.

12 package tests pass, lint, format and the layering guard are clean, and
typecheck is clean with the package included.

Refs #1478

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant