fix(desktop): align preview recording cursors and show input feedback

[juliusmarminge]

Preview recordings put the native mouse cursor at the wrong coordinates when the guest is offset or scaled, and leave agent navigation hard to follow. Draw human and agent cursors in guest CSS coordinates during capture, preserve the themed agent cursor with its 150 ms movement, and remove the redundant control pill.

Add independent opt-ins for key badges and mouse press rings under Settings → Integrations → Browser. Both default off. Click rings use the client theme primary color, with the standard T3 blue fallback. A detached compositor adds them only to the saved recording; password keys are excluded in the guest before forwarding. Recordings with both options off retain the native video stream.

Verification

• Focused tests pass across recording lifecycle, input gating, coordinate scaling, compositor cleanup, keyboard privacy, settings, and preview behavior, including a server-restart regression verified to fail before its fix.
• Scoped desktop, web, and contracts typechecks pass. Targeted formatting and lint pass with existing warnings.
• Verified the guest cursor and compositor through the built-in Browser panel against isolated worktree state. The installed desktop runtime predates this change, so the comparison below loads the actual head guest helper into that runtime. It verifies captured pixels, not the new native IPC lifecycle. The agent-run recording supplied by Julius exercises the integrated recorder and is trimmed to its first 19 seconds.
• Native preview recording is desktop-only. Web and mobile decode the new settings with defaults but do not expose native capture.

Settings before / after

Actual base web sources at c14f6015bfe and head web sources at 28dae3cf36e, with the same isolated settings, 1280×800 viewport, and scroll position. Native preview defaults are disabled in the web client.

Before	After

Guest capture before / after

Same page, viewport, and target point. Before, the agent cursor is absent from the captured guest; after, the guest helper places the themed cursor at the target.

Before	After

Agent run

This run predates the theme-color follow-up. The current ring color appears in the recorded-output screenshot below.

Recording-only input feedback

Live preview	Recorded output

Model: GPT-6. Harness: Codex.

Summary by CodeRabbit

• New Features

  • Added optional keyboard-press and mouse-activity overlays to browser recordings.
  • Added separate settings to enable or disable each overlay, disabled by default and available through settings search.
  • Recording overlays show synchronized pointer activity, key labels, and agent interactions.
  • Added privacy safeguards that hide key labels in sensitive fields, including password inputs.
  • Improved cursor visibility and behavior during recording, navigation, and runtime replacement.

• Bug Fixes

  • Improved recording cleanup and input handling after startup failures, navigation, and playback interruptions.
  • Improved overlay alignment, timing, and cleanup during recording playback.

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.5 KiB	13.5 KiB	+17 B (+0.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	7.1 KiB	7.1 KiB	+4 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.5 KiB	+13 B (+0.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	56.2 KiB	56.3 KiB	+44 B (+0.1%)	66.4 KiB	✅
Codex	Live turn messages	9	10	+1 (+11.1%)	21	✅
Claude	Total thread wire	13.5 KiB	13.5 KiB	−23 B (−0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	7.1 KiB	7.1 KiB	−7 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	−16 B (−0.2%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	57.0 KiB	57.0 KiB	0 B (0.0%)	66.4 KiB	✅
Claude	Live turn messages	9	9	0 (0.0%)	21	✅

Baseline: c14f601 · PR result: 253c23b · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 113.9 KiB
• Claude decoded thread snapshot: 114.6 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR introduces a cross-process recording pipeline with new guest cursor rendering, input forwarding, canvas compositing, controller behavior, and user-facing settings. It also adds persisted product settings with new defaults, so the scope and default changes warrant human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: pingdotgg/t3code/.coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 349ba94b-625b-40f3-b531-9356c6eec68e

📥 Commits

Reviewing files that changed from the base of the PR and between 2983a03 and 253c23b.

📒 Files selected for processing (2)

• apps/web/src/browser/recordingCompositor.test.ts
• apps/web/src/browser/recordingCompositor.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 6 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds configurable keyboard and mouse overlays to browser recordings. Preview guests emit validated input events through desktop IPC. The browser recording path composites these events into the captured video. Settings and tests cover configuration, privacy, lifecycle, and cursor behavior.

Changes

Recording input overlays

Layer / File(s)	Summary
Recording contracts and settings packages/contracts/src/*, apps/web/src/components/settings/*	Adds validated recording input types, independent key and mouse settings, settings controls, search entries, restore handling, and tests.
Desktop recording event pipeline apps/desktop/src/ipc/*, apps/desktop/src/preload.ts, apps/desktop/src/preview/Manager.ts	Forwards recording input through IPC. PreviewManager validates, filters, routes, restores, and cleans up recording state.
Guest recording cursor and input capture apps/desktop/src/preview/PickPreload.ts, apps/desktop/src/preview/RecordingCursor.ts, apps/desktop/src/preview/RecordingInput.ts	Adds cursor rendering, controller state, keyboard and pointer capture, key formatting, sensitive-focus checks, and cleanup.
Browser recording compositor apps/web/src/browser/browserRecording.ts, apps/web/src/browser/recordingCompositor.ts, apps/web/src/browser/*test.ts	Composites optional key badges and pointer rings onto the recording stream and disposes compositor resources during failure or finalization.
Recording validation and cursor presentation apps/desktop/src/preview/Manager.test.ts, apps/web/src/components/preview/*	Tests cursor lifecycle, input filtering, automation transitions, compositor behavior, and cursor visibility during active recording.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant RecordingCursor
  participant PickPreload
  participant PreviewManager
  participant DesktopPreviewBridge
  participant BrowserRecording
  participant RecordingCompositor
  participant MediaRecorder
  RecordingCursor->>PickPreload: capture pointer or key input
  PickPreload->>PreviewManager: forward validated recording input
  PreviewManager->>DesktopPreviewBridge: publish tab-scoped input
  BrowserRecording->>DesktopPreviewBridge: subscribe to recording input
  BrowserRecording->>RecordingCompositor: create decorated capture stream
  DesktopPreviewBridge->>RecordingCompositor: deliver matching input
  RecordingCompositor->>MediaRecorder: provide composited stream

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 24 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main changes: preview recording cursor alignment and input feedback.
Description check	✅ Passed	The description explains the changes, motivation, UI updates, verification, and media for interaction changes. It does not use the exact What Changed, Why, and Checklist headings, but the required inf…

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}

[juliusmarminge]

CodeRabbit completed its review of c14f601 → 2983a03 with no actionable comments. Its docstring-coverage warning does not call for code changes here: AGENTS.md asks for comments explaining use and durable constraints, rather than documenting every function. The new guest-coordinate constraint, keyboard privacy boundary, and compositor lifecycle already have focused comments and behavior tests. All running CI jobs pass. Macroscope correctness is green; its approvability check remains neutral and requests human review of the broader recording feature.