munim-computer-use

Download  •  Read the Documentation  •  Report Issues  •  munimtech.com/computer-use

Follow Munim Technologies

   

Introduction

Computer Use is an open-source MCP server — a computer-use agent (CUA) backend — that lets any coding agent use your computer the way a person does. It reads the screen through accessibility trees, clicks and types in the background so your mouse stays yours, shows an agent pointer where it is working, zooms in on small text, and drives tabs in your signed-in Chrome — on macOS, Windows and Linux.

Works with Claude Code, Codex, Cursor and MT Code, or any other MCP client, with any model — no vision model is required for interaction.

Built by Munim Technologies as the Computer Use engine of MT Code, and published here on its own.

Table of contents

• Quick start
• Capability matrix
• Why it works well
• Works alongside you
• Tools
• Repository layout
• Environment flags
• Embedding in an app
• Prompting your agent
• Contributing
• Credits and license

Quick start

1. Download the latest binary for your platform from Releases (munim-computer-use-macos-universal.zip, munim-computer-use-windows-x64.zip) or build from source.
2. Put it somewhere on your PATH (/usr/local/bin/munim-computer-use, or %LOCALAPPDATA%\Programs\munim-computer-use\munim-computer-use.exe).
3. macOS only: run munim-computer-use request-permissions once to be prompted for Accessibility and Screen Recording.
4. Register it with your agent:

# Claude Code — fastest: the npm launcher fetches the signed binary on first run.
# Add `--scope user` to register it for every project instead of just this one.
claude mcp add munim-computer-use -- npx -y munim-computer-use
# or point at a downloaded binary
claude mcp add munim-computer-use -- /usr/local/bin/munim-computer-use

# Codex — writes the entry below into ~/.codex/config.toml for you (needs a Codex CLI
# with `codex mcp`; check with `codex mcp --help`)
codex mcp add munim-computer-use -- npx -y munim-computer-use

# Codex — ~/.codex/config.toml, if you would rather edit it yourself
[mcp_servers.munim-computer-use]
command = "npx"
args = ["-y", "munim-computer-use"]

# Cursor — no MCP subcommand in its CLI, so write the config. ~/.cursor/mcp.json applies
# to every project; .cursor/mcp.json in a repo applies to that one.
mkdir -p ~/.cursor && [ -s ~/.cursor/mcp.json ] || echo '{}' > ~/.cursor/mcp.json
jq '.mcpServers["munim-computer-use"] = {"command":"npx","args":["-y","munim-computer-use"]}' \
  ~/.cursor/mcp.json > ~/.cursor/mcp.json.tmp && mv ~/.cursor/mcp.json.tmp ~/.cursor/mcp.json

// Cursor — the entry that produces, in .cursor/mcp.json
{ "mcpServers": { "munim-computer-use": { "command": "npx", "args": ["-y", "munim-computer-use"] } } }

Then ask: "Open Safari, find the cheapest flight to Denver on Tuesday and put it in a note." The agent reads the UI with get_app_state, acts by element id, and verifies with screenshot.

Capability matrix

Munim Computer Use is the highlighted first column; the others are the computer-use servers people reach for. each cell comes from that project's own README in September 2026 (sources under Credits and license). ✅ present · ❌ absent or not documented · ⚠️ partial.

Capability	Munim Computer Use	Codex Computer Use	Anthropic reference demo	Windows-MCP	MacOS-MCP	open-computer-use	computer-use-mcp (zavora)	Notes
macOS	✅	✅	❌	❌	✅	✅	✅	The Anthropic demo drives a Linux desktop inside Docker, not your machine.
Windows	✅	✅	❌	✅	❌	✅	✅	Windows-MCP is Windows only; MacOS-MCP is macOS only.
Linux	✅	❌	✅ (sandbox)	❌	❌	✅	✅	Munim Computer Use uses AT-SPI + X11; native Wayland apps get element actions but not coordinate clicks.
Accessibility tree with element ids	✅	❌	❌	✅	✅	✅	✅	Codex and the Anthropic demo are screenshot-driven. Ids let the agent press the button instead of a pixel.
Background input (your mouse never moves)	✅	✅	n/a	❌	❌	❌	❌	Munim Computer Use addresses events to the target window: always on macOS (SkyLight and per-process events), for UI Automation patterns and classic Win32 controls on Windows; other Windows UI and all Linux pointer input (XTEST) still move the real pointer. Codex does this too, with a second cursor of its own. Every other server in this table drives the real cursor.
Agent pointer overlay	✅	✅	❌	⚠️	❌	❌	❌	Windows-MCP flashes a border around captures; Codex draws its own cursor on your screen.
Zoom into a region at full resolution	✅	❌	✅	❌	❌	❌	❌	Anthropic's toolset has zoom; here it is a tool on every platform.
Screenshots carry screen-coordinate mapping	✅	n/a	n/a	❌	❌	❌	❌	Origin and pixels-per-point in every capture, so clicks from Retina or downscaled images land.
Hover, wait, label query	✅	⚠️	⚠️	✅	⚠️	❌	⚠️	Windows-MCP has Wait/WaitFor; MacOS-MCP has Wait; Anthropic has wait/mouse_move.
Your signed-in Chrome, own tab group or yours	✅	⚠️	❌	⚠️	❌	❌	❌	Codex uses its in-app browser; Windows-MCP reads the DOM of open browsers. Munim Computer Use opens its own labelled tab group in your real Chrome, and can also take over a tab you already have open when you ask it to.
Works with any MCP client	✅	❌	❌	✅	✅	✅	✅	Codex Computer Use is Codex only; the Anthropic demo is Claude only.
Identical tool surface on every platform	✅	n/a	n/a	n/a	n/a	⚠️	✅	30 tools with byte-identical schemas across the Swift and Rust servers.
Prebuilt signed binaries + npm launcher	✅	✅	❌	❌	❌	✅ (npm)	✅ (npm)	macOS universal (Developer ID signed) and Windows x64 on Releases.
Open source	✅ Apache-2.0	❌	✅	✅ MIT	✅ MIT	✅ MIT	✅ MIT

Also looked at: mediar-ai/mcp-server-macos-use (macOS, accessibility, real input), deploymenttheory/windows-mcp-server (Windows, UIA Invoke patterns, WaitFor), nuphus-mcp (OCR + bring-your-own vision model, CDP Chrome), computer-control-mcp (PyAutoGUI + OCR), and microsoft/playwright-mcp (browser only). Corrections welcome — open an issue with a link.

Why it works well

• Accessibility first, pixels second. get_app_state returns the app's accessibility tree with stable element ids, so the agent presses the button instead of guessing at a coordinate. It costs a fraction of the tokens of a screenshot and it is what scores highest on OSWorld-style tasks. Screenshots are for verifying and for content the tree cannot describe.
• Background control. Events are addressed to the target window (SkyLight on macOS, UI Automation patterns and posted window messages on Windows). On macOS the agent never takes your mouse or keyboard; see Works alongside you for the exact guarantee on each platform.
• Pointer overlay, not your pointer. A soft lavender agent pointer shows where the agent is acting. Your cursor is untouched.
• Coordinates that land. Every screenshot and zoom carries its screen origin and pixels-per-point. zoom captures any region at full physical resolution.
• Your browser, your logins. The Chrome extension gives the agent its own labelled tab group in your signed-in Chrome, and leaves your tabs alone unless you point it at one.
• Or the tab you already have open. browser_list_tabs all=true shows every tab in the browser and browser_use_tab takes one over in place — useful when the page is already signed in or mid-flow and re-opening the URL would throw that away. An adopted tab is not moved into the agent's group, not activated and not reloaded; cleanup releases it rather than closing it, and browser_release_tab hands it back early.
• Model-agnostic. No vision model is required for interaction; local models work too.
• Look → act → verify. hover for mouse-over menus, wait for loads, query to find a control by label without reading a whole tree.

Works alongside you

The agent has its own pointer; yours stays yours.

macOS — guaranteed. Every action goes through accessibility (press, set value, select text, show menu, scroll bars) or through events addressed to the target app's process and window. The server never moves your pointer, never posts into the system-wide input stream, and never holds or blocks your input, so you can keep clicking and typing in other apps while the agent works — even in the same app, on another window. Events come from a private source, so a modifier you are holding does not leak into the agent's clicks. The target app may be brought forward when that is the point of the step (activate_app, or handing you a password field), but not on every action.

The exceptions refuse instead of borrowing your pointer: a click, hover or scroll with no target app (coordinates over the desktop before any get_app_state), and drags that leave the source window (between apps, or onto the desktop). The error says what to pass instead.

Windows — best effort, reported. Element presses (Invoke), set_value, select_text and scrolling through UI Automation's ScrollPattern never touch your pointer, and classic Win32 controls also take clicks, hovers, wheel and drags as posted window messages. Other UI (Chromium, Electron, WPF, UWP) only reacts to real mouse input, so those clicks, hovers and drags move your pointer, and type_text/press_key go to the focused window. Whenever the real pointer was used, the result says via cursor.

Linux — pointer actions use it. XTEST input moves the real pointer and goes to the focused window, and results say via cursor. Element actions through AT-SPI (press, set value, insert text, select) do not move it.

Tools (30)

Area	Tools
See	list_apps, get_app_state (with query), screenshot, zoom, list_displays
Act	click, right_click, hover, drag, scroll, type_text, set_value, select_text, press_key, activate_app, wait
Clipboard	clipboard_read, clipboard_write (plain text)
Browser	browser_open_tab, browser_list_tabs, browser_use_tab, browser_release_tab, browser_select_tab, browser_navigate, browser_snapshot, browser_click, browser_type, browser_press_key, browser_close_tab, browser_close_all_tabs

Names, argument shapes and descriptions are identical on every platform, and CI enforces it (node scripts/check-tool-parity.mjs); a model that learned them on a Mac needs nothing new on Windows.

Repository layout

Directory	What	Build
macos/	Swift server on the Accessibility API and ScreenCaptureKit (macOS 14+)	swift build -c release → .build/release/munim-computer-use
windows-linux/	Rust server: UI Automation on Windows, AT-SPI + X11 on Linux	cargo build --release → target/release/munim-computer-use
chrome-extension/	Chrome extension + native messaging host for the browser_* tools	Load unpacked; sh install.sh / install.ps1 registers the host; node background.test.mjs
Dockerfile	Headless Linux build of the Rust server for registry introspection (Glama and similar); no desktop control inside a container	docker build -t munim-computer-use .

Build from source

# macOS
cd macos && swift build -c release
# Windows / Linux
cd windows-linux && cargo build --release

Linux notes: element actions work everywhere; coordinate clicks need an X11 or XWayland client, since native Wayland apps do not expose absolute geometry.

Chrome extension (optional)

1. chrome://extensions → Developer mode → Load unpacked → select chrome-extension/.
2. Register the native messaging host: sh chrome-extension/install.sh (macOS/Linux) or powershell -File chrome-extension/install.ps1 (Windows). Point COMPUTER_USE_PATH at the binary if it is not in the default build location.
3. The binary can also register itself for its current identity: munim-computer-use install-native-host (see Embedding).

Environment flags

Variable	Effect
COMPUTER_USE_BROWSER=0	Hide the browser_* tools
COMPUTER_USE_AGENT_CURSOR=0	Do not draw the agent pointer
COMPUTER_USE_AGENT_CURSOR_TASK_FADE_SECS	How long the pointer stays after the last tool call (default 8)
COMPUTER_USE_ALLOW_SECURE_FIELD_INPUT=1	Allow typing into password fields (refused by default)
COMPUTER_USE_REMOTE_CONTROL=1	Remote-desktop mode: input takes over the real pointer

Remote control

Normally this server never touches the pointer: coordinate clicks are routed to a specific window, keystrokes are posted to a specific process, and an action that cannot be targeted is refused rather than taking over the machine. That is what lets an agent work while the user keeps using their computer.

COMPUTER_USE_REMOTE_CONTROL=1 inverts that contract for one process, for the case where a person is watching this machine's screen from another one and is steering it themselves. Then click, right_click, drag, hover and scroll move the real cursor and type_text and press_key go to whatever is focused, the way Chrome Remote Desktop or Screen Sharing behave. scroll also accepts x/y so the wheel acts over the point the viewer scrolled at, and the agent-cursor overlay stays hidden — there is only one pointer now.

A host that runs both an agent and a viewer runs them as two processes, so turning this on for the viewer never takes the pointer away from the user on the agent's behalf.

Embedding in an app

An app can ship this binary inside its own bundle and run it under its own identity, so it never shares a browser bridge, agent-cursor app or native-messaging host with a standalone install on the same machine. MT Code does exactly this. Pass a profile, either as a JSON object or a path to a JSON file, with --profile <json|file> (any position) or COMPUTER_USE_PROFILE:

{
  "name": "example-desktop",
  "envPrefix": "EXAMPLE_DESKTOP_",
  "agentCursorName": "ExampleAgentCursor",
  "agentCursorBundleId": "com.example.agent-cursor",
  "nativeHostNames": ["com.example.desktop"],
  "extensionIds": ["abcdefghijklmnopabcdefghijklmnop"],
  "nativeHostDescription": "Example desktop control bridge"
}

Key	Default	Meaning
name	none (standalone paths)	Moves the support dir, bridge socket and Windows pipe under this name
supportDir	~/Library/Application Support/computer-use, $XDG_DATA_HOME/munim-computer-use, %LOCALAPPDATA%\munim-computer-use	Native-host wrapper, profile copy; on macOS also the bridge socket and a bare build's overlay app
bridgeSocket	<supportDir>/bridge.sock (macOS), $XDG_RUNTIME_DIR/<name>/bridge.sock (Linux), <name>-bridge-<user> pipe (Windows)	Where the MCP server and the Chrome relay meet
envPrefix	none	Tunables are read as <prefix>BROWSER, <prefix>AGENT_CURSOR, … before COMPUTER_USE_*
agentCursorName / agentCursorBundleId	MunimAgentCursor / com.munimtech.computer-use.agent-cursor	The pointer overlay's app, executable and window-class name, and its macOS bundle id
historyDir	none	Default --root for computer-history
nativeHostNames / extensionIds	com.munim.mtcode.desktop, com.munimtech.computer-use.desktop / kgdolgnijopbghhomnblabjkmjhnoage	What install-native-host registers, and for which extension

Each of supportDir, bridgeSocket, envPrefix, agentCursorName, agentCursorBundleId and historyDir can also be overridden by COMPUTER_USE_<SNAKE_CASE> (for example COMPUTER_USE_SUPPORT_DIR), which wins over the profile. munim-computer-use identity prints the resolved values.

• Browser bridge. Run munim-computer-use install-native-host with the same profile. It writes a wrapper that relays Chrome into this identity's bridge (replaying the profile), and a host manifest for each name in every Chrome/Chromium profile directory (the registry on Windows). It rewrites nothing that is already current, so an app can call it on every launch.
• Extension. Use the stock extension, or build a variant with its own host names, tab-group title and key: node scripts/build-extension.mjs --out <dir> --host com.example.desktop --group-title "Example" --key <base64>. It prints the variant's extension id for extensionIds.
• macOS permissions. The MCP server is a bare executable, so Accessibility and Screen Recording are granted to the app that spawns it. Only the agent-cursor overlay has a bundle of its own; ship <agentCursorName>.app (a copy of the binary plus an LSUIElement Info.plist) beside the binary, or it is materialised under supportDir on first use.

Prompting your agent

Look → act → verify. get_app_state for ids, act by id, then get_app_state or screenshot again before the next step. Use zoom for small text, hover for menus that appear on mouse-over, wait after loads, keyboard shortcuts for stubborn widgets. The system-prompt text MT Code gives its agents lives in CodexDeveloperInstructions.ts and is a good starting point.

Contributing

This repository mirrors the native/ tree of munimtechnologies/mtcode, where the server is developed and shipped inside MT Code. Issues and discussions are welcome here; code changes land in mtcode first and are synced.

Credits and license

Designed and built by Munim Technologies (Munim, Inc.) for MT Code. Copyright 2026 Munim, Inc. Licensed under the Apache License 2.0; see LICENSE.

Comparison sources: Codex Computer Use · Anthropic computer-use demo · CursorTouch/Windows-MCP · CursorTouch/MacOS-MCP · QwenLM/open-computer-use · zavora-ai/computer-use-mcp · mediar-ai/mcp-server-macos-use · deploymenttheory/windows-mcp-server · nuphus-mcp · computer-control-mcp · microsoft/playwright-mcp