Skip to content

fix(server): stop Windows terminal polling from spiking CPU - #9476

Merged
juliusmarminge merged 13 commits into
pingdotgg:mainfrom
UtkarshUsername:fix/windows-native-process-snapshot
Sep 6, 2026
Merged

fix(server): stop Windows terminal polling from spiking CPU#9476
juliusmarminge merged 13 commits into
pingdotgg:mainfrom
UtkarshUsername:fix/windows-native-process-snapshot

Conversation

@UtkarshUsername

@UtkarshUsername UtkarshUsername commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What Changed

Terminal subprocess polling no longer spawns a PowerShell CIM query every second while terminals are active. It now reads one shared process snapshot per polling tick from the existing resource-monitor sidecar (new processTable command, protocol v3), on both Windows and POSIX.

  • One snapshot per tick shared across every terminal, instead of per-terminal process probes.
  • Failed snapshots back off exponentially up to 60 seconds and reset on success or when no sessions are running. The backoff also applies when the sidecar is down and the ps/PowerShell fallback serves instead, so a stalled sidecar can't hot-loop the expensive fallback.
  • No process-count cap on snapshots, and terminal activity labels plus process registration keep working as before.
  • As a side effect of reusing the sidecar, no new native dependency was needed: the windows-process-tree packaging added earlier in this branch is reverted.

Verification:

  • vp test run apps/server/src/terminal/Manager.test.ts apps/server/src/resourceTelemetry/NativeTelemetryClient.test.ts scripts/lib/cli-external-packages.test.ts (82 passed, incl. a new test that fails the sidecar table and asserts fallback data is applied while fallback spawns back off)
  • vp test run apps/server/src/resourceTelemetry/ResourceTelemetry.test.ts apps/server/src/resourceTelemetry/ResourceTelemetryHistory.test.ts apps/server/src/resourceTelemetry/Model.test.ts apps/server/src/diagnostics/ProcessDiagnostics.test.ts (31 passed)
  • vp run --filter t3 typecheck and vp run --filter @t3tools/scripts typecheck (clean)

Why

T3 Code queried Win32_Process through a fresh PowerShell process every second while terminals were active. On affected Windows machines those calls averaged roughly one second, repeatedly timed out, and kept CPUs and fans busy. The sidecar already enumerates the full process table with sysinfo and ships in desktop and CLI builds, so reusing it removes PowerShell/WMI from the polling loop without loading native code into the server process. This matches docs/internals/resource-telemetry.md, which exists to replace recurring ps/PowerShell subprocess probes.

Checklist

  • This PR is small and focused
  • I explained what changed and why
  • I included before/after screenshots for any UI changes (not applicable: no UI changes)
  • I included a video for animation/interaction changes (not applicable: no UI changes)

Implemented with GPT-5.6 using the Codex harness.


Note

Medium Risk
Bumps RESOURCE_MONITOR_PROTOCOL_VERSION to 3 and changes terminal activity detection on all platforms; mismatched or missing sidecar binaries fall back to spawned probes with backoff rather than failing silently.

Overview
Reduces Windows terminal subprocess polling CPU by sourcing one shared process table per tick from the existing resource-monitor sidecar instead of spawning PowerShell/ps on every interval.

The resource-monitor protocol moves to v3 with a new processTable command and {pid, ppid, name} response, implemented in the Rust sidecar, contracts, and NativeTelemetryClient (5s timeout, same request/deferred pattern as sampleNow).

TerminalManager calls NativeTelemetryClient.processTable by default (wired via NativeTelemetryLayerLive on the terminal layer). PowerShell CIM and POSIX ps remain fallbacks when the sidecar fails; fallback data still updates activity, but ticks count as failures. subprocessSnapshotPollDelayMs doubles the poll delay per failure (max 60s) and resets after a successful snapshot or when no sessions are running.

Tests and fixtures use protocol v3; docs list the new command/event.

Reviewed by Cursor Bugbot for commit 3bb2be6. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Replace Windows terminal polling with sidecar processTable requests

  • Adds a processTable command to the resource-monitor sidecar (protocol v2→v3) and a matching NativeTelemetryClient.processTable effect, so the terminal manager queries the running sidecar instead of spawning a process for each poll.
  • TerminalManager uses the sidecar as its primary process-table source. On failure it falls back to one spawned query and marks the snapshot unsuccessful for retry scheduling.
  • Failed process-table snapshots back off exponentially from the configured poll interval, capped at 60 seconds; success resets the counter.
  • Replaces the Windows text-based process parser with a structured ProcessTableEntry parser; the PowerShell fallback validates and converts lines into structured entries, omitting invalid PIDs.
  • Risk: Protocol version bump from 2 to 3 in resourceTelemetry.ts and main.rs requires sidecar and client to deploy together; mismatched versions will reject processTable commands.

Macroscope summarized e298346.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Sep 3, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR changes the default terminal polling path across the server, native sidecar, and shared wire protocol, including a protocol-version bump and new fallback/backoff behavior. The cross-component runtime impact and deployment compatibility require human review.

You can add or adjust custom eligibility rules. Learn more.

Comment thread apps/server/src/terminal/Manager.ts Outdated
Comment thread apps/server/src/terminal/Manager.ts
@UtkarshUsername UtkarshUsername changed the title [WIP] fix(server): use native Windows process snapshots fix(server): stop Windows terminal polling from spiking CPU Sep 3, 2026
Comment thread apps/server/src/terminal/Manager.ts Outdated
Comment thread apps/server/src/terminal/Manager.ts Outdated
@UtkarshUsername
UtkarshUsername marked this pull request as draft September 3, 2026 21:45
@UtkarshUsername
UtkarshUsername marked this pull request as ready for review September 3, 2026 21:45
@UtkarshUsername UtkarshUsername changed the title fix(server): stop Windows terminal polling from spiking CPU [WIP] fix(server): stop Windows terminal polling from spiking CPU Sep 3, 2026
Comment thread apps/server/src/terminal/Manager.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a0bc756. Configure here.

Comment thread native/resource-monitor/src/main.rs
Comment thread apps/server/src/terminal/Manager.ts
UtkarshUsername and others added 2 commits September 4, 2026 03:57
Apply exponential backoff when the sidecar fails and the spawned fallback serves instead, so a stalled sidecar no longer hot-loops PowerShell. Skip pid 0 in the Rust process table so one kernel entry cannot fail the whole event decode. Revert the Windows sidecar packaging added for the removed native dependency.
@UtkarshUsername UtkarshUsername changed the title [WIP] fix(server): stop Windows terminal polling from spiking CPU fix(server): stop Windows terminal polling from spiking CPU Sep 3, 2026
Comment thread native/resource-monitor/src/main.rs Outdated

@SunkenInTime SunkenInTime left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Windows x64 in a dedicated worktree at e298346f84613e632cabb21c55b4e268d786cdee. Approving the polling change.

  1. Validation passed: bun fmt, bun lint, and bun typecheck; 139 focused TypeScript tests; 17 Rust tests; Rust formatting and release build. Direct native probes verified process ancestry across 12 snapshots. A real PowerShell PTY followed Node activity, terminal-owned HTTP discovery, Ctrl+C, and port removal. Computer-use smoke testing opened the isolated web app and terminal drawer. These are functional checks, not a sustained CPU benchmark.

  2. Separate existing Windows issue: rapid restart followed by close crashed the server with Signals not supported on windows. The unchanged adapter forwards SIGTERM to node-pty, which can defer and throw outside synchronous error handling before readiness. A standalone reproduction also exits 1. The adapter, dependency lockfile, and signal path predate this PR. Please track Windows termination without POSIX signals and close-before-ready coverage separately; the restart acknowledgement is not evidence of clean shutdown.

  3. Nonblocking cleanup: add direct response, timeout, interruption, and sidecar-restart tests for processTable. The existing NativeTelemetryClient suite passes, but this PR adds no tests to that file. If more commands are planned, a small helper for the repeated deferred-map/request/finalizer lifecycle would reduce drift.

Blueprint review and design report includes numbered findings, architecture, source links, UI evidence, verification boundaries, and raw outputs. Download the evidence bundle.

@dakrer

dakrer commented Sep 6, 2026

Copy link
Copy Markdown

Thanks for this. I have been chasing the same loop on a managed corporate laptop and want to add measurements from that kind of machine, because they support the sidecar approach and point at one detail in the fallback path.

The machine. Windows 11 Enterprise, 4 cores and 8 threads, several endpoint security agents that hook process creation in kernel filter drivers. On this machine process creation itself is the expensive operation. Measured on T3 Code 0.0.38 and on a source checkout of main at 223ff44:

Command Idle machine Under the CPU load the loop itself caused
cmd /c echo 1 3.2 s
Bare Windows PowerShell 5.1 start (-NoProfile -NonInteractive -Command 1) 2.0 s 12.5 s
The terminal snapshot command, exactly as shipped 3.9 s 42 s
One git invocation about 1.3 s
tasklist /fo csv /nh 12.1 s

What the shipped loop does here. With one idle terminal open: 50 snapshots in 4.5 minutes, every one timed out at 1.5 s, about 4.3 s each including the kill, one every 5.4 s. Per minute under the server process: 12 powershell.exe, 22 cmd.exe, 20 taskkill.exe. CPU at 100 percent, 70 to 80 percent of it kernel time, the System process at 2 to 3.5 cores. The snapshot never succeeded once on this machine, and each failure paid for a process tree kill on top. Reading the process table from the sidecar removes all of that from the main path, which is the right fix.

One observation on the fallback. windowsProcessTableSnapshot keeps timeout: "1500 millis". On a machine like this one that budget can never be met, because PowerShell start-up alone takes longer. So whenever the sidecar is unavailable, the fallback fails every tick, the back-off climbs to 60 s, and terminal activity never updates at all. With a 10 s budget the same command succeeded 10 times out of 10 here, 5.0 s on average and 9.9 s at worst. Where the fallback is the only path (a source checkout without a built sidecar, an unsupported platform or architecture, a crashed sidecar) a larger budget keeps the feature alive at a slower cadence instead of switching it off silently. Pacing the next poll by the duration of the last snapshot, for example four times its cost with a cap, also keeps a slow but successful fallback from running back to back. I have that as a small local patch and can send it as a follow-up on top of this PR once it lands, if you think it is worth having. The same budget point applies to the PowerShell fallback in #9520: the port probe took 7.4 s here with 67 listeners (#5900).

Two smaller findings from the same investigation, in case they save someone time: tasklist is not a cheap alternative on such machines (12 s here) and it has no parent PIDs anyway. The doubled taskkill after a timeout comes from Effect's Node spawner, which kills once from the scope finalizer and once more from the exit handler because a killed process exits non-zero, so every timed-out helper costs two cmd.exe plus taskkill.exe pairs on Windows.

Offer. I can test this branch on that laptop and report before and after numbers, on the fallback path right away and on the sidecar path once I have built the monitor. Say the word if that would help.

Measurements and analysis were done with Claude Code (Fable 5.1). An independent review of the same traces by Codex (GPT-6 Astra) confirmed the loop attribution.

@juliusmarminge
juliusmarminge merged commit ea646c0 into pingdotgg:main Sep 6, 2026
25 checks passed
BarretoDiego pushed a commit to BarretoDiego/t3code that referenced this pull request Sep 7, 2026
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Sep 7, 2026
## What's Changed
* fix(clients): show feedback results in composer banners by @juliusmarminge in pingdotgg/t3code#10398
* fix(server): stop Windows terminal polling from spiking CPU by @UtkarshUsername in pingdotgg/t3code#9476
* fix(web): onboarding installs agents without needing Node or npm by @t3dotgg in pingdotgg/t3code#10402
* fix(server): allow settling threads with unanswered async questions by @t3dotgg in pingdotgg/t3code#10400
* feat(ci): ship stable releases from the latest nightly commit by @t3dotgg in pingdotgg/t3code#10410
* feat(marketing): add a nightly channel to the download page by @t3dotgg in pingdotgg/t3code#10408
* fix(web): keep settings inputs focused during IME composition by @Lucenx9 in pingdotgg/t3code#10262
* fix(server): preserve Codex reset credits during usage updates by @yashranaway in pingdotgg/t3code#10308
* docs: link the repository security reporting policy by @yashranaway in pingdotgg/t3code#10303
* fix(web): only show auto balance errors after failed checks by @maria-rcks in pingdotgg/t3code#10407
* fix(web): improve preview recording frame delivery by @maria-rcks in pingdotgg/t3code#10403
* fix(server): preserve inline provider secrets on redacted saves by @maxwellyoung in pingdotgg/t3code#10054
* fix(web, mobile): replace Apple desktop machine labels by @extoci in pingdotgg/t3code#10396
* fix(web): hide browser when the right panel starts closing by @Neel2107 in pingdotgg/t3code#10385
* fix(web): keep settings section headings description-free by @maria-rcks in pingdotgg/t3code#10415
* fix(usage): read and redeem hub reset credits through CLIProxyAPI by @juliusmarminge in pingdotgg/t3code#10395
* fix(web): deduplicate expanded tool labels and keep errors expandable by @Yash-Singh1 in pingdotgg/t3code#10420
* fix(server): skip git status scans while the index is locked by @Gigioxx in pingdotgg/t3code#9845
* fix(mcp): allow text-only preview snapshots by @juliusmarminge in pingdotgg/t3code#10232
* fix(claude): name the expired login or usage limit instead of a generic API error by @vitalyiegorov in pingdotgg/t3code#10321
* feat(mobile): queue a message while its attachment is still uploading by @juliusmarminge in pingdotgg/t3code#10404
* feat(mobile): show when an existing thread has a message waiting in the outbox by @juliusmarminge in pingdotgg/t3code#10405
* fix(codex): accept misalignment policy errors on thread resume by @realbakari in pingdotgg/t3code#10373
* fix(server): skip disabled settlement lookups by @t3dotgg in pingdotgg/t3code#10424
* fix(server): run OpenCode CLI commands sequentially by @t3dotgg in pingdotgg/t3code#10427
* feat(web): name the drop action while dragging sidebar threads by @SunkenInTime in pingdotgg/t3code#10378
* perf(web): keep the sidebar responsive during bulk thread updates by @t3dotgg in pingdotgg/t3code#10413
* fix(web): onboarding wizard now supports light mode by @t3dotgg in pingdotgg/t3code#10432
* feat(threads): dismiss async questions without replying by @t3dotgg in pingdotgg/t3code#10431
* fix(web): stop collapsing the composer when it loses focus by @t3dotgg in pingdotgg/t3code#10437
* fix(server): keep interrupted threads resumable after restarts by @maria-rcks in pingdotgg/t3code#10421

## New Contributors
* @Neel2107 made their first contribution in pingdotgg/t3code#10385
* @realbakari made their first contribution in pingdotgg/t3code#10373

**Full Changelog**: pingdotgg/t3code@v0.0.39-nightly.20260906.1316...v0.0.39-nightly.20260907.1325

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.39-nightly.20260907.1325
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants