feat: stream Android emulators on Windows and Linux with OpenH264 - #86
Open
NathanWalker wants to merge 9 commits into
Open
feat: stream Android emulators on Windows and Linux with OpenH264#86NathanWalker wants to merge 9 commits into
NathanWalker wants to merge 9 commits into
Conversation
Non-macOS builds link native_stubs.c instead of the macOS native bridge, so
neither iOS simulator nor Android emulator H.264 streaming can work there.
The browser previously showed the raw JSON stub error in the stream area and
kept retrying WebRTC offers that could never succeed.
Server:
- Add platform.rs as the single source of truth for the live-video
capability, its user-facing reason, and the CLI note.
- Report `hostOs` and a `liveVideo` block from /api/health and
/api/stream-quality.
- Reject WebRTC offers early with 501 and the platform explanation via a new
AppError::Unsupported variant.
- Print a "Live video:" note after the service URLs on non-macOS hosts.
- Share one clear message across the H.264 encoder stubs; drop the unused
WIP helper and the duplicated #[test] attribute.
Client:
- Parse `{"error": ...}` bodies from failed offer and stream-config posts.
- Read `liveVideo` from the stream-quality response, pause the live stream,
and show the reason in the viewport instead of the reconnect loop.
- Disable the stream transport, encoder, frame rate, and resolution controls
with a "Requires macOS" note when the server cannot stream.
Docs: platform support table, health/REST field docs, video guide, and a
troubleshooting entry for the message.
Non-macOS builds link native_stubs.c instead of the macOS simulator bridge,
so the Android emulator WebRTC source had no H.264 encoder and every stream
failed with the stub error. Frame capture (emulator gRPC screenshots) and the
WebRTC transport were already cross-platform Rust; only the encoder was
missing.
Server:
- Add transport/software_h264.rs: an OpenH264 encoder (openh264 crate,
compiled from source, BSD) that turns RGBA/BGRA emulator frames into
Annex B baseline H.264. Bitrate follows the macOS budget (bits per pixel
with a minimum), keyframes honor RTCP requests, and the encoder rebuilds
on size or quality changes. Unit tests cover the budget math, frame
validation, keyframe forcing, and reinitialization.
- Route the Android WebRTC source through it on non-macOS builds; macOS
keeps the native VideoToolbox/x264 path behind the C ABI. Frame
publishing is shared between both paths.
- Reframe platform.rs around iOS simulator support: liveVideo now reports
supported: true everywhere with encoder ("native" or "openh264"),
androidEmulator, iosSimulator, and an iosSimulatorReason off macOS.
WebRTC offers for iOS simulators on non-macOS answer 501; Android offers
proceed.
- CLI banner note now says Android streams with software H.264 and iOS
simulators require macOS.
CI: add a Linux and Windows cargo clippy/test job so the OpenH264 module and
the native stubs are linted and tested where they are actually compiled.
Docs: per-platform encoder table, health/REST field docs, troubleshooting
entries for the iOS message and slow software encoding, AGENTS pointer.
Cargo.lock is not updated here (no Rust toolchain on the authoring machine);
the first cargo build adds the openh264 entries.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
NathanWalker
had a problem deploying
to
npm-publish
September 9, 2026 21:57 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
simdeckciproxy | 558220d | Commit Preview URL Branch Preview URL |
Sep 09 2026, 11:49 PM |
CI's first run of the Linux and Windows clippy/test job surfaced: - rustfmt layout in software_h264.rs - clippy manual_is_multiple_of in the frame validator - an unused platform constant - FFI H.264 encoder declarations that are now macOS-only - test constructors that referenced the macOS-only encoder fields - pre-existing non-macOS lints that had never run: needless_return and an unread fd in the POSIX shared-video stub, an unused c_void import in performance.rs, parse_io_platform_uuid dead off macOS, and result_large_err in the generated tonic client
Deploying simdeck with
|
| Latest commit: |
558220d
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://167790fb.simdeck.pages.dev |
| Branch Preview URL: | https://fix-windows-live-video-messa.simdeck.pages.dev |
Pre-existing test code on main; the newer stable clippy flags chunks_exact with a constant size, which fails all three Rust jobs.
OpenH264's screen-content mode emits an IDR when the whole frame changes color, which the Linux test run showed; only the forced keyframe and the Annex B framing are guaranteed.
Pre-existing tests that spawn /bin/sh; the new Windows CI job cannot run them. The OpenH264 encoder tests and the other 275 tests pass there.
Windows clippy flagged the imports and new_for_tests helper that only the /bin/sh tests use.
The native artifacts were compiled in a job that ran before the release job bumped package.json and Cargo.toml, so simdeck@0.1.35 shipped binaries whose `--version` still said 0.1.34. - Resolve the release version in a new `resolve-version` job that runs first and feeds both the native builds and the release job. - Sync packages/server/Cargo.toml (and Cargo.lock) to that version in each native build job before `npm run build:cli`. - Run every built binary with `--version` and fail the job if it does not report the release version; do the same for the universal macOS binary in the release job before committing, tagging, or publishing. - Apply the resolved version verbatim in the release job instead of recomputing it, and guard `bump=current` against a mismatch. - Cover the new ordering in scripts/github-actions.test.mjs and fix the pre-existing `test -f "$artifact"` assertion, which expected backslash-escaped quotes and always failed. - Document the flow in CONTRIBUTING.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Android emulator live streaming now works on Windows and Linux. Those builds link
native_stubs.cinstead of the macOS simulator bridge, so the Android WebRTC source had no H.264 encoder and every stream failed with the stub error (rendered as raw JSON in the browser). Frame capture (emulator gRPC screenshots) and the WebRTC transport were already cross-platform Rust; only the encoder was missing.Encoder (non-macOS)
packages/server/src/transport/software_h264.rs: Cisco OpenH264 via theopenh264crate, compiled from source (BSD, no system deps, supported onx86_64-pc-windows-gnu/msvcand Linux). Emits Annex B baseline H.264, which the existing packetizer and browsers already accept.androidEncoders[].encoder.native.cfg-gated: macOS keeps the native VideoToolbox/x264 path behind the C ABI; other targets use the software encoder. Frame publishing is shared.Capability reporting and messaging
platform.rsnow describes iOS simulator support rather than "live video".liveVideoin/api/healthand/api/stream-qualityreportssupported: true,encoder(nativeoropenh264),androidEmulator: true,iosSimulator(macOS only) andiosSimulatorReasonelsewhere.501with that reason (newAppError::Unsupported). Android offers proceed.Live video:note on Windows/Linux; the browser unwraps JSONerrorbodies instead of showing raw JSON, and still pauses with a reason if a server ever reportssupported: false.CI and docs
rust-non-macosjob runscargo clippy -D warningsandcargo teston Ubuntu and Windows, since the macOS job compiles out both the OpenH264 module and the native stubs.Test plan
npm run --prefix packages/client typecheckandnpm run --prefix packages/client test(104 tests)rust(macOS fmt/clippy/test) and the newrust-non-macosjob (Ubuntu + Windows clippy/test, including the OpenH264 encoder tests) are green as of7fb0e39. Release run 34411749072 built all four native artifacts (macOS arm64/x64, Linux, Windows) with OpenH264.simdeckprints theLive video:note,/api/healthreportsliveVideo.encoder: "openh264", boot an Android emulator and confirm the browser shows live video with touch input;/api/metricsshowsandroidEncoders[].encoder.native.outputFramesincreasing.liveVideo.encoder: "native".Cargo.lockgains theopenh264entries via the release version-bump commit (the workflow stages it).Also fixed along the way: pre-existing lints that the new clippy 1.98 and the first non-macOS clippy run surfaced (
chunks_exact,needless_return, unused imports, an unreadfd,result_large_errin generated tonic code), and two/bin/shlog-stream tests are now Unix-only.Follow-up worth considering: install
nasmon the release runners for OpenH264's assembly paths (up to ~3x faster encoding); left out to keep the release build unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_013K2GgwpF2vLSgz4otg8tmT