From 348ce39ad338c2d40374885641ca44eab5445592 Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Thu, 9 Apr 2026 14:20:14 -0600 Subject: [PATCH 1/7] AGENTS.md --- AGENTS.md | 158 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 158 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..a83e2f61 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,158 @@ +# AGENTS.md — LiveKit C++ Client SDK + +## Project Overview + +This is **client-sdk-cpp**, the official LiveKit C++ client SDK. It wraps a Rust core (`client-sdk-rust/`) via a protobuf-based FFI bridge. All WebRTC, networking, and media logic lives in Rust; the C++ layer provides an ergonomic API for C++ consumers. + +## Architecture + +### Core Principle +Rust owns as much of the business logic as possible. If a feature may be used by another SDK it should be implemented in Rust. Since this is an SDK, +ensure backwards compatibility is maintained when possible. + + +### Platform Support +The SDK must be supported on the following platforms: +1. windows x64 +2. linux x86_64 +3. linux arm64 +3. macOS x86_64 +4. macOS arm64 + + +SDK features follow pattern: + +1. Serialize a protobuf `FfiRequest` message. +2. Send it to Rust via `FfiClient::instance().sendRequest(req)`. +3. Receive a synchronous `FfiResponse` or an asynchronous `FfiEvent` callback. +4. Expose the result through the C++ API. + +When making larger scale changes, check with the developer before committing to architecture changes involving changes to the `client-sdk-rust/` submodule. + +### Directory Layout + +| Path | Description | +|------|-------------| +| `include/livekit/` | Public API headers (what SDK consumers include) | +| `src/` | Implementation files and internal-only headers (`ffi_client.h`, `lk_log.h`, etc.) | +| `src/tests/` | Google Test integration and stress tests | +| `examples/` | In-tree example applications | +| `bridge/` | **Deprecated** C-style bridge layer — do not add new functionality | +| `client-sdk-rust/` | Git submodule holding the Rust core of the SDK| +| `client-sdk-rust/livekit-ffi/protocol/*.proto` | FFI contract (protobuf definitions, read-only reference) | +| `cmake/` | Build helpers (`protobuf.cmake`, `spdlog.cmake`, `LiveKitConfig.cmake.in`) | +| `docker/` | Dockerfile for CI and SDK distribution images | +| `.github/workflows/` | GitHub Actions CI workflows | + +### Key Types + +- **`FfiClient`** — Singleton that sends FFI requests to Rust and dispatches event callbacks. Defined in `src/ffi_client.h` (internal). +- **`FfiHandle`** — RAII wrapper for Rust-owned resource handles; drop releases the FFI resource. +- **`Room`** — Central object for connecting to a LiveKit server room. +- **`RoomDelegate`** — Virtual callback interface for room lifecycle events. +- **`SubscriptionThreadDispatcher`** — Owns callback registrations and per-subscription reader threads for audio, video, and data tracks. `Room` delegates callback management here. +- **`LocalParticipant` / `RemoteParticipant`** — Participant objects for publishing and receiving tracks. + +## Build +for building, use the build.sh script for Linux and macOS, and the build.cmd script for Windows. Do not invoke CMake directly. + +``` +./build.sh debug # Debug build +./build.sh debug-tests # Debug build with tests +./build.sh debug-examples # Debug build with examples +./build.sh release # Release build +./build.sh release-tests # Release build with tests +./build.sh release-examples # Release build with examples +./build.sh build-all # All of the above +./build.sh clean # Clean build artifacts +./build.sh clean-all # Full clean (C++ + Rust targets) +``` + +**Requirements:** CMake 3.20+, C++17, Rust toolchain (cargo), protoc. On macOS: `brew install cmake ninja protobuf abseil spdlog`. On Linux: see the CI workflow for apt packages. + +### SDK Packaging + +``` +./build.sh release --bundle --prefix /path/to/install +``` + +This installs headers, libraries, and CMake config files so downstream projects can use `find_package(LiveKit CONFIG)`. + +## Coding Conventions + +### Copyright Header + +All source files must have the LiveKit Apache 2.0 copyright header. Use the current year for new files. Do not copyright externally pulled/cloned files. + +### Logging + +- Use `LK_LOG_*` macros from `src/lk_log.h` (internal header, **not** public API). +- `LK_LOG_ERROR` — when something is broken. +- `LK_LOG_WARN` — when something is unexpected but recoverable. +- `LK_LOG_INFO` — initialization, critical state changes. +- `LK_LOG_DEBUG` — potentially useful diagnostic info. +- Do not spam logs. Keep it concise. +- In production/internal SDK code, do not use `std::cout`, `std::cerr`, `printf`, or raw spdlog calls; use the logging APIs instead. Tests may use standard streams sparingly for ad hoc diagnostics when appropriate. +- The public logging API is `include/livekit/logging.h` (`setLogLevel`, `setLogCallback`). spdlog is a **private** dependency — it must never leak into public headers. + +### Public API Surface + +- Keep public APIs small. Minimize what goes into `include/livekit/`. +- Never introduce backwards compatibility breaking changes to the public API. +- `lk_log.h` lives under `src/` (internal). The public-facing logging API is `include/livekit/logging.h`. +- spdlog must not appear in any public header or installed header. + +### Error Handling + +- Throw exceptions for broken/unrecoverable states. +- Use `LK_LOG_WARN` for non-fatal unexpected conditions. +- Use `Result` for operations that can fail with typed errors (e.g., data track publish/subscribe). + +### Git Practices + +- Use `git mv` when moving or renaming files. + +### CMake + +- spdlog is linked **PRIVATE** to the `livekit` target. It must not appear in exported/installed dependencies. +- protobuf is vendored via FetchContent on non-Windows platforms; Windows uses vcpkg. +- The CMake install produces a `find_package(LiveKit CONFIG)`-compatible package with `LiveKitConfig.cmake`, `LiveKitTargets.cmake`, and `LiveKitConfigVersion.cmake`. + +### Readability and Performance +Code should be easy to read and understand. If a sacrifice is made for performance or readability, it should be documented. +## Dependencies + +| Dependency | Scope | Notes | +|------------|-------|-------| +| protobuf | Private (built-in) | Vendored via FetchContent (Unix) or vcpkg (Windows) | +| spdlog | **Private** | FetchContent or system package; must NOT leak into public API | +| client-sdk-rust | Build-time | Git submodule, built via cargo during CMake build | +| Google Test | Test only | FetchContent in `src/tests/CMakeLists.txt` | + +## Testing + +Tests are under `src/tests/` using Google Test: + +``` +./build.sh debug-tests +cd build-debug && ctest +``` + +Integration tests (`src/tests/integration/`) cover: room connections, callbacks, data tracks, RPC, logging, audio processing, and the subscription thread dispatcher. + +When adding new client facing functionality, add a new test case to the existing test suite. +When adding new client facing functionality, add benchmarking to understand the limitations of the new functionality. + +## Formatting +Adhere to the formatting rules if TODO: @alan + +## Deprecated / Out of Scope + +- **`bridge/`** (`livekit_bridge`) is deprecated. Do not add new functionality to it. + +## Common Pitfalls + +- A `Room` with `auto_subscribe = false` will never receive remote audio/video frames — this is almost never what you want. +- A single participant cannot subscribe to its own tracks as "remote." Testing sender/receiver requires two separate room connections with distinct identities. +- macOS dylibs require `install_name_tool` fixups for `@rpath` — the CMakeLists.txt handles this automatically. Do not manually adjust RPATH unless you understand the implications. +- When consuming the installed SDK, use `-DLIVEKIT_LOCAL_SDK_DIR=/path/to/sdk` to point cmake at a local install instead of downloading a release tarball. From 2d9ab5882db1aa720de80ed9eb4fb9d884a6a539 Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Tue, 14 Apr 2026 13:29:27 -0600 Subject: [PATCH 2/7] mr comments --- AGENTS.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a83e2f61..cc4ed20d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,11 +13,11 @@ ensure backwards compatibility is maintained when possible. ### Platform Support The SDK must be supported on the following platforms: -1. windows x64 -2. linux x86_64 -3. linux arm64 -3. macOS x86_64 -4. macOS arm64 +- windows x64 +- linux x86_64 +- linux arm64 +- macOS x86_64 +- macOS arm64 SDK features follow pattern: @@ -30,6 +30,7 @@ SDK features follow pattern: When making larger scale changes, check with the developer before committing to architecture changes involving changes to the `client-sdk-rust/` submodule. ### Directory Layout +Be sure to update the directory layout in this file if the directory layout changes. | Path | Description | |------|-------------| @@ -54,7 +55,8 @@ When making larger scale changes, check with the developer before committing to - **`LocalParticipant` / `RemoteParticipant`** — Participant objects for publishing and receiving tracks. ## Build -for building, use the build.sh script for Linux and macOS, and the build.cmd script for Windows. Do not invoke CMake directly. +for building, use the build.sh script for Linux and macOS, and the build.cmd script for Windows. Do not invoke CMake directly to build the SDK. +Updates to ./build.sh and ./build.cmd should be accompanied by updates to this file and the README.md file. ``` ./build.sh debug # Debug build @@ -104,7 +106,6 @@ All source files must have the LiveKit Apache 2.0 copyright header. Use the curr ### Error Handling -- Throw exceptions for broken/unrecoverable states. - Use `LK_LOG_WARN` for non-fatal unexpected conditions. - Use `Result` for operations that can fail with typed errors (e.g., data track publish/subscribe). From dec87bded96695a845f1209997516f245e1c6524 Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Tue, 14 Apr 2026 13:39:34 -0600 Subject: [PATCH 3/7] rm todo --- AGENTS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index cc4ed20d..ab62b480 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -145,7 +145,7 @@ When adding new client facing functionality, add a new test case to the existing When adding new client facing functionality, add benchmarking to understand the limitations of the new functionality. ## Formatting -Adhere to the formatting rules if TODO: @alan +Use clang formatting that aligns with the existing codebase. ## Deprecated / Out of Scope From b8e30456380785f0f9399446c7864d4d83c89cc9 Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Tue, 14 Apr 2026 13:44:56 -0600 Subject: [PATCH 4/7] note the synchronous/asynchonous calls of ffi --- AGENTS.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ab62b480..242aa7a9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,13 +20,21 @@ The SDK must be supported on the following platforms: - macOS arm64 -SDK features follow pattern: +SDK features follow one of two FFI patterns: +**Synchronous calls:** 1. Serialize a protobuf `FfiRequest` message. 2. Send it to Rust via `FfiClient::instance().sendRequest(req)`. -3. Receive a synchronous `FfiResponse` or an asynchronous `FfiEvent` callback. +3. Receive a `FfiResponse` with the result. 4. Expose the result through the C++ API. +**Asynchronous calls:** +1. Set up an async handler that listens for an event keyed by a `request_async_id`. +2. Serialize a protobuf `FfiRequest` message containing the `request_async_id`. +3. Send it to Rust via `FfiClient::instance().sendRequest(req)`. +4. Receive a synchronous `FfiResponse` (acknowledgement) and, later, an asynchronous `FfiEvent` callback with the actual result. +5. Expose the result through the C++ API. + When making larger scale changes, check with the developer before committing to architecture changes involving changes to the `client-sdk-rust/` submodule. ### Directory Layout From 80513c0d001a39a40b1a704f03be61bc2d0babf0 Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Tue, 14 Apr 2026 13:47:51 -0600 Subject: [PATCH 5/7] note that cpp should be a thin wrapper --- AGENTS.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 242aa7a9..9c3cfe63 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,10 +7,9 @@ This is **client-sdk-cpp**, the official LiveKit C++ client SDK. It wraps a Rust ## Architecture ### Core Principle -Rust owns as much of the business logic as possible. If a feature may be used by another SDK it should be implemented in Rust. Since this is an SDK, +Rust owns as much of the business logic as possible. The C++ layer should be a thin wrapper around the Rust core. If a feature may be used by another SDK it should be implemented in Rust. Since this is an SDK, ensure backwards compatibility is maintained when possible. - ### Platform Support The SDK must be supported on the following platforms: - windows x64 From e434d343b1b5e04401f98fd2d14098e1fe9364ec Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Tue, 14 Apr 2026 14:10:56 -0600 Subject: [PATCH 6/7] explain threading in AGENTS.md --- AGENTS.md | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 9c3cfe63..95c73c9d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,7 +18,7 @@ The SDK must be supported on the following platforms: - macOS x86_64 - macOS arm64 - +### FFI Patterns SDK features follow one of two FFI patterns: **Synchronous calls:** @@ -36,6 +36,35 @@ SDK features follow one of two FFI patterns: When making larger scale changes, check with the developer before committing to architecture changes involving changes to the `client-sdk-rust/` submodule. +### Threading Model + +The SDK has three categories of threads: + +**FFI callback thread** — The Rust FFI layer calls `LivekitFfiCallback` from a Rust-managed thread (typically a Tokio runtime thread). This single entry point deserializes the `FfiEvent` and calls `FfiClient::PushEvent`, which: +1. Completes any pending async `std::promise` matched by `async_id`. +2. Invokes all registered `FfiClient` listeners (including `Room::OnEvent`). + +All `RoomDelegate` callbacks and stream handler callbacks (e.g., `registerTextStreamHandler`) are invoked on this FFI callback thread. **Handlers must not block**; spawn a background thread if synchronous work is needed. + +**Per-subscription reader threads** — `SubscriptionThreadDispatcher` creates a dedicated `std::thread` for each active audio, video, or data track subscription. These threads block on `AudioStream::read()`, `VideoStream::read()`, or `DataTrackStream::read()` and invoke the registered `AudioFrameCallback`, `VideoFrameCallback`, or `DataFrameCallback` on that reader thread — not on the FFI callback thread. A hard limit of 20 concurrent reader threads is enforced. + +**Application threads** — The calling thread for public API methods such as `Room::Connect`, `LocalParticipant::publishTrack`, `AudioSource::captureFrame`, etc. These may block while waiting for FFI responses or future completion. + +#### Thread-safety guarantees + +| Component | Thread-safe? | Notes | +|-----------|-------------|-------| +| `FfiClient::sendRequest` | Yes (C++ side) | No C++ mutex; relies on the Rust FFI being safe for concurrent calls. Multiple threads may call concurrently. | +| `FfiClient` listener/async registration | Yes | Protected by internal `std::mutex`. | +| `Room` | Yes | Internal `std::mutex` protects all mutable state. `RoomDelegate` is called outside the lock. | +| `SubscriptionThreadDispatcher` | Yes | Internal `std::mutex` protects registrations and active readers. Thread joins happen outside the lock. | +| `AudioStream` / `VideoStream` / `DataTrackStream` | Yes | Internal `std::mutex` + `condition_variable` coordinate the FFI producer thread and the consumer reader thread. | +| `AudioSource::captureFrame` | No | Not safe to call concurrently from multiple threads. | +| `VideoSource::captureFrame` | No | Not safe to call concurrently from multiple threads. | +| `LocalAudioTrack` / `LocalVideoTrack` | No | Thin `sendRequest` wrappers with no internal synchronization. | +| `LocalDataTrack::tryPush` | No | Thin `sendRequest` wrapper with no internal synchronization. | +| `TextStreamWriter` / `ByteStreamWriter` | Serialized | `write()` is serialized by an internal `write_mutex_`. | + ### Directory Layout Be sure to update the directory layout in this file if the directory layout changes. From 9fcf398bedf07216690fc09ab3d30813546beafb Mon Sep 17 00:00:00 2001 From: Stephen DeRosa Date: Tue, 14 Apr 2026 14:18:13 -0600 Subject: [PATCH 7/7] a few of NASAs 10 rules for SWD --- AGENTS.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 95c73c9d..e5cb5f32 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -183,6 +183,13 @@ When adding new client facing functionality, add benchmarking to understand the ## Formatting Use clang formatting that aligns with the existing codebase. +## General C++ Development +- Do not use dynamic memory allocation after initialization +- Keep each function short (roughly ≤ 60 lines) +- Declare all data objects at the smallest possible level of scope +- Each calling function must check the return value of nonvoid functions, and each called function must check the validity of all parameters provided by the caller + + ## Deprecated / Out of Scope - **`bridge/`** (`livekit_bridge`) is deprecated. Do not add new functionality to it.