feat(cli): add --secret-material-env to provider refresh configure - #2178
Conversation
|
All contributors have signed the DCO ✍️ ✅ |
|
I have read the DCO document and I hereby sign the DCO. |
Signed-off-by: Hung Le <hple@nvidia.com>
|
Label |
PR Review StatusValidation: This PR is project-valid because it implements a focused CLI-side improvement for the provider refresh secret-material workflow described in #2104. I checked @johntmyers's maintainer feedback on #2104 about duplicate-key rejection and off-argv secret handling, and @hunglp6d's PR implements the env-based path with duplicate rejection while leaving the file/stdin path for follow-up work. Review findings:
Docs: Updated in E2E: I applied Next state: |
Monitoring CompleteMonitoring is complete because this PR has merged. Final status: The last active gator state was I removed the active |
<!-- markdownlint-disable MD041 --> ## Summary Adds Google Chat as an experimental OpenClaw messaging channel, served at `/googlechat` on the gateway through a dedicated Google-Chat-only cloudflared tunnel and a loopback webhook proxy. Outbound auth is minted gateway-side (OpenShell `google-service-account-jwt` provider) and injected by the L7 proxy, so the service-account private key never enters the sandbox. Inbound cert verification and outbound replies both route by hostname through the trusted L7 proxy (the SSRF guard's trusted-proxy modes), so the channel works in the DNS-less proxy-only sandbox — no local DNS resolve, no sentinel IP. **Scope: OpenClaw only.** Hermes uses a different inbound model (Cloud Pub/Sub pull, no webhook) and a different credential path, so Hermes Google Chat is a planned follow-up rather than part of this PR. Supersedes #6120: same change, recommitted as a single verified commit with clean history. ## Architecture Google Chat is the first bridge channel: inbound webhooks are verified inside the sandbox, but the outbound bot token is minted on the host and the service-account private key never enters the sandbox. Two JWTs run in opposite directions — Google signs one that the plugin **verifies** (inbound), and the gateway signs one to **mint** the outbound token (outbound). ```text INBOUND — Google → bot (Google signs a JWT; the plugin VERIFIES it) [GOOGLE] chat.googleapis.com — signs a JWT, POSTs the webhook │ ▼ [HOST] cloudflared (dedicated tunnel) │ ▼ [HOST] loopback webhook proxy — admits only POST /googlechat, 404s all else │ (the public URL is a webhook pinhole, not the dashboard) ▼ [HOST] OpenClaw gateway :18789 — routes /googlechat into the sandbox │ ▼ [SANDBOX] @openclaw/googlechat plugin │ VERIFY the JWT — fetch Google's certs (GET www.googleapis.com, out through the │ L7 proxy via trusted-proxy-fetch) → check signature + audience + appPrincipal ▼ [SANDBOX] agent handles the message OUTBOUND — bot → Google (the gateway GENERATES the token; the L7 proxy swaps it in — key never in the sandbox) [SANDBOX] @openclaw/googlechat plugin — builds the reply, stamps «Authorization: Bearer <placeholder>» │ outbound-auth put the placeholder there → the sandbox holds no key, no real token; │ trusted-proxy-fetch routes the send out through the L7 proxy (no local DNS) ▼ [HOST] L7 egress proxy │ ① swap «placeholder» → the real Bearer minted by the gateway: │ GENERATE — sign an SA-JWT with private_key → Google OAuth → access token (cached ~1h) │ ② policy-check — chat.googleapis.com, POST/PATCH/DELETE within /v1/spaces ▼ [GOOGLE] chat.googleapis.com — user sees the reply Note — trusted-proxy-fetch is ROUTING (every plugin→Google call goes via the L7 proxy, no local DNS), so it appears in both flows: the inbound cert fetch and the outbound send. The token swap, the outbound-auth placeholder, and the JWT verify/generate are each one-directional. ``` ## OpenShell Requirement The Google Chat service-account private key is configured through OpenShell's `--secret-material-env` ([OpenShell#2178](NVIDIA/OpenShell#2178)), which keeps it off process arguments. That flag ships in OpenShell v0.0.82+; current `main` pins OpenShell 0.0.85, so the requirement is already satisfied on `main` — no prerequisite PR. ## Related Issue Parts of #5492 ## Result <img width="582" height="800" alt="image" src="https://github.com/user-attachments/assets/02153f1f-ed55-4e62-9163-17271cacd895" /> ## Changes **Channel** - New `googlechat` manifest (inputs, enrollment notes, render) + built-in registration; served on the shared gateway dashboard port at `/googlechat` (no host forward — on its own dedicated tunnel, see below). - Enroll hooks: tunnel/audience gate (derives the public webhook URL from the cloudflared tunnel), token-paste for the SA JSON (validates the paste is parseable JSON and re-prompts before skip), config prompts (audience, appPrincipal, DM allowlist); channel template resolver + `nemoclaw tunnel` service wiring. - **Dedicated webhook tunnel.** A Google-Chat-only cloudflared tunnel (separate from `nemoclaw tunnel start`) fronts a loopback-only webhook proxy that forwards **only** `POST /googlechat` (1 MiB body cap) and returns 404 for every other method or path, so the public URL is a narrow webhook pinhole that never exposes the dashboard or control UI. NemoClaw tears down both the proxy and its cloudflared on skip, remove, and destroy. - Always skips in non-interactive mode (mirrors WeChat host-QR): enrollment needs manual, out-of-band Google Cloud Console + appPrincipal steps that no environment variable can satisfy, so the tunnel/audience gate skips rather than enroll a half-configured channel that silently 404s on inbound webhooks. - appPrincipal discovery sentinel: a blank appPrincipal renders an all-zeros placeholder so the first DM logs `unexpected add-on principal: <N>` (the real value to copy) instead of a numberless `missing add-on principal binding`; inert for Google Workspace accounts. Enrollment notes are rewritten into an always-printed capture guide with the exact `logs --follow` command and the persist-and-rebuild steps. **Gateway reliability** - Renders `gateway.reload.mode=off`. OpenClaw rewrites its own config ~60s after boot (auto-enabling default provider plugins); with reactive hot-reload on, that self-write rebuilds the gateway's HTTP route table and drops the Google Chat inbound webhook route, so `/googlechat` starts returning 404 and the bot goes silent ~60s after every start. The sandbox's `openclaw.json` is sealed at build time (0600 + integrity hash), so disabling reactive reload is safe; NemoClaw still restarts the gateway explicitly on rebuild / `gateway restart`. **Inbound + outbound routing (proxy-only sandbox)** - `googlechat-trusted-proxy-fetch` boot preload: rewrites the plugin's three googleapis fetch sites (inbound cert verify + all outbound sends/edits) to the SSRF guard's `trusted_env_proxy`/`trusted_explicit_proxy` modes, so they skip the local `getaddrinfo` and route by hostname through the L7 proxy — fixes `EAI_AGAIN` in the DNS-less netns, no sentinel IP. Interim; clean fix is upstream in OpenClaw (like web_fetch #50650). - `googlechat` network-policy preset scoped to the Chat spaces tree, including `PATCH` for streaming message edits. **Outbound auth (key out of sandbox)** - `google-chat-bridge` OpenShell provider profile + `googlechat-bridge-provider` onboard wiring: import the profile pre-create, run `provider refresh configure --strategy google-service-account-jwt` post-create from the pasted SA JSON — the gateway mints/rotates the token and the L7 proxy injects `Authorization: Bearer` on chat.googleapis.com. The private key is supplied through `--secret-material-env`, so it never reaches argv; the profile authorizes only the Node runtime for the injected bearer. - `googlechat-outbound-auth` boot preload: load-time rewrite of the plugin's single token producer to emit the OpenShell credential placeholder (revision-less alias, so it resolves to the latest re-minted token and survives rotation) instead of signing in-process. - Drops in-sandbox SA-key file delivery for outbound; `serviceAccountFile` is kept only as a channel-start-gate marker (never delivered or read); keeps the `BEGIN PRIVATE KEY` secret scan. **Documentation** - Adds OpenClaw-only setup, onboarding, lifecycle, and security guidance. The docs build completed with zero errors; two pre-existing warnings remain (redirect authentication and light-accent contrast). ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior — unit tests for the two boot preloads (anchor rewrite, idempotency, drift-throw, runtime short-circuit), the bridge provider (env resolution, fail-closed refresh, Node-only credential boundary), the tunnel/audience gate (unconditional non-interactive skip, audience does not bypass), the template resolver (appPrincipal discovery sentinel when blank), and a manifest runtime-security-contract test (both boot preloads present and non-optional, `gateway.reload.mode=off` rendered, the non-existent `serviceAccountFile` sentinel, and no credentials delivered into the sandbox) - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — [nine-category review on the predecessor #6120](#6120 (comment)); exact current-main delta review PASS at `8897acb3fbbffcfba59eb8ad27238e566d6292ad`, feature diff fingerprint `2badf49554a0bc79fbd104eb88d1f4dfba26ba6f9c52a0b848c0d191653e1e8b`. The sync changes no Google Chat feature behavior. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Exact-head revalidation: `8897acb3fb`; current-main synced. The exact-head Google Chat suite passed 69/69 with repository checks and normal pre-push CLI/type/version gates; the unchanged feature diff previously passed 106 Google Chat CLI tests + 6 E2E-support tests, the standalone loader contract, and `npm run validate:pr`. Feature diff fingerprint: `2badf49554a0bc79fbd104eb88d1f4dfba26ba6f9c52a0b848c0d191653e1e8b`; nine-category security delta review PASS because the follow-up uses Vitest-tracked environment stubs only and the standalone E2E bridge retains the same sandbox-name and process-local capability boundaries. The prior exact-head docs build reported 0 errors and two pre-existing warnings; the later current-main sync only imported the already-reviewed release changelog and requires no Google Chat docs change. - Evidence: Reviewed the latest Google Chat doc changes — set-up-google-chat.mdx (agent-variant scope + non-interactive-skip wording), enable-channels-during-onboarding.mdx (non-interactive-skip wording + app-principal term consistency), and the NEMOCLAW_SKIP_GOOGLECHAT_TUNNEL row in docs/reference/commands.mdx. Accurate and style-compliant; `npm run docs:prepare` exits 0, markdownlint clean. No blocking findings. - Agent: Claude Code <!-- docs-review-head-sha: 8897acb --> <!-- docs-review-agents-blob-sha: c052d60 --> ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [x] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Hung Le <hple@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added experimental Google Chat messaging for OpenClaw sandboxes. * Added interactive setup with service-account credentials, public HTTPS webhooks, audience controls, and user allowlists. * Added lifecycle controls for the dedicated webhook endpoint. * **Bug Fixes** * Improved cleanup reliability when webhook shutdown fails, preserving channel state for retry. * Improved fail-closed behavior for incomplete or invalid configuration. * **Documentation** * Added Google Chat setup and onboarding guidance. * **Security** * Added credential validation, masked secret entry, protected token handling, and restricted webhook routing. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Hung Le <hple@nvidia.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: Apurv Kumaria <akumaria@nvidia.com> Co-authored-by: San Dang <sdang@nvidia.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Senthil Ravichandran <senthilr@nvidia.com>
Summary
Adds
--secret-material-env KEY[=ENVVAR]toprovider refresh configureso secretrefresh material (service-account private keys, OAuth client secrets) can be supplied
off-argv. Today
--material KEY=VALUEis the only ingestion path, which exposes thevalue in the host process table (
/proc/<pid>/cmdline,ps) and to command-line audittooling. The new flag resolves the value from the CLI's own environment — mirroring the
existing
--credential KEYingestion — merges it into the material map, and auto-addsthe key to
secret_material_keys. Duplicate keys are rejected rather than givenprecedence: the same
KEYsupplied via both--materialand the new flag, or repeatedwithin the flag, is an error. Missing or blank env fails closed naming the variable,
before any request is sent. Client-side only: the gRPC request, server, and refresh
strategies are unchanged.
Related Issue
Refs #2104 (implements the env-styled complementary option;
--secret-material-file/--material-stdinfrom the proposal remain open).Changes
openshell-cli/src/main.rs: new--secret-material-env KEY[=ENVVAR]flag onprovider refresh configure; CLI parse test extended.openshell-cli/src/run.rs:parse_secret_material_env_pairs(error style mirrors the--credentialenv ingestion; never echoes values) and the merge inprovider_refresh_config; 5 unit tests.openshell-cli/tests/provider_commands_integration.rs: the mock now recordsmaterial/secret_material_keys; new end-to-end tests (env value reaches the requestauto-marked secret; a
--materialduplicate is rejected client-side before anyrequest) and a fails-closed test (unset env → error names the variable, no request
leaves the client).
docs/sandboxes/manage-providers.mdx: the Credential Refresh example no longerexpands a secret onto argv; documents the new flag.
Testing
mise run pre-commitpasses — the full alias (workspace-wide lint+check+test)does not fit on my dev box; ran every lane the diff touches manually:
cargo test -p openshell-cli(351 passed, 0 failed),cargo fmt -p openshell-cli -- --check(clean),cargo clippy -p openshell-cli --all-targets(no warnings),markdownlint-cli2over the repo docs globs (101 files, 0 errors).Deferring the untouched python/helm lanes to CI.
Also manually verified against a stock v0.0.78 gateway (release deb install; this
branch's CLI
0.0.79-devtalking to an unmodified released server):argv exposure: old path reproduces it, new flag closes it
fails closed when the env var is unset
value round-trips intact to the mint path; status never echoes material
(the fake env-delivered value reached the server and was parsed at mint time — the
transport carried it verbatim; no surface echoes the material itself)
Checklist