Skip to content

feat(cli): add --secret-material-env to provider refresh configure - #2178

Merged
johntmyers merged 2 commits into
NVIDIA:mainfrom
hunglp6d:feat/cli-secret-material-env
Jul 9, 2026
Merged

feat(cli): add --secret-material-env to provider refresh configure#2178
johntmyers merged 2 commits into
NVIDIA:mainfrom
hunglp6d:feat/cli-secret-material-env

Conversation

@hunglp6d

@hunglp6d hunglp6d commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds --secret-material-env KEY[=ENVVAR] to provider refresh configure so secret
refresh material (service-account private keys, OAuth client secrets) can be supplied
off-argv. Today --material KEY=VALUE is the only ingestion path, which exposes the
value in the host process table (/proc/<pid>/cmdline, ps) and to command-line audit
tooling. The new flag resolves the value from the CLI's own environment — mirroring the
existing --credential KEY ingestion — merges it into the material map, and auto-adds
the key to secret_material_keys. Duplicate keys are rejected rather than given
precedence: the same KEY supplied via both --material and the new flag, or repeated
within the flag, is an error. Missing or blank env fails closed naming the variable,
before any request is sent. Client-side only: the gRPC request, server, and refresh
strategies are unchanged.

Related Issue

Refs #2104 (implements the env-styled complementary option; --secret-material-file /
--material-stdin from the proposal remain open).

Changes

  • openshell-cli/src/main.rs: new --secret-material-env KEY[=ENVVAR] flag on
    provider refresh configure; CLI parse test extended.
  • openshell-cli/src/run.rs: parse_secret_material_env_pairs (error style mirrors the
    --credential env ingestion; never echoes values) and the merge in
    provider_refresh_config; 5 unit tests.
  • openshell-cli/tests/provider_commands_integration.rs: the mock now records
    material/secret_material_keys; new end-to-end tests (env value reaches the request
    auto-marked secret; a --material duplicate is rejected client-side before any
    request) and a fails-closed test (unset env → error names the variable, no request
    leaves the client).
  • docs/sandboxes/manage-providers.mdx: the Credential Refresh example no longer
    expands a secret onto argv; documents the new flag.

Testing

  • Unit tests added/updated
  • E2E tests added/updated (if applicable) — n/a, covered by crate integration tests
  • mise run pre-commit passes — the full alias (workspace-wide lint+check+test)
    does not fit on my dev box; ran every lane the diff touches manually:
    cargo test -p openshell-cli (351 passed, 0 failed),
    cargo fmt -p openshell-cli -- --check (clean),
    cargo clippy -p openshell-cli --all-targets (no warnings),
    markdownlint-cli2 over the repo docs globs (101 files, 0 errors).
    Deferring the untouched python/helm lanes to CI.

Also manually verified against a stock v0.0.78 gateway (release deb install; this
branch's CLI 0.0.79-dev talking to an unmodified released server):

argv exposure: old path reproduces it, new flag closes it
# old path (stock binary, --material): secret visible to any local user
$ openshell provider refresh configure sme-test ... --material private_key=SIEU-BI-MAT-123 ... &
$ cat /proc/$!/cmdline | tr '\0' ' '
openshell provider refresh configure sme-test ... --material private_key=SIEU-BI-MAT-123 ...

# new flag (this branch): argv carries only the variable NAME
$ read -rs PK_TEST; export PK_TEST
$ osnew provider refresh configure sme-test ... --secret-material-env private_key=PK_TEST &
$ cat /proc/$!/cmdline | tr '\0' ' '
.../openshell provider refresh configure sme-test ... --secret-material-env private_key=PK_TEST
✓ Configured refresh for sme-test SME_TEST_TOKEN
fails closed when the env var is unset
$ unset PK_TEST
$ osnew provider refresh configure sme-test ... --secret-material-env private_key=PK_TEST
Error:   × --secret-material-env private_key requires local env var 'PK_TEST' to be set to a non-empty value
value round-trips intact to the mint path; status never echoes material
$ openshell provider refresh rotate sme-test --credential-key SME_TEST_TOKEN
Error:   × code: 'Client specified an invalid argument', message: "google_service_account_jwt private_key must be RSA PEM"

$ openshell provider refresh status sme-test
PROVIDER   CREDENTIAL_KEY   STRATEGY                     STATUS  ...  LAST_ERROR
sme-test   SME_TEST_TOKEN   google_service_account_jwt   error   ...  google_service_account_jwt private_key must be RSA PEM

(the fake env-delivered value reached the server and was parsed at mint time — the
transport carried it verbatim; no surface echoes the material itself)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable) — n/a (user docs updated instead)

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

All contributors have signed the DCO ✍️ ✅
Posted by the DCO Assistant Lite bot.

@hunglp6d

hunglp6d commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

I have read the DCO document and I hereby sign the DCO.

Signed-off-by: Hung Le <hple@nvidia.com>
@johntmyers johntmyers added gator:watch-pipeline Gator is monitoring PR CI/CD status test:e2e Requires end-to-end coverage labels Jul 8, 2026
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Label test:e2e applied for d426bb8. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements a focused CLI-side improvement for the provider refresh secret-material workflow described in #2104. I checked @johntmyers's maintainer feedback on #2104 about duplicate-key rejection and off-argv secret handling, and @hunglp6d's PR implements the env-based path with duplicate rejection while leaving the file/stdin path for follow-up work.
Head SHA: d426bb84d6ad77cfae9a84c84682e4aaa96ea30e

Review findings:

  • No blocking findings remain.
  • Non-blocking reviewer suggestion: parse_secret_material_env_pairs returns a HashMap, so multiple env-sourced keys may be auto-added to secret_material_keys in nondeterministic order. This appears harmless if the server treats the list as a set, but preserving input order would make stored requests and tests more predictable.

Docs: Updated in docs/sandboxes/manage-providers.mdx; no docs/index.yml navigation change is needed because the page is already under the included sandboxes folder.

E2E: I applied test:e2e because this touches provider credential flow.

Next state: gator:watch-pipeline

@johntmyers johntmyers added gator:merge-ready and removed gator:watch-pipeline Gator is monitoring PR CI/CD status labels Jul 9, 2026
@johntmyers
johntmyers merged commit 83131d7 into NVIDIA:main Jul 9, 2026
66 of 68 checks passed
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: The last active gator state was gator:merge-ready; maintainer approval was present and the PR has now been merged.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.

prekshivyas pushed a commit to NVIDIA/NemoClaw that referenced this pull request Aug 1, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Adds Google Chat as an experimental OpenClaw messaging channel, served
at `/googlechat` on the gateway through a dedicated Google-Chat-only
cloudflared tunnel and a loopback webhook proxy. Outbound auth is minted
gateway-side (OpenShell `google-service-account-jwt` provider) and
injected by the L7 proxy, so the service-account private key never
enters the sandbox. Inbound cert verification and outbound replies both
route by hostname through the trusted L7 proxy (the SSRF guard's
trusted-proxy modes), so the channel works in the DNS-less proxy-only
sandbox — no local DNS resolve, no sentinel IP.

**Scope: OpenClaw only.** Hermes uses a different inbound model (Cloud
Pub/Sub pull, no webhook) and a different credential path, so Hermes
Google Chat is a planned follow-up rather than part of this PR.

Supersedes #6120: same change, recommitted as a single verified commit
with clean history.

## Architecture

Google Chat is the first bridge channel: inbound webhooks are verified
inside the sandbox, but the outbound bot token is minted on the host and
the service-account private key never enters the sandbox. Two JWTs run
in opposite directions — Google signs one that the plugin **verifies**
(inbound), and the gateway signs one to **mint** the outbound token
(outbound).

```text
INBOUND — Google → bot   (Google signs a JWT; the plugin VERIFIES it)

   [GOOGLE]   chat.googleapis.com — signs a JWT, POSTs the webhook
                  │
                  ▼
   [HOST]     cloudflared (dedicated tunnel)
                  │
                  ▼
   [HOST]     loopback webhook proxy — admits only POST /googlechat, 404s all else
                  │                    (the public URL is a webhook pinhole, not the dashboard)
                  ▼
   [HOST]     OpenClaw gateway :18789 — routes /googlechat into the sandbox
                  │
                  ▼
   [SANDBOX]  @openclaw/googlechat plugin
                  │  VERIFY the JWT — fetch Google's certs (GET www.googleapis.com, out through the
                  │  L7 proxy via trusted-proxy-fetch) → check signature + audience + appPrincipal
                  ▼
   [SANDBOX]  agent handles the message


OUTBOUND — bot → Google   (the gateway GENERATES the token; the L7 proxy swaps it in — key never in the sandbox)

   [SANDBOX]  @openclaw/googlechat plugin — builds the reply, stamps «Authorization: Bearer <placeholder>»
                  │  outbound-auth put the placeholder there → the sandbox holds no key, no real token;
                  │  trusted-proxy-fetch routes the send out through the L7 proxy (no local DNS)
                  ▼
   [HOST]     L7 egress proxy
                  │  ① swap «placeholder» → the real Bearer minted by the gateway:
                  │      GENERATE — sign an SA-JWT with private_key → Google OAuth → access token (cached ~1h)
                  │  ② policy-check — chat.googleapis.com, POST/PATCH/DELETE within /v1/spaces
                  ▼
   [GOOGLE]   chat.googleapis.com — user sees the reply

  Note — trusted-proxy-fetch is ROUTING (every plugin→Google call goes via the L7 proxy, no local DNS),
  so it appears in both flows: the inbound cert fetch and the outbound send. The token swap, the
  outbound-auth placeholder, and the JWT verify/generate are each one-directional.
```

## OpenShell Requirement
The Google Chat service-account private key is configured through
OpenShell's `--secret-material-env`
([OpenShell#2178](NVIDIA/OpenShell#2178)), which
keeps it off process arguments. That flag ships in OpenShell v0.0.82+;
current `main` pins OpenShell 0.0.85, so the requirement is already
satisfied on `main` — no prerequisite PR.

## Related Issue
Parts of #5492

## Result
<img width="582" height="800" alt="image"
src="https://github.com/user-attachments/assets/02153f1f-ed55-4e62-9163-17271cacd895"
/>

## Changes
**Channel**
- New `googlechat` manifest (inputs, enrollment notes, render) +
built-in registration; served on the shared gateway dashboard port at
`/googlechat` (no host forward — on its own dedicated tunnel, see
below).
- Enroll hooks: tunnel/audience gate (derives the public webhook URL
from the cloudflared tunnel), token-paste for the SA JSON (validates the
paste is parseable JSON and re-prompts before skip), config prompts
(audience, appPrincipal, DM allowlist); channel template resolver +
`nemoclaw tunnel` service wiring.
- **Dedicated webhook tunnel.** A Google-Chat-only cloudflared tunnel
(separate from `nemoclaw tunnel start`) fronts a loopback-only webhook
proxy that forwards **only** `POST /googlechat` (1 MiB body cap) and
returns 404 for every other method or path, so the public URL is a
narrow webhook pinhole that never exposes the dashboard or control UI.
NemoClaw tears down both the proxy and its cloudflared on skip, remove,
and destroy.
- Always skips in non-interactive mode (mirrors WeChat host-QR):
enrollment needs manual, out-of-band Google Cloud Console + appPrincipal
steps that no environment variable can satisfy, so the tunnel/audience
gate skips rather than enroll a half-configured channel that silently
404s on inbound webhooks.
- appPrincipal discovery sentinel: a blank appPrincipal renders an
all-zeros placeholder so the first DM logs `unexpected add-on principal:
<N>` (the real value to copy) instead of a numberless `missing add-on
principal binding`; inert for Google Workspace accounts. Enrollment
notes are rewritten into an always-printed capture guide with the exact
`logs --follow` command and the persist-and-rebuild steps.

**Gateway reliability**
- Renders `gateway.reload.mode=off`. OpenClaw rewrites its own config
~60s after boot (auto-enabling default provider plugins); with reactive
hot-reload on, that self-write rebuilds the gateway's HTTP route table
and drops the Google Chat inbound webhook route, so `/googlechat` starts
returning 404 and the bot goes silent ~60s after every start. The
sandbox's `openclaw.json` is sealed at build time (0600 + integrity
hash), so disabling reactive reload is safe; NemoClaw still restarts the
gateway explicitly on rebuild / `gateway restart`.

**Inbound + outbound routing (proxy-only sandbox)**
- `googlechat-trusted-proxy-fetch` boot preload: rewrites the plugin's
three googleapis fetch sites (inbound cert verify + all outbound
sends/edits) to the SSRF guard's
`trusted_env_proxy`/`trusted_explicit_proxy` modes, so they skip the
local `getaddrinfo` and route by hostname through the L7 proxy — fixes
`EAI_AGAIN` in the DNS-less netns, no sentinel IP. Interim; clean fix is
upstream in OpenClaw (like web_fetch #50650).
- `googlechat` network-policy preset scoped to the Chat spaces tree,
including `PATCH` for streaming message edits.

**Outbound auth (key out of sandbox)**
- `google-chat-bridge` OpenShell provider profile +
`googlechat-bridge-provider` onboard wiring: import the profile
pre-create, run `provider refresh configure --strategy
google-service-account-jwt` post-create from the pasted SA JSON — the
gateway mints/rotates the token and the L7 proxy injects `Authorization:
Bearer` on chat.googleapis.com. The private key is supplied through
`--secret-material-env`, so it never reaches argv; the profile
authorizes only the Node runtime for the injected bearer.
- `googlechat-outbound-auth` boot preload: load-time rewrite of the
plugin's single token producer to emit the OpenShell credential
placeholder (revision-less alias, so it resolves to the latest re-minted
token and survives rotation) instead of signing in-process.
- Drops in-sandbox SA-key file delivery for outbound;
`serviceAccountFile` is kept only as a channel-start-gate marker (never
delivered or read); keeps the `BEGIN PRIVATE KEY` secret scan.

**Documentation**
- Adds OpenClaw-only setup, onboarding, lifecycle, and security
guidance. The docs build completed with zero errors; two pre-existing
warnings remain (redirect authentication and light-accent contrast).

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
- [x] Tests added or updated for changed behavior — unit tests for the
two boot preloads (anchor rewrite, idempotency, drift-throw, runtime
short-circuit), the bridge provider (env resolution, fail-closed
refresh, Node-only credential boundary), the tunnel/audience gate
(unconditional non-interactive skip, audience does not bypass), the
template resolver (appPrincipal discovery sentinel when blank), and a
manifest runtime-security-contract test (both boot preloads present and
non-optional, `gateway.reload.mode=off` rendered, the non-existent
`serviceAccountFile` sentinel, and no credentials delivered into the
sandbox)
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — [nine-category review on the predecessor
#6120](#6120 (comment));
exact current-main delta review PASS at
`8897acb3fbbffcfba59eb8ad27238e566d6292ad`, feature diff fingerprint
`2badf49554a0bc79fbd104eb88d1f4dfba26ba6f9c52a0b848c0d191653e1e8b`. The
sync changes no Google Chat feature behavior.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review
- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Exact-head revalidation: `8897acb3fb`; current-main synced. The
exact-head Google Chat suite passed 69/69 with repository checks and
normal pre-push CLI/type/version gates; the unchanged feature diff
previously passed 106 Google Chat CLI tests + 6 E2E-support tests, the
standalone loader contract, and `npm run validate:pr`. Feature diff
fingerprint:
`2badf49554a0bc79fbd104eb88d1f4dfba26ba6f9c52a0b848c0d191653e1e8b`;
nine-category security delta review PASS because the follow-up uses
Vitest-tracked environment stubs only and the standalone E2E bridge
retains the same sandbox-name and process-local capability boundaries.
The prior exact-head docs build reported 0 errors and two pre-existing
warnings; the later current-main sync only imported the already-reviewed
release changelog and requires no Google Chat docs change.
- Evidence: Reviewed the latest Google Chat doc changes —
set-up-google-chat.mdx
(agent-variant scope + non-interactive-skip wording),
enable-channels-during-onboarding.mdx
(non-interactive-skip wording + app-principal term consistency), and the
NEMOCLAW_SKIP_GOOGLECHAT_TUNNEL row in docs/reference/commands.mdx.
Accurate and
style-compliant; `npm run docs:prepare` exits 0, markdownlint clean. No
blocking findings.
- Agent: Claude Code
<!-- docs-review-head-sha: 8897acb -->
<!-- docs-review-agents-blob-sha: c052d60 -->

## Verification
- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [x] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Hung Le <hple@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added experimental Google Chat messaging for OpenClaw sandboxes.
* Added interactive setup with service-account credentials, public HTTPS
webhooks, audience controls, and user allowlists.
  * Added lifecycle controls for the dedicated webhook endpoint.
* **Bug Fixes**
* Improved cleanup reliability when webhook shutdown fails, preserving
channel state for retry.
* Improved fail-closed behavior for incomplete or invalid configuration.
* **Documentation**
  * Added Google Chat setup and onboarding guidance.
* **Security**
* Added credential validation, masked secret entry, protected token
handling, and restricted webhook routing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Hung Le <hple@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: Apurv Kumaria <akumaria@nvidia.com>
Co-authored-by: San Dang <sdang@nvidia.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Senthil Ravichandran <senthilr@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants