feat(help): support JSON output for --help flags - #1337
Conversation
Rewrite flag-based `--help --json` requests to the dedicated `help` command in argv preprocessing, so `sentry --help --json` and `sentry <command> --help --json` emit the same structured JSON as `sentry help --json`. Agents and tooling reach for `--help` first, but Stricli handles it internally and ignores `--json`. Only fires when both `--help` and `--json` are present before any `--` escape; a bare `--help` keeps Stricli's text usage output unchanged. `--fields` is carried through for field selection. Fixes #1265
|
|
fix-ci: attempt 1 — the failure is |
|
Marked ready for review. All CI is green — Unit Tests, E2E Tests, Lint & Typecheck, and the security/CodeQL/semgrep checks all passed (the earlier E2E failure was a flaky bundle-build race that cleared on re-run). Self-review found nothing to change. Flagging that the risk-scoring workflow labeled this |
Two issues in scanHelpJsonToken when a --help --json request carried other flags: - A dropped value flag kept its spaced value, so `--org acme` / `--limit 5` pushed `acme` / `5` into the command path and resolved the wrong command (or a not-found error). Value flags now drop their spaced value too; known boolean flags (`--verbose`) still leave the following token as a real path segment. - `--fields` unconditionally consumed the next token, so `--fields --json` swallowed `--json` and the rewrite never fired. It now only takes a spaced value when the next token isn't a flag. Reuses the existing GLOBAL_FLAGS metadata to tell boolean from value-taking flags. Added tests for value-flag dropping, boolean-flag path retention, and the --fields swallow case.
scanHelpJsonToken treated any --flag=value token as possibly value-taking and dropped the following token as its spaced value. Since isBooleanFlagToken looks up the full name including =value, an =-form flag never matched a known boolean and its next token was discarded, so `--org=acme issue list --help --json` resolved the wrong help path. An =-form flag carries its value inline and never consumes a following token, so skip the drop-next logic when the token contains =. Added a test covering --org=acme / --limit=5 before the command path.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0d4f5d6. Configure here.
| } | ||
| if (!token.startsWith("-")) { | ||
| scan.commandPath.push(token); | ||
| return 1; |
There was a problem hiding this comment.
Positionals leak into help path
Medium Severity
rewriteHelpJsonRequest treats every non-flag token as a command path segment, so positionals like an issue id are forwarded into help --json …. introspectCommand then rejects the extra segment and returns a not-found JSON error instead of help for the leaf command. Stricli’s bare --help ignores those positionals, so adding --json regresses those invocations from usable text help to an error.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 0d4f5d6. Configure here.
There was a problem hiding this comment.
Leaving this as-is by design. The rewrite is scoped to command paths (sentry [<group> <command>] --help --json) — the forms agents actually use to discover a command's shape. When a positional like an issue id is present, forwarding it produces a structured JSON Command not found error (exit 60), which is the documented, machine-readable behavior for the JSON/agent use case this feature targets; it doesn't crash or hang. Making <command> <positional> --help --json fall back to the leaf command's help would require running route-tree resolution inside the argv preprocessor (argv-hoist.ts is currently a dependency-light string transform on the hot preprocessArgv path). Given this PR is already risk: high, I'd rather not couple the preprocessor to the command graph here — happy to do it as a follow-up if maintainers want the fallback.
| for (let i = 0; i < argv.length; ) { | ||
| // Tokens after -- are positional/pass-through — a --help there is not ours. | ||
| if (argv[i] === "--") { | ||
| return null; |
There was a problem hiding this comment.
Escape aborts valid help rewrite
Low Severity
On encountering --, rewriteHelpJsonRequest returns null immediately even when --help and --json were already seen earlier in argv. The documented rule is only that those flags must appear before the escape; a later -- should stop scanning, not discard a rewrite that already qualified.
Reviewed by Cursor Bugbot for commit 0d4f5d6. Configure here.
There was a problem hiding this comment.
This is intentional. Returning null on any pre--- scan hitting -- keeps the rule simple and conservative: a -- escape means everything after it is opaque pass-through, and I'd rather defer to Stricli's normal handling than commit to a JSON rewrite when an escape is in play (documented in the PR as -- tool --help --json not being rewritten). In practice --help --json ... -- combined with a trailing escape is not a form agents use for help discovery, so the extra complexity to salvage it isn't worth the risk on this path. Leaving as-is.
Stricli treats `-h` as a built-in alias for `--help`, but scanHelpJsonToken only matched the literal `--help` token, so `sentry -h --json` fell through to Stricli's text usage instead of producing JSON. Recognize `-h` alongside `--help` and add tests.
|
Jared, would this be easier if we made a patch to Stricli itself (like we did for the |
|
The `-H` patch works because it just deletes an alias registration from Stricli's scanner — a self-contained edit with no dependency on our code. `--help --json` is a different shape: the structured output comes from our own `introspectAllCommands`/`introspectCommand` in `src/lib/help.ts`, which a vendored There's also a maintenance angle: the patch is against The preprocessor keeps it in normal TS next to the existing That said — if you'd rather push the |
I'd rather have this upstream but in a simpler way: just support "top-level args" concept so we don't need any pre-processing. This applies to everything in that preprocessor file like |
|
Agreed, that's the better shape — a "top-level flags" allow-list in the scanner ( One nuance to sequence around: How do you want to sequence it? Options I see:
I lean toward (1) since the refactor touches every global flag and wants its own review, but happy to do (2) if you'd rather not ship the preprocessor at all. Your call. |
|
Go with 1. That said I'd argue we should allow modifying Feel free to submit PRs to upstream both for this and |
|
Sounds good. Filed #1339 for the top-level-flags refactor and captured your terminal-flag note there — This PR is unchanged and ready as the interim fix — leaving the merge call to you given the |
…ge, and auth examples (#1350) ## Documentation Audit Report (2026-08-03) Weekly automated audit that reads the full codebase and cross-references it against documentation to find gaps. --- ### Changes Made | File | Fix | |------|-----| | `getting-started.mdx` | Corrected Node.js requirement from "22.15+" to "20+" (matches `engines.node >= 20.0` in `package.json`; 22.15+ gives native `node:sqlite`, 20–22.14 uses WASM fallback) | | `getting-started.mdx` | Updated version pin example from `0.19.0` to `0.40.0` | | `install` script | Updated version examples from `0.19.0` to `0.40.0` | | `env-registry.ts` | Updated `SENTRY_VERSION` example from `0.19.0` to `0.40.0` | | `.craft.yml` | Fixed Homebrew formula license `FSL-1.1-MIT` → `FSL-1.1-Apache-2.0` (matches `package.json` and `LICENSE.md`) | | `agentic-usage.md` | Expanded supported agent list from 2 (Claude Code, Cursor) to all 11 agents recognized by `detect-agent.ts`: Claude Code, Cursor, Windsurf, GitHub Copilot, Gemini CLI, OpenAI Codex, Goose, Amp, Augment, OpenCode | | `auth.md` fragment | Added examples for `--read-only`, `--scope`, and `--url` flags on `sentry auth login` | --- ### Full Gap Report #### A. Undocumented or Missing Commands/Subcommands **No gaps.** All 107 commands + 17 hidden aliases have auto-generated doc pages via `generate-command-docs.ts`. Hand-written fragments exist for every command group under `apps/cli-docs/src/fragments/commands/`. #### B. Undocumented Flags **Fixed in this PR:** - `sentry auth login --read-only` — new flag for requesting read-only OAuth scopes, useful for AI agents and CI. Not previously documented with examples. - `sentry auth login --scope` — new flag for requesting specific OAuth scopes. Not previously documented with examples. - `sentry auth login --url` — existed but examples used `SENTRY_URL` env var syntax instead of the recommended `--url` flag. **Remaining (low priority — flags are visible in auto-generated Options tables):** - `sentry auth login --force` — re-authenticate without prompting. No example in fragment. - `sentry auth login --timeout` — OAuth flow timeout. No example in fragment (default 900s is rarely changed). - `sentry help --json` — new feature from `feat(help): support JSON output for --help flags (#1337)`. Mentioned in `fullDescription` but no dedicated doc section yet. #### C. Missing Usage Examples **All documented subcommands have bash examples.** Minor gaps: - `sentry cli fix` has an example but no flags documented in the fragment. - `sentry release propose-version` only shown embedded in `$(...)` subshell, not standalone. #### D. Stale Descriptions **No meaningful drift detected.** The `brief` strings in code match the auto-generated doc descriptions. The doc generation pipeline keeps these in sync automatically. #### E. Missing Route Mappings in Skill Generator **N/A.** The `ROUTE_TO_REFERENCE` map was removed in favor of automatic 1:1 mapping via `groupRoutesByReference()` in `script/generate-skill.ts`. Every visible route automatically gets its own reference file. #### F. Installation / Distribution Gaps **Fixed in this PR:** - `getting-started.mdx` claimed npm packages require "Node.js 22.15+" — actual `engines.node` is `>=20.0`. Fixed to say "Node.js 20+" with a note about 22.15+ for native sqlite. - Version pin examples used `0.19.0` (current release is `0.40.0`). Updated in `getting-started.mdx`, `install` script, and `env-registry.ts`. **Remaining (low priority):** - Install script flags `--no-modify-path`, `--no-completions` are documented in `install --help` but not in `getting-started.mdx`. These are advanced/niche. - `SENTRY_INSTALL_DIR` env var is documented in `install --help` and `env-registry.ts` (generated into `configuration.md`) but not in `getting-started.mdx`. - Two install URLs coexist: `cli.sentry.dev/install` and `sentry.io/get-cli/` (the latter redirects — cosmetic only). #### G. Undocumented Environment Variables **No gaps for user-facing variables.** The `env-registry.ts` contains all 28 user-facing env vars and they are generated into `configuration.md`. Variables not in the registry are intentionally excluded: - `SENTRY_PIPELINE` — internal CI variable used by `build/upload.ts` - `SENTRY_SPOTLIGHT` — injected by `local run`, not user-set - `SENTRY_MONITOR_SLUG` — injected into child processes by `monitor run` - `SENTRY_CLI_NO_EXIT_TRAP` — bash hook internal - `SENTRY_STRICT_SILENT_CATCH` — dev-only CI enforcement flag #### H. Auth / Self-Hosted Gaps **Fixed in this PR:** - Auth fragment now shows `--url` flag syntax (recommended) alongside env var syntax for self-hosted. - Added `--read-only` and `--scope` examples to auth fragment. **Remaining:** - OAuth `--scope` validation details (which scopes are valid) not documented in user docs — available via `--help`. - Host-scoped tokens behavior (tokens are bound to the instance they were created on) is an internal safety mechanism, not documented for end users. #### I. Plugin/Skills Gaps **Fixed in this PR:** - `agentic-usage.md` now lists all 11 detected agents instead of just Claude Code + Cursor. **Remaining:** - `agent-skills.ts` only installs to `.claude` and `.agents` directories — Windsurf, Copilot, Gemini, etc. are detected for telemetry but don't have skill installation paths yet. This is a feature gap, not a doc gap. - `plugins/README.md` mentions Claude Code marketplace commands (`claude plugin marketplace add`) and Cursor — could mention the broader `~/.agents` ecosystem. - `plugin.json` version (`0.41.0`) is ahead of CLI version (`0.40.0-dev.0`) — likely intentional from the post-release bump. #### J. README / DEVELOPMENT.md Drift **Fixed in this PR:** - `.craft.yml` Homebrew formula had `FSL-1.1-MIT`; all other files have `FSL-1.1-Apache-2.0`. **No drift detected in:** - Root `README.md` accurately describes the monorepo structure and delegates to `packages/cli/README.md`. - `packages/cli/README.md` correctly states Node.js 20+ for npm, 22.15+ for dev, pnpm as package manager. - `DEVELOPMENT.md` (at `packages/cli/DEVELOPMENT.md`) has generated sections for prerequisites, env vars, and OAuth scopes — these stay in sync via `generate-docs-sections.ts`. - `contributing.md` has generated sections for prerequisites and project structure. --- ### Top 5 Most Impactful Fixes (Prioritized) 1. **Node.js version claim** — `getting-started.mdx` said 22.15+ but `engines.node >= 20.0`. Users on Node 20/21 would skip the npm install path unnecessarily. **Fixed.** 2. **License mismatch** — `.craft.yml` Homebrew formula had wrong license identifier. Every Homebrew install showed `FSL-1.1-MIT` instead of `FSL-1.1-Apache-2.0`. **Fixed.** 3. **Stale version pins** — Example version `0.19.0` is 21 releases old (`0.40.0` is current). Users copying examples would install very outdated versions. **Fixed.** 4. **Agent coverage** — `agentic-usage.md` only mentioned 2 of 11 supported agents. Users of Windsurf, Copilot, Gemini, Codex, Goose, Amp, etc. wouldn't know skill support exists. **Fixed.** 5. **Auth flag examples** — `--read-only` and `--scope` flags had no usage examples, making them hard to discover for users wanting scoped OAuth tokens. **Fixed.** <div><a href="https://cursor.com/agents/bc-542ca324-a467-4e43-9875-7c4ba3393d09?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/automations/8b0c0f35-da5e-409d-984c-5e39518ffb8a"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/view-automation-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/view-automation-light.png"><img alt="View Automation" width="141" height="28" src="https://cursor.com/assets/images/view-automation-dark.png"></picture></a> </div> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Miguel Betegón <miguelbetegongarcia@gmail.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ons, dataset list (#1400) ## Documentation Audit Report (2026-08-10) Weekly automated audit of the Sentry CLI repository comparing documentation against implementation. --- ## Changes in This PR ### 1. AGENTS.md — Zod → Valibot migration drift (HIGH IMPACT) PR #1389 (merged 2026-08-07) removed all remaining Zod usage and migrated to Valibot. However, AGENTS.md was not updated, leaving AI agents with incorrect guidance that would produce non-compiling code. **Fixed:** - Renamed "Zod Schemas for Validation" → "Valibot Schemas for Validation" - Rewrote code examples to use the Valibot API (`object`, `string`, `optional`, `InferOutput`, `safeParse`) - Fixed import example from `import { z } from "zod"` → `import { object, string, optional } from "valibot"` - Fixed stale import path `../../lib/config.js` → `../../lib/db/auth.js` for `getAuthToken` - Updated architecture description ("TypeScript types and Zod schemas" → "Valibot schemas") - Updated "No Runtime Dependencies" rule ("redundant Zod schemas" → "redundant Valibot schemas") ### 2. Version pin examples — 0.40.0 → 0.42.2 The latest release is 0.42.2 (three minor versions ahead of the documented pin). **Fixed in:** - `apps/cli-docs/src/content/docs/getting-started.mdx` — `SENTRY_VERSION=0.40.0` → `0.42.2` - `packages/cli/install` — help text and examples updated from `0.40.0` → `0.42.2` ### 3. agent-guidance.md — Stale dashboard dataset list The documented dataset list used internal API names (`tracemetrics`, `error-events`) instead of user-facing aliases. Also missing `transactions` alias. **Fixed:** Updated to show user-facing names: `spans` (default), `errors`, `transactions`, `metrics`, `issue`, `logs`. --- ## Full Gap Report ### A. Undocumented or missing commands/subcommands **No gaps.** All commands in `src/commands/` have corresponding doc fragments in `apps/cli-docs/src/fragments/commands/`. Command docs are auto-generated from code metadata + fragments, so coverage is inherently complete. The new `platform` command (#1366) already has its fragment. ### B. Undocumented flags **No gaps.** Non-hidden flags are auto-generated into Options tables by the doc generator (`script/generate-command-docs.ts`). This was verified by checking the generated output for recent additions. ### C. Missing usage examples All command groups have bash examples in their fragments. Lower-priority gaps: - `sentry help --json` (new in #1337) has no dedicated example in the help fragment (only available via `--help`) - `sentry cli fix` fragment exists but is minimal ### D. Stale descriptions **No gaps found.** The `brief` strings in code match the generated doc descriptions. ### E. Missing route mappings in skill generator **Not applicable.** `ROUTE_TO_REFERENCE` was removed in favor of automatic 1:1 mapping via `groupRoutesByReference()`. All routes are automatically covered. ### F. Installation / distribution gaps | Gap | Source | Doc | |-----|--------|-----| | Install script `--no-modify-path` / `--no-completions` flags | `packages/cli/install` | Not in `getting-started.mdx` (available via `--help`) | | Two install URLs coexist: `cli.sentry.dev/install` vs `sentry.io/get-cli/` | redirect config | Not documented (redirect is transparent) | | **Version pin examples stale (0.40.0)** | install script, getting-started.mdx | **Fixed in this PR** | ### G. Undocumented environment variables **No gaps.** `configuration.md` is generated from `src/lib/env-registry.ts`, which is the single source of truth. Internal-only variables (`SENTRY_PIPELINE`, `SENTRY_MONITOR_SLUG`, etc.) are intentionally excluded. ### H. Auth / self-hosted gaps **No new gaps.** Self-hosted docs (26.1.0+ OAuth requirement, `SENTRY_CLIENT_ID`, trust anchors) are accurate. The new `sentry auth` smart default (login when logged out, status when logged in, PR #1380) is already documented in the auth fragment. ### I. Plugin/skills gaps Low-priority items (unchanged from prior audit): - `agent-skills.ts` only installs to `.claude` and `.agents` directories — other detected agents (Windsurf, Copilot, etc.) are detected for telemetry only, not skill installation - This is technically accurate in `agentic-usage.md` ("Skills are also refreshed... skill files are embedded in the binary") but could be clearer about which agents get auto-installed skills vs. which are only detected ### J. README / DEVELOPMENT.md drift | Gap | Source | Doc | |-----|--------|-----| | **AGENTS.md references Zod throughout** | `src/types/` uses Valibot after #1389 | **Fixed in this PR** | | AGENTS.md import example uses stale path `lib/config.js` | `getAuthToken` is in `lib/db/auth.js` | **Fixed in this PR** | --- ## Top 5 Most Impactful Fixes (Prioritized) 1. **✅ AGENTS.md Zod → Valibot** — AI agents will write non-compiling code using `import { z } from "zod"` because AGENTS.md instructs them to. This causes immediate build failures for any AI-assisted contribution. 2. **✅ Version pin examples** — Users following the install docs will pin to a version 3 releases behind, potentially missing security fixes and new features. 3. **✅ Dashboard dataset aliases** — Agents using the documented `tracemetrics` or `error-events` names work, but the user-facing aliases (`metrics`, `errors`) are more discoverable and match `--help` output. 4. **Low priority: `--no-modify-path` / `--no-completions` installer flags** — Power users in CI/Docker may want these, but they're available via `--help` on the script itself. 5. **Low priority: Skill install target clarification** — Only `.claude` and `.agents` get auto-installed skills; other agents are detected for telemetry only. This is technically correct in the docs but could be made more explicit. <div><a href="https://cursor.com/agents/bc-e1379371-9ad9-4d53-8665-bb60c5e961e0?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/automations/8b0c0f35-da5e-409d-984c-5e39518ffb8a"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/view-automation-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/view-automation-light.png"><img alt="View Automation" width="141" height="28" src="https://cursor.com/assets/images/view-automation-dark.png"></picture></a> </div> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Miguel Betegón <miguelbetegongarcia@gmail.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>


sentry help --jsonreturned structured help, but the--helpforms agents reach for first (sentry --help --json,sentry issue --help --json,sentry issue list --help --json) fell through to Stricli's built-in text usage and ignored--json.This rewrites a flag-based
--help --jsonrequest to the dedicatedhelpcommand during argv preprocessing (preprocessArgv), so all--helpforms now emit the same JSON assentry help --json. The rewrite only fires when both--helpand--jsonare present before any--escape — a bare--helpkeeps Stricli's text usage unchanged — and--fieldsis carried through for field selection.Testing
pnpm exec vitest run test/lib/argv-hoist.test.ts test/lib/argv-hoist.property.test.ts test/commands/help.test.ts(89 passed)tsc --noEmitclean,biome checkclean on changed filessentry --help --json,sentry issue --help --json, andsentry issue list --help --jsonproduce valid JSON identical to thehelpcommand; bare--helpstill shows text usage; invalid command returns a JSON error (exit 60);-- tool --help --jsonafter an escape is not rewrittenCloses #1265