Skip to content

feat(codex): validate bundles against pinned schemas with honest host-devtools evidence (#133) - #326

Merged
ScriptedAlchemy merged 2 commits into
mainfrom
wave/133-codex-validator
Sep 2, 2026
Merged

feat(codex): validate bundles against pinned schemas with honest host-devtools evidence (#133)#326
ScriptedAlchemy merged 2 commits into
mainfrom
wave/133-codex-validator

Conversation

@ScriptedAlchemy

Copy link
Copy Markdown
Owner

Summary

Codex ecosystem validator for #133, mirroring the Claude devtools precedent (#193) under the issue's honesty rules:

  • validateCodexPlugin (new host-contracts/codex-plugin-validation.ts): bounded codex --version probe (5 s / 1 MiB; ENOENT is an honest info skip), local validation of .codex-plugin/plugin.json, hooks/hooks.json, .mcp.json, and .agents/plugins/marketplace.json against the VENDORED pinned Codex schemas (ajv + ajv-formats), each with malformed negative fixtures so a permissive schema cannot pass vacuously.
  • No invented host verbs: dated capability evidence (2026-09-02, codex-cli 0.147.0) records that codex plugin exposes install/marketplace management only — no validate command — so the report always carries an info diagnostic naming the vendored pin actually used.
  • App-server schema generation as a drift signal only: codex app-server generate-json-schema --out <dir> runs bounded; observed 0.147.0 emits 285 app-server protocol schema files and none of the pinned hook command schemas, so that case is an evidence-backed unassessable info (strict-independent). Genuine drift (hash mismatch or partial subset) stays warning, error under --strict. Live/pinned hashes recorded in capabilities/codex-0.147.0.json.
  • Diagnostics AB6030AB6033 registered in docs/diagnostics.md; exported via api.ts (wired into validate({ hostValidation }) for the codex target) and test/index.ts; frozen reports; injectable command-runner proof seam.

Changeset: minor, agent-bundle.

Test plan (all local, on the rebase over 0e3c444)

  • pnpm build
  • pnpm typecheck
  • pnpm lint — 0 errors, 0 warnings (1012 files)
  • pnpm test:unit — 187 files, 2583 passed, 0 failed, 5 skipped
  • Focused suite: 9 cases incl. per-schema malformed fixtures, ENOENT skip, missing-verb honesty, app-server-only unassessable info under strict, drift warning, timeout/output-limit terminations

Tracking

@changeset-bot

changeset-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 070a326

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
agent-bundle Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@ScriptedAlchemy
ScriptedAlchemy merged commit 6370e7d into main Sep 2, 2026
4 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-02T22:58:30.884634Z 070a326 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

const pinnedDirectory = new URL('../adapters/schemas/codex/generated/', import.meta.url);

P2 Badge Package the schemas used by the drift comparison

When this API runs from the published package and Codex emits any of these comparable hook-schema files, this URL is resolved relative to the bundled dist entry, but the generated schema directory is neither imported into the bundle nor copied by packages/agent-bundle/rslib.config.ts (and the package publishes only dist). Consequently each pinned readFile raises ENOENT, which is misclassified as a missing live schema and produces a false drift warning/error even when the live files match exactly. Embed the pinned bytes/hashes or explicitly copy these files into a package-relative runtime location.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant