Skip to content

feat(catalog+fetcher): strict schema validation + bundle refresh + rename sync (closes #183)#184

Merged
hyperpolymath merged 2 commits into
mainfrom
feat/strict-cartridge-validation-183
Jun 1, 2026
Merged

feat(catalog+fetcher): strict schema validation + bundle refresh + rename sync (closes #183)#184
hyperpolymath merged 2 commits into
mainfrom
feat/strict-cartridge-validation-183

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Summary

Closes the boj-server side of the cartridge-schema-validation campaign that landed in boj-server-cartridges#21+#23+#25+#26+#27 this session. Three changes ship together:

  1. `BojRest.Catalog` validates strictly at boot — adds `ex_json_schema`, compiles `schemas/cartridge-v1.json` once, validates every manifest against it on load, and rejects (does not insert into ETS, logs an error) any cartridge that fails. Strict from day one per owner direction.
  2. `scripts/fetch-cartridges.sh` runs registry-side strict validation as a defense-in-depth pre-flight, and now copies the registry's `schemas/` directory beside the cartridges cache so the catalog's default `:schema_path` finds the mirror without configuration.
  3. In-tree `cartridges/` refreshed against the cleaned canonical — the 2026-05-26 snapshot had 43/125 manifests failing current strict validation. Refresh script (`scripts/refresh-bundled-cartridges.sh`) preserves the 125-cartridge curation per the owner's "curate-keep at 125" direction; the three rename targets from boj-server-cartridges#27 propagate (`boj-health` → `boj-health-mcp`, `origenemcp` → `origene-mcp`, `opendatamcp` → `opendata-mcp`). Post-refresh: 125/125 strict-valid.

Tests

`elixir/test/catalog_test.exs` updated:

  • `boj-health` references → `boj-health-mcp`.
  • "auth method known value" test tightened to the canonical schema enum `[none, api-key, oauth2, vault]` only.
  • New: every loaded cartridge carries every schema-required top-level field.
  • New: `schemas/cartridge-v1.json` SHA-256 equals the `Pin SHA-256:` line in `schemas/PINNED-SHA` — guards silent mirror drift.

Compatibility

Any external consumer that called `BojRest.Catalog.get("boj-health")` / `get("origenemcp")` / `get("opendatamcp")` now gets `:not_found`. The old names violate the canonical schema's role-suffix pattern; alias support was deliberately not added because re-introducing the violating names defeats the strict gate.

The fetcher remains backward-compatible: if the registry is an older snapshot without `tools/validate-cartridges/`, or if Deno isn't installed, the strict pre-flight is skipped with a clear log line and the catalog still validates at load.

Test plan

  • Refresh script run locally against current canonical; 125/125 strict-valid.
  • All commits GPG-signed.
  • CI green (Elixir 1.18 / OTP 25; deno strict in fetcher path).

Closes #183.

🤖 Generated with Claude Code

…name sync (closes #183)

Closes the boj-server side of the cartridge-schema-validation campaign
that landed in boj-server-cartridges#21+#23+#25+#26+#27 this session.

## Strict validation in BojRest.Catalog

`elixir/lib/boj_rest/catalog.ex` now:

- Loads `schemas/cartridge-v1.json` at boot (defaults to the bundled
  mirror at `<root>/../schemas/cartridge-v1.json`; overridable via the
  `:schema_path` option).
- Validates each `cartridge.json` against the schema during load.
- **Rejects** non-conforming manifests with an error log entry (not a
  warning) — they do not enter the ETS table. Strict from day one per
  owner direction.
- Logs final counts as `loaded=N rejected=M` so operators can compare
  the catalog against the on-disk surface.

Companion test suite changes (`elixir/test/catalog_test.exs`):

- `boj-health` references updated to the canonical name
  `boj-health-mcp` (rename landed in
  boj-server-cartridges#27).
- "auth method known value" test tightened to the canonical schema enum
  `[none, api-key, oauth2, vault]` only — the wide-list test no longer
  matches reality after the strict gate flip.
- New test: every loaded cartridge carries every schema-required
  top-level field (a sanity check that strict validation actually
  enforces what we think it does).
- New test: `schemas/cartridge-v1.json` SHA-256 equals the
  `Pin SHA-256:` line in `schemas/PINNED-SHA` — guards against silent
  mirror drift.

Adds `ex_json_schema ~> 0.10` to deps.

## Defense-in-depth validation in fetch-cartridges.sh

The fetcher now runs `deno task strict` in the cloned registry's
`tools/validate-cartridges/` before adopting any of its manifests, and
aborts the fetch if the registry fails strict validation. The check is
**skipped** (with a note) when either the validator or Deno is absent,
since the catalog validates at load regardless.

The fetcher also now copies the registry's `schemas/` directory to a
sibling of the cartridges cache, so the resulting layout
(`<parent>/cartridges/` + `<parent>/schemas/`) lines up exactly with
what the catalog's default `:schema_path` looks for. Caches are now
self-contained — the catalog can validate without reaching out for the
schema.

## Bundle refresh + 3 renames

The legacy bundled `cartridges/` (snapshot 2026-05-26) had 43 of 125
manifests failing current strict validation. Refreshed in-place against
the cleaned canonical, preserving the existing 125-cartridge curation
per the owner's "curate-keep at 125" direction. The three rename targets
from boj-server-cartridges#27 are propagated:

- `boj-health` → `boj-health-mcp`
- `origenemcp` → `origene-mcp`
- `opendatamcp` → `opendata-mcp`

Post-refresh: 125/125 strict-valid. The refresh script ships at
`scripts/refresh-bundled-cartridges.sh` and stays in-tree as the
precedent for any future curation-preserving sync.

## Compatibility note

Any external consumer that called `BojRest.Catalog.get("boj-health")` /
`get("origenemcp")` / `get("opendatamcp")` now gets `:not_found`. The
cartridge names changed to satisfy the canonical schema's role-suffix
pattern; alias support was deliberately not added because the old names
violate the schema and shouldn't be re-introduced through a back door.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 219 issues detected

Severity Count
🔴 Critical 15
🟠 High 128
🟡 Medium 76

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Stale AI session file -- delete",
    "type": "stale",
    "file": "GEMINI.md",
    "action": "delete",
    "rule_module": "root_hygiene",
    "severity": "medium"
  },
  {
    "reason": "Action    if: always()\n        uses: actions/upload-artifact@ea165 needs attention",
    "type": "unpinned_action",
    "file": "e2e.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in abi-drift.yml",
    "type": "missing_timeout_minutes",
    "file": "abi-drift.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in codeql.yml",
    "type": "missing_timeout_minutes",
    "file": "codeql.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in container-publish.yml",
    "type": "missing_timeout_minutes",
    "file": "container-publish.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

🏁 path-claims bench

Commit 9e2e3c5

Numbers
path-claims bench  (node v22.22.3)

  scenario                                              iters       ms        ns/op          ops/s
  --------------------------------------------------------------------------------------------------------------
  register: 10 active claims, 3 new paths               50000 iters    187 ms      3.75 µs/op    266.8k ops/s
  register: 100 active claims, 3 new paths              20000 iters    321 ms     16.10 µs/op     62.1k ops/s
  register: 1000 active claims, 3 new paths              5000 iters    981 ms    196.35 µs/op      5.1k ops/s
  register: 100 active claims, 20 new paths              5000 iters    369 ms     73.92 µs/op     13.5k ops/s

  pathsOverlap: deep diverge at segment 4             1000000 iters    163 ms     163.9 ns/op     6.10M ops/s
  pathsOverlap: short prefix match                    1000000 iters    140 ms     140.0 ns/op     7.14M ops/s

  refresh (existing claim)                             100000 iters     11 ms     112.6 ns/op     8.88M ops/s
  list (100 active claims)                              50000 iters    299 ms      5.99 µs/op    167.0k ops/s

  (Bench numbers depend on host; use deltas across commits, not absolute values.)

Host-dependent — compare deltas across commits, not absolute values.

ex_json_schema 0.11.3 only supports drafts 4/6/7; the canonical
cartridge-v1.json declares draft 2020-12. The schema's actual
constructs (type, enum, pattern, required, properties, items, minItems)
are all draft-4-compatible, so we rewrite the in-memory $schema URL
before resolving.

The on-disk file is untouched — schemas/PINNED-SHA stays valid and the
mirror SHA-256 check in catalog_test.exs continues to pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hyperpolymath hyperpolymath merged commit 06834c4 into main Jun 1, 2026
23 of 31 checks passed
@hyperpolymath hyperpolymath deleted the feat/strict-cartridge-validation-183 branch June 1, 2026 19:07
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 221 issues detected

Severity Count
🔴 Critical 15
🟠 High 130
🟡 Medium 76

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Stale AI session file -- delete",
    "type": "stale",
    "file": "GEMINI.md",
    "action": "delete",
    "rule_module": "root_hygiene",
    "severity": "medium"
  },
  {
    "reason": "Action    if: always()\n        uses: actions/upload-artifact@ea165 needs attention",
    "type": "unpinned_action",
    "file": "e2e.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in abi-drift.yml",
    "type": "missing_timeout_minutes",
    "file": "abi-drift.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in codeql.yml",
    "type": "missing_timeout_minutes",
    "file": "codeql.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in container-publish.yml",
    "type": "missing_timeout_minutes",
    "file": "container-publish.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in dogfood-gate.yml",
    "type": "missing_timeout_minutes",
    "file": "dogfood-gate.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(fetcher+catalog): consume boj-server-cartridges schema + validator (closes consumer-side of cartridges#19)

1 participant