Skip to content

Native Claude proofs run claude plugin validate on the bundle directory, which Claude treats as a marketplace (hooks/skills/agents never checked) #475

Description

@ScriptedAlchemy

Problem

The binary-gated Claude proofs validate the emitted plugin directory:

  • packages/agent-bundle/tests/host-adapters.native.test.ts:52-53runClaudeValidation(cwd, target)claude plugin validate --strict <target>, called with (root, root) at lines 617, 642, 935, 1042, 1061, 1081, and with pluginRoot at line 1000.
  • packages/agent-bundle/tests/support/packed-native-smoke.ts:303claude plugin validate <artifact>/claude --strict.

writeClaudeArtifact emits .claude-plugin/marketplace.json beside plugin.json, so Claude Code resolves each of those runs as a marketplace run, which never opens hooks/, skills/, agents/, or commands/:

From a marketplace directory, Claude Code doesn't open the plugins' skill, agent, command, or hook files.
plugin-marketplaces-0.mdTroubleshooting › Marketplace validation errors (L1338–1348)

Several tests therefore record marketplace-mode behaviour as if it were the plugin validator's:

  • records that strict validation does not catch a path-escaping plugin symlink (line 628–650) asserts exit 0 and that the output does not mention outside-skill. A plugin run on 2.1.259 warns and exits 1 under --strict:
    Validating skill: /tmp/ab-val2/skills
      ❯ directory: 1 entry here is a symlink and was not read — components are read without following symlinks. A session loading this plugin does follow them, so validate the real paths separately.
    
    matching plugin-marketplaces-0.mdValidate a plugin or a directory without a manifest › Check files behind symlinks (L1392–1404).
  • The monitors/monitors.json, theme, and settings.json "does not inspect" tests (lines 928–940, 1036–1085) are true in both modes per the docs, but their evidence is currently marketplace-mode output.
  • The packed smoke (packed-native-smoke.ts:303) would pass a bundle whose hooks/hooks.json is invalid JSON; a plugin run reports Invalid JSON syntax as an error (Read the validation results, L1406–1416).

Proposed fix

Point each proof at join(root, '.claude-plugin', 'plugin.json') (what #474 does for validate --artifact and the native eval gate), and add a second marketplace.json run where the test is actually about the marketplace. Re-record the symlink test's expectation to the plugin-run warning. Both forms work on the CI pin 2.1.250 (verified locally with ~/.local/share/claude/versions/2.1.250).

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    host-integrationAgent host capabilities, projection, discovery, and lifecycletestingFramework test harnesses and integration evidence

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions