Skip to content

feat: add specify artifact introspection - #4305

Merged
nicolehaugen merged 116 commits into
mainfrom
nicolehaugen-contribution-ids-and-artifacts
Sep 11, 2026
Merged

feat: add specify artifact introspection#4305
nicolehaugen merged 116 commits into
mainfrom
nicolehaugen-contribution-ids-and-artifacts

Conversation

@nicolehaugen

@nicolehaugen nicolehaugen commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Introduces the read-only specify artifact command group as an incremental foundation for artifact discovery and composition inspection.

Included in this PR

  • specify artifact list --json for commands, templates, and scripts visible to a Spec Kit project.
  • specify artifact info <name> --json for the ordered composition stack behind one artifact.
  • Source-agnostic public artifact IDs using {kind}:{name}.
  • Stack metadata including active/hidden state, strategy, source provenance, manifest path, source path, and artifact-local lookupId values for project, preset, and extension layers.
  • Runtime-aware script discovery using the project's existing sh, ps, or py selection.
  • Reuse of existing preset resolution, manifest parsing, registrar output paths, runtime selection, and core asset lookup behavior.
  • No production changes to existing preset, extension, hook, resolver, registrar, asset, or integration behavior.

lookupId in this PR is artifact-stack provenance. It is derived inside the artifact package from the existing resolver layer and manifest data. It is not yet a public foreign key into preset or extension manifest APIs. PresetResolver.collect_all_layers() intentionally retains its established return shape in this PR, and parity tests assert that artifact projection does not mutate that shared resolver contract.

Deferred follow-up work

#4210 will introduce the shared contribution-ID contract:

  • Promote named and hook contribution-ID derivation into shared functionality.
  • Expose computed contribution IDs through public preset and extension manifest/info representations.
  • Extend provenance-backed non-built-in PresetResolver.collect_all_layers() rows with stable lookupId values; built-in rows will retain absent contribution provenance.
  • Make specify artifact consume resolver-provided lookupId values instead of reconstructing them privately.
  • Require each manifest-backed resolver/artifact lookupId to equal the originating public manifest contribution id, with exact round-trip and repeatability tests.
  • Replace the current resolver-shape assertion with exact contribution-ID assertions once that shared contract exists.
  • Add deterministic, collision-free hook IDs without narrowing the currently accepted hook manifest syntax.

Why cross-reference support is needed

A consumer such as the Spec Kit Wizard can use this PR to learn that command:speckit.plan is currently supplied by a particular preset or extension layer. Today, however, it cannot take that layer's lookupId and request the exact originating contribution from a public preset or extension API. It must still infer the provider and contribution coordinates or parse manifests itself.

After #4210, the flow will be explicit: the artifact stack returns a lookupId, the corresponding preset or extension info response contains a contribution with that exact id, and the consumer joins the two values to retrieve contribution metadata without reproducing manifest matching rules. This supports source-detail views, “why this layer wins” diagnostics, cross-navigation, and composition comparisons while keeping Spec Kit authoritative for identity and resolution.

Hook artifacts themselves remain outside this PR and are planned separately in #4348.

Compatibility

This PR intentionally preserves existing name-based resolution and accepted manifest behavior. In particular, colon-containing extension hooks remain valid, valid empty extension registries do not block resolution, and shared resolver/runtime selection behavior is unchanged.

Fixes

Fixes #4212

Supersedes

Supersedes #4267

Testing

The focused artifact contract and resolver-parity suite passes with 87 tests and 1 platform-dependent skip. Coverage includes commands, templates, scripts, all provider tiers, manifest-declared and convention-only contributions, runtime selection, JSON/error contracts, lookup provenance, and resolver composition parity.


Updated on behalf of @nicolehaugen by GitHub Copilot (model: GPT-5.6 Sol, autonomous).

nicolehaugen and others added 30 commits August 24, 2026 20:09
…artifacts

Every command, template, script, and hook contribution returned by
preset and extension manifest surfaces now carries a computed opaque
identifier of the form {layer}:{sourceId}:{kind}:{name}, and every
resolved artifact-stack layer carries a matching lookupId derived from
the same recipe.

Identifiers are computed at read time from author-declared manifest
content only. No paths, timestamps, or file-content hashes contribute
to derivation, so identifiers are stable across machines, reinstalls,
and directory moves. Nothing is persisted to .specify/ or any cache.

Hooks that collide within a source on (eventName, command) get a
12-hex SHA-256 discriminator computed from the canonical JSON of the
entry's declared fields minus eventName/command. Two hook entries
with byte-identical remaining fields are rejected at manifest load
because there is no meaningful way to distinguish them.

The change is purely additive: all existing name-based resolution
behaviour is preserved, and no consumer keys off the new id or
lookupId fields.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous)
Adds a new `specify artifact` command group with two subcommands:

* `specify artifact list --json` — flat inventory of every command,
  template, and script SpecKit exposes for the current project. Each row
  carries a stable `id` (`{kind}:{name}`), an author-declared
  `name`, its `kind`, and a `description` string that is never
  omitted (empty string when the author declared none).

* `specify artifact info <name> --json` — the same row plus its full
  ordered composition `stack`: highest-priority contributor first, with
  `active` marking the winner `PresetResolver.resolve_content` would
  return and `hidden` marking rows shadowed by a higher-priority
  `replace`. Each stack entry carries a portable POSIX `manifestPath`
  (or `null` for the core baseline) and a `lookupId` from the
  contribution-id grammar so the output round-trips against
  `specify preset info` and `specify extension info`.

The two commands share one strict JSON error envelope on stderr
(`{ "error": "..." }`) with exit code 1 for the three logical errors
(unknown artifact, ambiguous artifact, not a Spec Kit project) and exit
code 2 for the "`--json` is required" usage error. stdout is always
empty on error, so the two streams stay independently parseable.

Implementation lives in a new `src/specify_cli/artifacts/` subpackage
that mirrors the existing `presets/` and `extensions/` layout — pure
logic in `__init__.py` and thin Typer wiring in `_commands.py`. The
subpackage reuses `PresetResolver.collect_all_layers` for the actual
composition math and only reshapes each layer into a `StackLayer` JSON
row, so `active` and `hidden` stay in lockstep with the resolver's
winner-selection logic.

Skills (`.github/skills/**/SKILL.md`) are intentionally excluded from
the inventory — they are integration-specific installation output, not a
shipped asset family. The command still surfaces the underlying command
that a skill was generated from.

Tests:

* `tests/test_artifact_command.py` — 32 tests: contract shape, sort
  order, empty-inventory behavior, kind-hint parsing, ambiguous-name
  error, unknown-artifact error, not-a-project error, skills exclusion,
  CLI wiring end-to-end (`--json` required, JSON envelope shape,
  stderr-only errors, empty stdout on error, UTF-8 with no BOM), and
  preset-replace hiding the core layer.

* `tests/test_artifact_command_parity.py` — 6 tests: `manifestPath`
  uses forward slashes on every OS and is never absolute, the `active`
  row corresponds to the resolver's actual winner, and the pretty-printed
  JSON has no trailing whitespace and ends in exactly one newline.

All 38 new tests pass. Full presets + extensions regression suite is
green modulo pre-existing Windows-symlink-privilege failures that
predate this branch.

Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4a40fb96-1bbe-4fb2-99d8-411170046cb0
…rt' and 'import from''

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
…rt' and 'import from''

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
…g from parent dir

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: Claude Sonnet 4.6, autonomous)

Co-authored-by: nicolehaugen <10600161+nicolehaugen@users.noreply.github.com>
Document that built-in artifact layers omit lookupId and round-trip through their source-agnostic public kind:name ID, while project overrides retain a synthetic stack identity.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Remove the artifact-specific preset corruption guard and retain Spec Kit's existing behavior of treating malformed preset registry data as an empty registry. Keep extension registry validation unchanged.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Do not establish a new artifact-specific contract test for the preset registry's pre-existing malformed-data fallback.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Leave release-note generation to the existing release workflow, which derives versioned changelog entries from commit subjects.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
@nicolehaugen

Copy link
Copy Markdown
Contributor Author

Addressed the latest review round in 842aa879..c903bb5b:

  • Reused the extension manifest ID from the original successful parse so transient rereads cannot change lookupId identity.
  • Enforced the project/underscore source-sentinel contract consistently in identifier construction and parsing.
  • Clarified that built-in artifacts use the source-agnostic kind:name ID and intentionally have no contribution lookupId.
  • Removed the PR-specific preset-registry corruption probe so artifact commands retain Spec Kit's existing malformed-registry fallback behavior; extension-registry validation remains unchanged.
  • Removed the manually maintained Unreleased changelog block because the release workflow generates versioned entries from commit subjects.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: GPT-5.6 Sol, autonomous).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

A critical resolver-parity defect and a moderate inventory-scaling issue remain unresolved.

Review tier: Balanced
Findings: 1 High severity

New issues introduced by this change (1)
Severity Finding
High severity src/​specify_cli/​artifacts/​__init__.pyactive is derived only from stack position, so this returns a successful stack even when the…
Issues resolved since last review (2)
Severity Finding
Medium severity src/​specify_cli/​_identifier.py — This parser accepts project:foo:command:x and preset:_:command:x, even though… View resolved comment
High severity src/​specify_cli/​presets/​__init__.py — Once entry is non-null, the layer was resolved from a successfully parsed manifest. If this… View resolved comment
Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/specify_cli/artifacts/init.py:855

  • The full inventory invokes collect_all_layers() once per candidate. Each invocation scans every extension and reparses each extension.yml in _extension_manifest_declared_template, so artifact list --json performs artifact-count × extension-count YAML reads. Cache validated extension manifests in the resolver (as presets already do) or build the inventory from one bulk resolution pass.

Comment thread src/specify_cli/artifacts/__init__.py Outdated
Document that active reflects Spec Kit's existing layer precedence rather than successful content composition, and limit artifact resolution failures to errors encountered while collecting the stack.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Document why artifact inventory resolves each candidate through Spec Kit's existing single-artifact path and defers unmeasured shared caching.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
@nicolehaugen

Copy link
Copy Markdown
Contributor Author

Addressed the artifact-resolution review follow-up in a5b779ca..429579c3:

  • Clarified that active represents the highest-precedence layer selected by Spec Kit's existing resolution order, not successful content composition.
  • Limited artifact resolution failed documentation to failures collecting the layer stack or reading the extension registry.
  • Documented that inventory intentionally reuses the existing single-artifact resolver path for behavioral parity; shared manifest caching is deferred unless typical extension sets show measurable cost.

No resolver behavior was changed: artifact inspection continues to follow the existing preset resolve layer-discovery semantics.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: GPT-5.6 Sol, autonomous).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Lookup-ID derivation breaks resolver parity for legacy colon-containing filenames, and artifact listing performs multiplicative manifest rescans.

Review tier: Balanced
Findings: 1 High severity

Pre-existing issues (1)
Severity Finding
High severity src/​specify_cli/​artifacts/​__init__.pyactive is derived only from stack position, so this returns a successful stack even when the… View comment
Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

src/specify_cli/artifacts/init.py:870

  • This resolves every candidate by rescanning every preset/extension stack. collect_all_layers() rebuilds the extension list and reparses each extension.yml, so an inventory with A artifacts and E extensions performs O(A×E) registry/filesystem/YAML work; stack serialization then rereads registry metadata per layer. Since artifact list is the batch API intended for external tooling, cache extension enumeration/manifests or build all stacks in one resolver pass.
    src/specify_cli/presets/init.py:5901
  • A project override whose filename contains : still resolves through resolve(), but this unconditional ID derivation now makes collect_all_layers() raise IdentifierComponentError for the same name. This breaks the documented resolver parity and existing name-based behavior on POSIX. Preserve stack collection for unrepresentable legacy names (for example, omit lookupId) or reject them consistently at every resolver entry point.

This issue also appears in the following locations of the same file:

  • line 5975
  • line 6018

src/specify_cli/presets/init.py:5977

  • Convention-only preset filenames are not manifest-regex constrained, so resolve() can return a POSIX file such as templates/foo:bar.md, while this derivation makes collect_all_layers("foo:bar") fail. Guard lookup-ID creation for filesystem-derived names or apply the same explicit rejection to resolve() so the resolver APIs do not disagree.
                        "lookupId": derive_named_id(
                            "preset", source_id_for_lookup, template_type, template_name
                        ),

src/specify_cli/presets/init.py:6020

  • The same resolver divergence affects convention-only extensions: an undeclared templates/foo:bar.md is found by resolve(), but collect_all_layers() raises here when constructing its lookup ID. Handle filesystem-derived names that cannot enter the ID grammar without changing successful name-based resolution, or reject them consistently in both paths.
                    "lookupId": derive_named_id(
                        "extension", source_id_for_lookup, template_type, template_name
                    ),

Keep filesystem-derived project, preset, and extension layers resolvable when legacy names cannot be represented by the contribution-ID grammar. Such layers omit lookupId while manifest-declared contributions remain strict.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
@nicolehaugen

Copy link
Copy Markdown
Contributor Author

Addressed the legacy resolver-parity findings in 165e9bd6.

Filesystem-derived project, preset, and extension layers now remain resolvable when a legacy POSIX filename cannot be represented by the colon-delimited contribution-ID grammar. Those compatibility layers report lookupId: null; manifest-declared contributions remain strictly validated. This preserves existing Spec Kit name-based resolution without weakening the new identifier contract.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: GPT-5.6 Sol, autonomous).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The cross-cutting resolver and new public JSON-contract changes warrant final human validation despite extensive coverage.

Review tier: Balanced
Findings: None

Issues resolved since last review (1)
Severity Finding
High severity src/​specify_cli/​artifacts/​__init__.pyactive is derived only from stack position, so this returns a successful stack even when the… View resolved comment

@mnriem

mnriem commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

The automated review is green with extensive coverage and flags this for final human validation rather than any specific defect — which is right, given it introduces a new public JSON contract (specify artifact) and touches the resolver. That's a compatibility commitment, so I want to do a proper review of the schema and the deterministic-ID grammar before merging rather than merge on the automated pass. Nothing needed from you meanwhile — thanks for the thorough iteration. I'll follow up with the review.

Keep root-level README templates aligned with the existing resolver and artifact inventory instead of introducing a filename-specific exclusion.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Keep lookup identifiers and provenance projection inside the artifact catalog while restoring existing resolver, extension, hook, asset, registrar, and integration behavior.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Separate artifact models, catalog inventory, and resolver stack projection while preserving the existing package API and command behavior.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The resolver and manifest contribution-ID surfaces promised by the PR remain unimplemented, alongside identified inventory and asset-resolution inconsistencies.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 4 Medium severity

New issues introduced by this change (4)
Severity Finding
Medium severity src/​specify_cli/​artifacts/​_identifiers.py — Expose IDs on manifest contributions View comment
Medium severity src/​specify_cli/​artifacts/​catalog.py — Use the promised shared core asset locator View comment
Medium severity src/​specify_cli/​artifacts/​catalog.py — Do not discard a valid override kind View comment
Medium severity tests/​test_artifact_command_parity.py — Populate lookupId in resolver layers View comment

Comment thread src/specify_cli/artifacts/_identifiers.py
Comment thread src/specify_cli/artifacts/catalog.py
Comment thread src/specify_cli/artifacts/catalog.py Outdated
Comment thread tests/test_artifact_command_parity.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The contribution-ID contract remains incomplete, and corrupt preset registries can silently produce an inaccurate inventory.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 5 Medium severity · 1 Low severity

New issues introduced by this change (2)
Severity Finding
Medium severity src/​specify_cli/​artifacts/​catalog.py — Reject corrupt preset registries before building inventory View comment
Low severity src/​specify_cli/​artifacts/​_commands.py — Use the repository product name in CLI help View comment
Pre-existing issues (4)
Severity Finding
Medium severity tests/​test_artifact_command_parity.py — Populate lookupId in resolver layers View comment
Medium severity src/​specify_cli/​artifacts/​catalog.py — Do not discard a valid override kind View comment
Medium severity src/​specify_cli/​artifacts/​catalog.py — Use the promised shared core asset locator View comment
Medium severity src/​specify_cli/​artifacts/​_identifiers.py — Expose IDs on manifest contributions View comment

Comment thread src/specify_cli/artifacts/catalog.py
Comment thread src/specify_cli/artifacts/_commands.py Outdated
Remove filename-based kind guessing for root project overrides and let the existing resolver validate both command and template candidates.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Script-path containment and incomplete built-in script discovery must be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 High severity · 1 Low severity

New issues introduced by this change (1)
Severity Finding
High severity src/​specify_cli/​artifacts/​catalog.py — Keep script references inside the script roots View comment
Pre-existing issues (1)
Severity Finding
Low severity src/​specify_cli/​artifacts/​_commands.py — Use the repository product name in CLI help View comment
Issues resolved since last review (5)
Severity Finding
Medium severity src/​specify_cli/​artifacts/​catalog.py — Reject corrupt preset registries before building inventory View resolved comment
Medium severity tests/​test_artifact_command_parity.py — Populate lookupId in resolver layers View resolved comment
Medium severity src/​specify_cli/​artifacts/​catalog.py — Do not discard a valid override kind View resolved comment
Medium severity src/​specify_cli/​artifacts/​catalog.py — Use the promised shared core asset locator View resolved comment
Medium severity src/​specify_cli/​artifacts/​_identifiers.py — Expose IDs on manifest contributions View resolved comment
Previously missed findings (3)

In code that hasn't changed since last review

src/specify_cli/artifacts/catalog.py:636

  • Core script discovery is driven only by scripts: entries in command templates. However, shared installation copies every file in the selected runtime directory (shared_infra.py:510-523), so shipped scripts such as create-new-feature and common are visible in initialized projects but omitted here; this contradicts the documented “any script” inventory and “built-in artifacts always appear” contract. Enumerate the selected runtime directory and union those files with the command-referenced paths, updating the exact-four assertion accordingly.
    src/specify_cli/artifacts/_commands.py:103
  • This docstring is rendered as subcommand help and uses SpecKit, unlike the repository-standard Spec Kit spelling.
    src/specify_cli/artifacts/resolution.py:418
  • The new hidden calculation for stacks whose highest layers use wrap, prepend, or append is not covered. The current tests only verify a top-level replace and reject an append-only stack, so a regression that marks contributing lower layers hidden would pass. Add a multi-layer composing-strategy test that compares these flags with resolve_content().

Comment thread src/specify_cli/artifacts/catalog.py Outdated
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation is scoped, documented, compatibility-preserving, and comprehensively tested.

Review tier: Balanced
Findings: 1 High severity

Pre-existing issues (1)
Severity Finding
High severity src/​specify_cli/​artifacts/​catalog.py — Keep script references inside the script roots View comment
Issues resolved since last review (1)
Severity Finding
Low severity src/​specify_cli/​artifacts/​_commands.py — Use the repository product name in CLI help View resolved comment

Reject anchored, traversing, and symlink-escaping script references before artifact discovery reads files outside the selected script root.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The large new public JSON surface and extensive coupling to private resolver and registrar internals warrant final human review.

Review tier: Balanced
Findings: None

Issues resolved since last review (1)
Severity Finding
High severity src/​specify_cli/​artifacts/​catalog.py — Keep script references inside the script roots View resolved comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-nice-to-have Verdict: evidence-backed fix or greenlit feature — land after review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add specify artifact command exposing composition stacks as JSON

4 participants