docs: add evidence verification pattern for spec-driven development#3442
docs: add evidence verification pattern for spec-driven development#3442jiezeng2004-design wants to merge 2 commits into
Conversation
Add docs/concepts/evidence-verification.md describing a tool-agnostic pattern for producing bounded, traceable, secret-free evidence packs that trace an implementation back to its spec and tasks. The pattern complements the in-session /speckit.analyze and /speckit.converge commands by emitting durable, machine-readable verification artifacts for cross-session review, audit, and supply-chain intake. PatchWarden Evidence Pack v2 is included as one reference implementation. Wire the new page into docs/toc.yml under Concepts, next to Spec Persistence Models.
There was a problem hiding this comment.
Pull request overview
Adds a new Concepts documentation page describing an “evidence verification pattern” intended to complement Spec Kit’s in-session verification commands by producing a durable, bounded, machine-readable evidence pack for cross-session review/audit.
Changes:
- Add new concept page:
docs/concepts/evidence-verification.md. - Link the new page from
docs/toc.ymlunder the Concepts section.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| docs/toc.yml | Adds the new concept page to the documentation table of contents. |
| docs/concepts/evidence-verification.md | Introduces and explains the evidence verification pattern and provides a concrete reference example. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | Concern | Example file | Contents | Why it matters | | ||
| |---|---|---|---| | ||
| | Verification status | `verify.json` | Per-task pass/fail, audit verdict, command counts | Answers "did the implementation pass its checks?" without re-running them | | ||
| | Risk findings | `risk.json` | Aggregated findings with severity (high/medium/low) and category | Surfaces what `/speckit.converge` would classify as `missing`, `partial`, `contradicts`, or `unrequested`, plus non-convergence risks | |
| The `severity` mapping mirrors how `/speckit.converge` grades findings: a | ||
| failed check is high, a warning check is medium, and a lineage-level warning | ||
| is low. A reviewer can read `risk.json` alongside a converge report and expect | ||
| the two to agree on severity ordering, even though they come from different | ||
| tools. |
|
Thanks for this — it's genuinely well-written and the underlying idea (durable, portable evidence packs that trace an implementation back to its spec/tasks) is a real gap worth filling. The technical framing is accurate too: That said, I can't take this as a first-party docs page. Our concept docs cover patterns the toolkit itself ships, and roughly half of this page documents a specific external tool (PatchWarden) — including its file layout and schema. Putting one tool's implementation into core docs implies an endorsement and creates a maintenance dependency on a schema we don't control, so it's not something we can host here. The good news is we have a purpose-built home for exactly this: the community extension catalog. You've already spotted the right integration points ( To go that route:
I'm going to close this PR, but I'd be happy to see the extension submission — the pattern itself is a solid contribution in that form. |
|
Thanks for the guidance. I followed the extension-catalog path rather than reopening this docs PR:
The |
Description
Adds a new concept page,
docs/concepts/evidence-verification.md, naming a complementary pattern to Spec Kit's existing in-session verification commands.Spec Kit's
/speckit.analyze(before implementation) and/speckit.converge(after implementation) verify within a single agent session but do not emit a durable, machine-readable record of what was verified, what risks remain, or what changed. This PR documents the evidence verification pattern: producing a bounded, traceable, secret-free, provenance-tagged pack of files that traces an implementation back to its spec and tasks. The pack is positioned as a derivative artifact for cross-session review, audit, and supply-chain intake — not a replacement for the spec, plan, or tasks.The page is tool-agnostic: it describes the concerns an evidence pack should cover (verification status, risk findings, change scope, lineage, provenance, redactions) without mandating a specific tool. PatchWarden's Evidence Pack v2 is included as one reference implementation, with a file-to-concern mapping table and a sample
risk.jsonentry. The severity mapping is deliberately aligned with/speckit.converge's gap classification (missing/partial/contradicts/unrequested) so the two can be read together.It also notes how such a supervisor can be wired into Spec Kit via the existing extension hook system (
before_implement,after_converge, etc. in.specify/extensions.yml) without changing core commands.The page is added to
docs/toc.ymlunder Concepts, next to Spec Persistence Models, which it complements.This is a docs-only change. No code, templates, scripts, or tests are modified.
Testing
markdownlint-cli2passes on the new file (0 errors), using the repo's.markdownlint-cli2.jsoncconfig.docs/concepts/.docs/toc.ymldiff is a clean 2-line addition with no line-ending changes.uv run specify --help— N/A (no CLI/template/script changes).uv sync && uv run pytest— N/A (no code changes).This is a docs-only addition; no slash command or script behavior is affected, so the manual command-testing flow in CONTRIBUTING.md does not apply.
AI Disclosure
This PR was authored by an AI coding agent (GLM-5.2 via Trae IDE), acting on a delegated task. The agent read CONTRIBUTING.md, README.md, the existing
docs/concepts/pages, and the/speckit.analyzeand/speckit.convergecommand templates to align the new page with Spec Kit's terminology and verification model. The PatchWarden Evidence Pack v2 schema was referenced from its own repository. The doc content, structure, and wording were generated by the agent and have not yet received human review — one reason this is opened as a draft. A human reviewer should confirm the framing is appropriate and that naming PatchWarden as a reference is acceptable before merging.Opened as a draft per CONTRIBUTING.md guidance for changes that have not had prior maintainer discussion.