Skip to content

[Feature]: Document a test harness for scripting gate verdicts non-interactively #2594

Description

@doquanghuy

Problem Statement

Spec Kit's gate step prompts operators interactively for a
verdict. There's no documented way to script those verdicts in a
test runner, so:

  • Spec Kit's own CI cannot exercise gate behaviour end-to-end —
    changes to gate prompt format, output channel, or switch routing
    on verdict can regress without coverage.
  • Workflows authored on Spec Kit that depend on gate semantics
    cannot be CI-tested without hand-rolling a pexpect driver
    against an undocumented prompt format.

Proposed Solution

A documented way to script gate verdicts during
specify workflow run. Two possible shapes:

A. --gate-script flag

specify workflow run my-pipeline --gate-script verdicts.yaml
schema: speckit.gate-script/v1
verdicts:
  - {gate_id: "review-overview", iteration: 0, verdict: "improve"}
  - {gate_id: "review-overview", iteration: 1, verdict: "approve"}

B. Document the canonical gate-prompt regex (e.g. anchored
markers like SPECKIT_GATE_PROMPT_BEGIN: <id>
SPECKIT_GATE_PROMPT_END) so contributors can write pexpect
drivers against a stable contract.

A is more CI-friendly; B is more flexible for advanced cases.

Component

Specify CLI (initialization, commands)

AI Agent (if applicable)

All agents

Acceptance Criteria

  • Documented contract for scripting gate verdicts (A or B).
  • At least one example test in Spec Kit's suite exercising an
    improveapprove cycle.
  • Default behaviour preserved: gates still prompt
    interactively when no script is provided.

Additional Context

If gate behaviour is intentionally manual-only, please say so and
I'll close.

AI disclosure: drafted with Claude Opus, human-reviewed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions