Refine `xmd plan` against hand-authored reference workflows

[taras]

Story

As an Executable.md maintainer, I want to compare xmd plan output with
hand-authored solutions to the same real task, so we can understand the
differences and improve the planning workflow, its documentation, and the
practices it teaches.

Example evaluation

Use #678, Generate website reference documentation from xmd CLI output, as
the first reference task.

For one trial:

1. give xmd plan a prompt from the evaluation corpus;
2. preserve the exact approved program source without evaluating it;
3. compare it with Generate website reference documentation from xmd CLI output #678's settled contract and hand-authored workflow;
4. record which outcomes, decisions, and boundaries the generated program
   preserved or missed;
5. determine why each meaningful discrepancy exists; and
6. identify the appropriate improvement before running another trial.

Generated programs remain inert source artifacts throughout this evaluation.
They never run against the repository or production website.

Purpose of the comparison

The hand-authored workflow is a reference, not an answer that xmd plan must
copy word for word. #678's settled product and architecture contract decides
whether either program is correct.

A discrepancy may reveal:

• a defect in xmd plan or the packaged planning workflow;
• a prompt that omitted necessary context;
• repository documentation that did not make a required contract discoverable;
• an important practice that the project relies on but has never defined;
• normal model variation;
• disagreement in the human evaluation; or
• an error or outdated assumption in the hand-authored reference.

Each discrepancy is evaluated before becoming a product issue. Missing or
unclear documentation becomes a candidate documentation improvement. An
important but undocumented convention becomes a candidate best-practice
definition.

The purpose is to continue refining the planning workflow, not to make one
generated program resemble one reference by any means necessary.

Minimum residual autonomy

Judge whether each part of the generated program retains only the autonomy that
faithfully representing the user's intent requires. Prefer, in order:

static value
-> deterministic operation
-> control flow
-> bounded human decision
-> bounded Agent judgment
-> autonomous Agent

Do not reward a program for inventing deterministic behavior where intent is
genuinely unresolved. Do identify known procedure that was delegated back to a
broad Agent, an unbounded Agent used where a bounded judgment was enough, or a
human question used where the prompt already supplied a stable answer.
Structural validity alone cannot establish this property.

Reference corpus

Begin with progressively more constrained prompts for #678:

• the natural request to improve component, release, and CLI documentation with
  Markdown programs;
• deriving the component catalog from xmd syntax without using
  repository-local components accidentally;
• deriving command syntax and flags from released CLI help rather than
  maintaining another inventory;
• keeping version, backlinks, CLI help, and component syntax associated with the
  same released binary;
• combining generation, verification, site build, workflow changes, and
  specification updates; and
• an ambiguous request that suggests unsafe repository-root inspection or
  direct deployment, to see whether the program notices the conflict.

Preserve every prompt verbatim. Describe what a prompt is intended to test
without giving the expected answer to an unconstrained trial.

Independent trials use separate Agent sessions. One separately identified case
may test whether revising a Plan in the same session preserves useful context.

Do not assign a verdict until #678 has settled the behavior being evaluated.
Prompt collection may begin earlier.

Executable evaluation program

Create an executable Markdown program that explains the evaluation as it runs
and:

1. selects one preserved prompt;
2. invokes xmd plan as a source-producing command;
3. saves the exact approved program in a location owned by that invocation;
4. records the xmd revision, prompt identity, source digest, model, Agent
   configuration, session choice, result, and duration;
5. performs deterministic checks where possible;
6. asks a human the remaining judgment questions; and
7. produces a lasting report without evaluating the generated program or
   changing website source, release configuration, or deployment state.

Passing xmd plan validation proves only that the program is well-formed. It
does not prove that the program understood the task, chose the right degree of
autonomy, or made the correct architecture decisions.

Evaluation questions

For each generated program, ask:

• Did it preserve every requested outcome?
• Is it understandable without the planning conversation?
• Does its structure make the order of work clear?
• Are executable components placed beside prose explaining why they run?
• Did it identify every material product or architecture decision, or present
  unresolved decisions for review?
• Does each component catalog, command reference, version, and release identity
  have one authoritative source?
• Does it use Markdown for workflow, TypeScript for typed decisions, and shell
  only to invoke programs?
• Does each step use the minimum residual autonomy faithful to the prompt?
• Did it delegate known deterministic procedure to an Agent, or leave genuinely
  unresolved intent falsely deterministic?
• Does it avoid accidentally loading repository-local components when inspecting
  released syntax?
• Does it associate the source revision, binary, version, backlinks, and
  deployment with one coherent release?
• Is generation deterministic and publication fail-closed?
• Does it include the required specification changes and focused evidence?
• Did it invent a command, behavior, verification result, or production action?

A program may be structurally valid and still fail because it omitted a
necessary decision, violated the settled contract, or retained unnecessary
autonomy.

Results and follow-up

Preserve the generated program and completed evaluation for every recorded
trial.

For each meaningful discrepancy, record:

• the exact prompt and trial configuration;
• the relevant generated and hand-authored passages;
• the settled contract they are being compared against;
• why the difference matters;
• its classification; and
• the recommended product, documentation, best-practice, prompt, reference, or
  evaluation change.

Create a focused product issue only when the discrepancy is reproducible and
the expected behavior is settled. Do not change #678 merely to make a generated
program pass.

Repeat representative trials after an improvement so the corpus shows whether
the planning workflow became more reliable.

Acceptance

• One executable Markdown program evaluates the corpus without running generated
  programs or changing production state.
• Every prompt is stable, identified, and preserved verbatim.
• Every approved source artifact and digest is preserved exactly.
• Every trial records enough configuration and environment information to
  reproduce the observation.
• Independent trials cannot inherit context from one another.
• The evaluation compares both generated and hand-authored programs with Generate website reference documentation from xmd CLI output #678's
  settled contract.
• Deterministic findings are separated from human architectural judgment.
• The corpus includes natural, focused, integrated, and adversarial prompts.
• A well-formed but incomplete or unnecessarily autonomous program can fail with
  the reason stated clearly.
• Every meaningful discrepancy is classified before follow-up work is proposed.
• Documentation gaps and undefined practices are considered alongside product
  defects.
• No evaluation path evaluates a generated program, deploys the website, or
  changes release state.

Out of scope

• Implementing the website documentation workflow owned by Generate website reference documentation from xmd CLI output #678.
• Treating one model response as a deterministic guarantee.
• Evaluating generated programs against the repository or production services.
• Changing xmd plan before the evaluation supplies a reproducible finding.

Dependency and related work

• Generate website reference documentation from xmd CLI output #678 supplies the first authoritative reference contract and must settle its
  product behavior before trials receive verdicts.
• The portable planned-program Quest defines the source-artifact semantics this
  evaluation observes; it does not make delivery of the Quest a prerequisite
  for collecting prompts or comparing behavior already available.