docs(evaluations): show a typed outcome grader (EvalGraderFunction / EvalGraderContext / EvalScriptOutcome) — every example hand-typed the contract

[ScriptedAlchemy]

Problem

guide/development/evaluations.mdx shows how to declare an outcome grader (expectOutcome({ script: './graders/status-result.ts' })) but never shows what that script is: its argument, its return shape, or that agent-bundle/eval already exports the types for both (EvalGraderFunction, EvalGraderContext, EvalScriptOutcome). Authors reverse-engineer the contract from the runtime and hand-type it — which is exactly what all three shipped examples did until #471.

Evidence (main 833e48fdc)

• website/docs/en/guide/development/evaluations.mdx:17-33 — the only grader mention is the expectOutcome({ script }) line in the suite sample; there is no grader module sample and no reference to the grader types. Same gap in website/docs/zh/guide/development/evaluations.mdx.

• The types are public and unmentioned: packages/agent-bundle/src/eval/index.ts:75-86 exports EvalGraderContext ({ artifactRoot, fixturePath, prompt }), EvalGraderFunction, EvalScriptGraderSpec, …; packages/agent-bundle/src/eval/graders.ts:27-31 / :68-70 define them; packages/agent-bundle/src/eval/types.ts:92-95 defines EvalScriptOutcome = { detail: string; outcome: 'pass' | 'fail' | 'inconclusive' }.

• Runtime contract the doc leaves implicit: packages/agent-bundle/src/eval/graders.ts:189-201 loads the script through jiti and requires a default-exported function called with { artifactRoot, fixturePath, prompt }.

• Hand-typed graders replaced in chore(examples): use built-in framework surface — hooks-and-scripts, host-test, mcp-app, skills-starter #471 (2e59d6eeb), each restating ({ fixturePath }: { readonly fixturePath: string }) => … { outcome: 'pass' as const }:

  • examples/skills-starter/evals/graders/release-result.ts
  • examples/skills-starter/evals/graders/operations-result.ts
  • examples/mcp-app/evals/graders/status-result.ts

  They now read const grade: EvalGraderFunction = async ({ fixturePath }) => …; export default grade; — that is the sample the doc should carry.

Found while auditing the examples for hand-rolled equivalents of public surface (#471).

Proposed shape

Docs-only. Add a "Writing an outcome grader" subsection to evaluations.mdx (en + zh) right after the suite sample:

// evals/graders/status-result.ts
import { readFile } from 'node:fs/promises';
import { join } from 'node:path';

import type { EvalGraderFunction } from 'agent-bundle/eval';

const grade: EvalGraderFunction = async ({ fixturePath }) => {
  const result = JSON.parse(await readFile(join(fixturePath, 'result.json'), 'utf8')) as unknown;
  return isHealthy(result)
    ? { detail: 'The compiler service is healthy.', outcome: 'pass' }
    : { detail: 'The compiler service did not report a healthy status.', outcome: 'fail' };
};

export default grade;

Plus a three-row table for EvalGraderContext (artifactRoot, fixturePath, prompt) and one sentence on EvalScriptOutcome (outcome is pass | fail | inconclusive; detail is shown in the trial record). Link EvalGraderFunction to its TypeDoc page. No code change; the grader types already exist, so no changeset.

Acceptance

• evaluations.mdx (en and zh) shows a default-exported grader typed as EvalGraderFunction, documents the three context fields and the outcome shape, and the sample is the same shape as examples/mcp-app/evals/graders/status-result.ts.
• pnpm docs:site:build passes (twoslash on the new sample, dead-link and parity checks).
• A reader can write a grader without opening packages/agent-bundle/src/eval/graders.ts.