Skip to content

docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure - #1402

Merged
thymikee merged 3 commits into
mainfrom
claude/agent-docs-optimization-f81c50
Jul 25, 2026
Merged

docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure#1402
thymikee merged 3 commits into
mainfrom
claude/agent-docs-optimization-f81c50

Conversation

@thymikee

@thymikee thymikee commented Jul 25, 2026

Copy link
Copy Markdown
Member

Summary

Applies the Claude 5 context-engineering guidance (Thariq, 2026-07-24) to the repo's agent docs. The principle: the always-loaded file carries gotchas and invariants; everything situational lives one hop away and loads when the task calls for it.

AGENTS.md: 315 → 233 lines (~58% fewer words), with a routing table at the top.

Removed:

  • Generic agent-behavior boilerplate — "First 60 Seconds", "When Blocked", generic rg advice. The article's "judgement over rules" case.
  • Three-way duplication — "Common Mistakes" restated Hard Rules and the registry section almost line for line; "Finding Source Owners" restated the registry section; session cleanup appeared 3x, SkillGym commands 2x.
  • Facts visible from the repo — the tsconfig strictness flag list, typescript being a dev dependency.

Kept always-loaded, because this is the layer worth spending tokens on: the nine expensive-lessons principles, enforcement gates, Hard Rules, environment traps, the adopted-runner false negative, module-size tripwires.

Moved behind the routing table:

  • docs/agents/cli-flags.md — the 10-step flag checklist, plus where CLI help/schema live
  • docs/agents/pull-requests.md — readiness, PR body conventions, review checklist
  • docs/agents/device-verification.md — staleness defeat, helper-path proof, session hygiene, sandbox traps
  • Testing Matrix folded into docs/agents/testing.md, reframed around pnpm check:affected. The script derives the gate set from repository sources of truth, so the prose table was drift risk; it now carries only the two traps agents actually miss.

CONTEXT.md keeps all 51 terms — that vocabulary is exactly the team-specific knowledge worth the tokens — but adds a section index and groups them (sessions/devices, command surface, refs & guarantees, gestures, snapshots, recording, Maestro, providers), so a task loads one section instead of a 200-line wall.

Selector fix (not docs-only). Moving the Testing Matrix out of AGENTS.md broke a sentinel: scripts/check-affected/model.ts treated only AGENTS.md as selector-owning, so a later matrix edit would have been silently classified inert docs while the selector kept deriving gates from a spec that had changed underneath it. The sentinel now lives with the prose as a named SELECTOR_OWNING_DOCS set, with the three stale references to the old location fixed (testing.md, and comments in model.ts / checks.ts).

Validation

Static gates, all green locally: pnpm check:affected:test (31/31), typecheck, lint, format:check. Full CI green on the pre-restoration head (24/24).

Selector behavior, verified by running the model rather than trusting the unit test:

path failOpen rule checks
docs/agents/testing.md true selector-owning 18
scripts/check-affected/model.ts true selector-owning 18
AGENTS.md false 0

AGENTS.md becoming docs-only is intended: it no longer defines the selection model, so ordinary agent-doc edits should not force the full local suite. The updated test is revert-sensitive — revert the model change and failOpenReasons[0] is undefined, so it fails rather than passing vacuously. Ordering also confirmed: isSelectorOwning runs before the isDocs short-circuit, so a docs/ path reaches the rule.

Content preservation was the main risk in a split this size, so it was checked mechanically, not by eye. Diffing backtick-identifier sets old vs new confirmed all 51 CONTEXT.md terms survived and every AGENTS.md identifier was either intentionally cut or present in its new home; that caught two losses during the split (the over-budget session.ts / apps.ts note, and the full path to runner-contract.ts).

That check had a blind spot: it only sees rules containing a backticked identifier. Review found two pure-prose rules it missed, both now restored — "test through public interfaces / no unrelated production exports for tests" (to testing.md, next to the behavioral-tests rule) and the guidance-ownership rule (to the always-loaded Docs & skills section, since it governs all command-surface work, not just flags). A follow-up prose-level sweep over all 200 original bullets confirmed those two were the only real losses; the rest are intentional cuts or reworded survivors.

Rebased onto origin/main; the conflict with #1397 was one new term (iOS WebView semantic presentation), re-placed into the Snapshots & capture group and re-verified at 51/51 against main.

Follow-up (not blocking)

This PR proves content survived, not that agents reliably follow the routing hop. Worth a small eval — SkillGym or a focused agent-doc eval — over representative prompts (add a CLI flag; change a platform response; verify on Android; review a routing PR; add a test), measuring both compliance and context consumed. That would turn this from a plausible documentation optimization into an evidence-backed one.

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.84 MB 1.84 MB 0 B
JS gzip 587.4 kB 587.4 kB 0 B
npm tarball 701.9 kB 701.9 kB 0 B
npm unpacked 2.46 MB 2.46 MB 0 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 29.9 ms 28.5 ms -1.4 ms
CLI --help 60.7 ms 60.1 ms -0.6 ms

Top changed chunks: no changes in the largest emitted chunks.

@thymikee

Copy link
Copy Markdown
Member Author

Two required follow-ups before readiness:

  1. docs/agents/pull-requests.md names vitest run --project unit-core --project subprocess-stub, but subprocess-stub is not a configured Vitest project. Please restore android-adb, matching package.json’s test:unit script and vitest.config.ts.

  2. This PR moves the Testing Matrix from AGENTS.md to docs/agents/testing.md, while scripts/check-affected/model.ts still makes only AGENTS.md selector-owning/fail-open. A later matrix edit will now be treated as inert docs, and the moved prose still refers to the “AGENTS.md Testing Matrix.” Please transfer the sentinel to docs/agents/testing.md, update its focused model test and that prose, then run pnpm check:affected:test.

@thymikee

Copy link
Copy Markdown
Member Author

Thanks — (2) is done. On (1) I think the current text is correct; evidence below.

2. Selector-owning sentinel — fixed in 2930be9

You were right that this was a live hole, and the failure mode was worse than "inert docs": a matrix edit would have been silently classified docs-only while the selector kept deriving gates from a spec that had changed underneath it.

Moved the sentinel to docs/agents/testing.md as a named SELECTOR_OWNING_DOCS set rather than a second hardcoded filename, so the next move is a one-line edit with the reason written down next to it. Also fixed the three stale references to the old location: the paragraph in testing.md, and the two in-code comments in model.ts and checks.ts that still cited "the AGENTS.md testing matrix".

Ordering was worth confirming — isSelectorOwning is checked at model.ts:306, before the isDocs short-circuit at :335, so a docs/ path does reach the rule.

Verified behaviorally, not just via the unit test:

path failOpen rule checks
docs/agents/testing.md true selector-owning 18
scripts/check-affected/model.ts true selector-owning 18
AGENTS.md false 0
docs/agents/pull-requests.md false 0

The updated test is revert-sensitive: drop the model change and failOpenReasons[0] is undefined, so the assertion fails rather than passing vacuously.

pnpm check:affected:test 31/31, plus typecheck / lint / format:check green for the scripts/ edits.

One consequence to confirm: you said "transfer", so I moved it rather than covering both — AGENTS.md is now classified docs-only (0 checks, table above). That's the literal reading and it's defensible now that the matrix is gone from it, but AGENTS.md does still carry the enforcement-gates list. If you'd rather it stay selector-owning as cheap insurance, it's one more entry in the set — say the word.

1. subprocess-stub — I believe the current text is right

subprocess-stub is a configured Vitest project, and it is exactly what test:unit runs:

  • vitest.config.ts:58name: 'subprocess-stub' (the serialized, fileParallelism: false group)
  • package.json test:unitvitest run --project unit-core --project subprocess-stub

android-adb doesn't exist as a Vitest project — it's a public package subpath (package.json exports, tsdown.config.ts:53), so --project android-adb would select nothing.

For provenance: this line is carried over verbatim from the old AGENTS.md:241; this PR only relocated it to docs/agents/pull-requests.md. Happy to change it if you're seeing something I'm not — but as written it matches the config and the script, so I've left it alone rather than introduce a name that doesn't resolve.

thymikee added 2 commits July 25, 2026 11:47
Apply the Claude 5 context-engineering guidance to the repo's agent docs:
keep the always-loaded file to gotchas and invariants, and move situational
guidance one hop away behind a routing table.

AGENTS.md 315 -> 229 lines. Cut generic agent-behavior boilerplate, three-way
duplication (Common Mistakes restated Hard Rules; Finding Source Owners
restated the registry section), and facts visible from the repo itself.
Kept verbatim: the expensive-lessons principles, enforcement gates, Hard
Rules, and environment traps.

Split out docs/agents/{cli-flags,pull-requests,device-verification}.md and
folded the Testing Matrix into docs/agents/testing.md, reframed around
pnpm check:affected so the prose stops duplicating the selector.

CONTEXT.md keeps all 50 terms, now grouped under a section index so a task
loads one section instead of the whole glossary.
… Matrix

The Testing Matrix moved from AGENTS.md to docs/agents/testing.md, but the
affected-check selector still treated only AGENTS.md as selector-owning. A
later matrix edit would have been classified as inert docs and skipped the
fail-open, so the selector could keep deriving gates from a spec that had
changed underneath it.

Move the sentinel with the prose, as a named SELECTOR_OWNING_DOCS set so the
next move is one line, and fix the two in-code comments plus the testing.md
paragraph that still pointed at the AGENTS.md matrix.
@thymikee
thymikee force-pushed the claude/agent-docs-optimization-f81c50 branch from 2930be9 to c63aaae Compare July 25, 2026 09:48
@thymikee

Copy link
Copy Markdown
Member Author

Re-reviewed exact head c63aaae. The selector-owning fix is correct: docs/agents/testing.md now fails open before the docs-only short-circuit, and the focused regression is revert-sensitive. The earlier subprocess-stub concern no longer applies on current main; it is the configured serialized Vitest project. All reported checks are green, and no device evidence is required. One non-code readiness cleanup remains: the PR body still says this is docs-only/no gates and that the selector sentinel was left for follow-up, but this head includes TypeScript selector behavior plus its test. Please update Summary/Validation/Follow-up so the body is self-contained. Code review is otherwise clean.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 25, 2026
Review caught two repo-specific rules that did not survive the move. Both are
prose without any backticked identifier, so the identifier-diff used to verify
the split could not see them.

- "Test through public interfaces; do not add unrelated production exports
  solely to enable tests" returns next to the behavioral-tests rule in
  docs/agents/testing.md, with the reason it exists.
- The guidance-ownership rule (decide whether new guidance/schema/metadata
  belongs to the command surface, CLI grammar, CLI help, MCP projection, or
  daemon runtime) returns to the always-loaded Docs & skills section, since it
  governs all command-surface work and not just the flag case.

Also point the ADR routing row at docs/adr/README.md, which is already the
"read when you touch…" index, rather than at the bare directory.
@thymikee

Copy link
Copy Markdown
Member Author

All three points addressed in 15f2a9c, plus the PR body rewritten.

Both dropped rules were real, and the way they escaped is worth recording. My completeness check for the split diffed backtick-identifier sets old vs new — which is why it caught the session.ts/apps.ts note and the runner-contract.ts path during the split, but was structurally blind to rules containing no identifier at all. Both of these are pure prose. Good catch.

  • Test through public interfaces / no unrelated production exports for testsdocs/agents/testing.md, next to "Keep tests behavioral", with the reason attached (widening the public surface for a test is a product change, and the export outlives the test).
  • Guidance ownership → restored to the always-loaded Docs & skills section rather than cli-flags.md, exactly as argued: it governs all command-surface work, and an agent changing command metadata has no reason to open the flags doc. It cross-references cli-flags.md for the flag case.
  • ADR routing → now points at docs/adr/README.md. Agreed, that file is already the "read when you touch…" index; pointing at the bare directory wasted the hop.

Rather than fix only what was spotted, I re-ran the completeness check at the prose level — every one of the 200 original AGENTS.md bullets, matched by word-shingle against the new tree. 42 flagged; on review, 40 are intentional cuts (generic agent behavior, facts visible from the repo) or reworded survivors, and the 2 genuine losses are exactly the ones above. So the file is now clean by both checks.

PR body rewritten. It no longer claims docs-only, the selector fix moved from Follow-up into Summary with its failure mode described, and Validation now covers check:affected:test 31/31, the static gates, the empirical selector table, and the rebase resolution.

On AGENTS.md no longer being selector-owning — agreed, and that reasoning is now recorded in the PR body so it doesn't read as an accident. I'd raised it as an open question earlier; consider it settled.

CI: 24/24 green on the pre-restoration head (c63aaae9d), including the iOS smoke job — mergeStateStatus reached CLEAN. The restoration commit is docs-only; re-running now.

On the follow-up eval — I think that's the most valuable idea raised in this review, and I agree it shouldn't block. The honest limit of this PR is that it proves content survived, not that the routing hop gets followed. Worth noting the hop is genuinely cheaper here than for CLI help topics: AGENTS.md is always loaded, so the routing table is read every time regardless, and a missed hop costs a round trip rather than silently omitted guidance. Captured as a Follow-up section in the body.

@thymikee
thymikee merged commit 1a76344 into main Jul 25, 2026
24 checks passed
@thymikee
thymikee deleted the claude/agent-docs-optimization-f81c50 branch July 25, 2026 10:13
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-25 10:13 UTC

thymikee added a commit that referenced this pull request Jul 25, 2026
* origin/main:
  docs(adr): rules-first ADR restructure + ADR 0017 proposal (unified event journal) (#1399)
  feat: add first-class Vega VVD TV support (#1396)
  fix(replay): preserve cwd scope for opened sessions (#1401)
  docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure (#1402)
  fix(cli): compact stale device status (#1388)
  feat: add WebView accessibility lab (#1397)
  feat: parameterize sensitive recorded inputs (#1369)
  fix(daemon): keep an active replay session's daemon alive over the CLI path (#1390)

# Conflicts:
#	docs/adr/0012-interactive-replay.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant