feat(site): serve the docs to coding agents, and upgrade Astro

[vishr]

Makes the documentation legible to the AI coding agents developers now use to write Echo code, and takes the site off an Astro release with open advisories.

Why

Echo's docs are currently invisible to agents in three separate ways, and the fixes are independent:

1. No machine-readable surface. No llms.txt, no raw-Markdown twins. An agent that wants a page has to fetch and parse rendered HTML.
2. robots.txt declares nothing. The site has no robots.txt of its own — Cloudflare's managed Content Signals block is all that's served, and it is comments only: no user-agent rules, no signal values, no Sitemap: line.
3. Context7 serves v4. Context7 is what many coding agents query for library docs. Echo's entries there are fragmented across three records, and the one that scores highest is pinned to v4.15.0.

The third is the reason agents keep emitting v4 Echo code.

What changed

Astro 6.4.7 → 7.3.2, Starlight 0.40 → 0.42

Not cosmetic. npm audit on the previous tree reports 7 vulnerabilities, 1 critical, rooted in astro itself:

• Authorization bypass via decode iteration limit and rewrite path canonicalization mismatch
• Remote code execution through AVIF image optimization
• XSS via unescaped spread attribute names (incomplete fix for CVE-2026-54298)
• XSS via transition:* directive values on hydrated islands, and via View Transition animation properties
• Authorization bypass from a missing path-segment boundary check when stripping the configured base

After the upgrade the tree audits clean (0 vulnerabilities). The build produces the same pages; there are no content changes in this PR.

Per-page Markdown twins

starlight-dot-md serves every page as raw Markdown at <path>.md — e.g. /guide/routing.md. 296 twins, one per page per locale. Frontmatter and fenced code blocks are preserved.

llms.txt / llms-full.txt / llms-small.txt

starlight-llms-txt, by a Starlight core maintainer. The cookbook is excluded from the small set so it is actually smaller — 140K vs 204K, not the 2% difference the defaults produce. Code blocks stay in both; prose without snippets is useless for a Go framework.

This is not an SEO change and should not be reviewed as one. AI search crawlers overwhelmingly skip llms.txt — one study found 408 direct fetches across 500M AI-bot visits over 90 days — and Google states it has no effect on Search or AI Overviews. The audience is Claude Code, Cursor, Copilot and Cline.

The preamble is where the value is: it states that these docs are v5, that the module path is github.com/labstack/echo/v5, and that v4 is a separate incompatible API.

robots.txt

Explicit allow-everything, plus the sitemap location and Content-Signal: search=yes, ai-input=yes, ai-train=yes.

The per-agent groups are redundant against User-agent: * today. They are there so intent is recorded rather than inferred. Worth knowing which way the asymmetry runs: blocking training crawlers (GPTBot, ClaudeBot, CCBot) costs no citations, while blocking retrieval and agent crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, ChatGPT-User, Claude-User) is what removes a site from AI answers.

context7.json

Scopes Context7 indexing to the English docs and the cookbook — the four translations were diluting the snippet set — and states the v5/v4 split in rules, including the signature change:

// v4
func(c echo.Context) error   // Context is an interface
// v5
func(c *echo.Context) error  // Context is a struct

Two things needing your decision

1. ai-train=yes. I defaulted all three content signals to yes, on the reasoning that the goal is for models to know Echo v5. If you want training declined while keeping search and answers, it becomes ai-train=no.
2. Cloudflare's managed robots.txt. The zone currently serves Cloudflare's block. Once this deploys there will be two sources; the managed setting should be turned off, or verified to append rather than shadow, so exactly one file is authoritative. Separately, Cloudflare's three-tier Search/Agent/Training default change takes effect 2026-09-15 — worth checking the zone before then so Echo isn't opted out by a default flip.

Follow-ups, not in this PR

• context7.json in labstack/echo (v5 on master, v4 as a previous version) — sent separately.
• Claiming the Context7 entries needs url + public_key from context7.com/dashboard, which only you can generate. The three fragmented records (/labstack/echo, /labstack/echox, /websites/echo_labstack — the last has 15 snippets) need an upstream issue to merge.
• Per-page OG images, self-hosted fonts and icons (currently render-blocking from unpkg and Google Fonts), and JSON-LD were scoped out; JSON-LD in particular has weak evidence behind it for AI citation.

Verification

npm audit          # 0 vulnerabilities (was 7, 1 critical)
npm run build      # Complete, 301 routes
dist/llms.txt              1.4K
dist/llms-small.txt        140K   50 pages
dist/llms-full.txt         204K   70 pages
dist/robots.txt            1.9K
find dist -name '*.md'     296

https://claude.ai/code/session_01ABWLbEypvJUVLLY26pHxtT