Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions .agents/skills/pr-status-triage/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,47 @@ Use this skill when the user asks about PR status, CI failures, or review commen

## Workflow

1. Run `node scripts/pr-status.js --wait` in the background (timeout 1 min), then read `scripts/pr-status/results/index.md`.
2. Analyze each `job-{id}.md` and `thread-{N}.md` file in `scripts/pr-status/results/` for failures and review feedback.
Start by fetching data to answer the user's query:

- For a targeted question about specific review comments, run `node scripts/pr-status.js [PR] --comments-only`. This skips all Actions run, job, log, and flaky-test requests.
- For CI status or general triage, run `node scripts/pr-status.js [PR] --wait` in the background (timeout 1 min).

Then:

1. Read `scripts/pr-status/results/index.md`.
2. For a targeted review question, inspect the relevant `thread-{N}.md`, `review-{id}.md`, or `comment-{id}.md` files. For full triage, also analyze each `job-{id}.md` and review file for failures and feedback.
3. Prioritize blocking jobs first: build, lint, types, then test jobs.
4. Treat failures as real until disproven; check the "Known Flaky Tests" section before calling anything flaky.
5. Reproduce locally with the same mode and env vars as CI.
5. Reproduce test failures locally with the same mode and environment as CI (e.g. dev or start, webpack or turbopack).
6. After addressing review comments, reply to the thread describing what was done, then resolve it. Use `reply-and-resolve-thread` to do both in one step, or use `reply-thread` + `resolve-thread` separately. See `scripts/pr-status/results/thread-N.md` files for ready-to-use commands.
7. When the only remaining failures are known flaky tests and no code changes are needed, retrigger the failing CI jobs with `gh run rerun <run-id> --failed`. Then wait 5 minutes and go back to step 1. Repeat this loop up to 5 times.

## CI Analysis Tips

- Prioritize CI failures over review comments.
- Prioritize blocking jobs first: build, lint, types, then test jobs.
- Common fast checks:
- `rust check / build` → Run `cargo fmt -- --check`, then `cargo fmt`
- `lint / build` → Run `pnpm prettier --write <file>` for prettier errors
- test failures → Run the specific failing test path locally

Run tests in the mode (e.g.):

```bash
# Development mode with Turbopack
pnpm test-dev-turbo test/path/to/test.ts

# Production build and start with Webpack
pnpm test-start-webpack test/path/to/test.ts
```

## Quick Commands

```bash
node scripts/pr-status.js # current branch PR
node scripts/pr-status.js <number> # specific PR
node scripts/pr-status.js [PR] --wait # background mode, waits for CI to finish
node scripts/pr-status.js [PR] --comments-only # reviews/comments only; skips CI jobs
node scripts/pr-status.js --skip-flaky-check # skip flaky test detection
```

Expand Down
44 changes: 4 additions & 40 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,46 +258,10 @@ That symlink does not bring in per-package `node_modules` or a built `packages/n

## PR Status (CI Failures and Reviews)

When the user asks about CI failures, PR reviews, or the status of a PR, run the pr-status script:

```bash
node scripts/pr-status.js # Auto-detects PR from current branch
node scripts/pr-status.js <number> # Analyze specific PR by number
```

This generates analysis files in `scripts/pr-status/`.

General triage rules (always apply; `$pr-status-triage` skill expands on these):

- Prioritize blocking failures first: build, lint, types, then tests.
- Assume failures are real until disproven; use "Known Flaky Tests" as context, not auto-dismissal.
- Reproduce with the same CI mode/env vars (especially `IS_WEBPACK_TEST=1` when present).
- For module-resolution/build-graph fixes, use the normal mode-specific test command so package resolution is exercised.

For full triage workflow (failure prioritization, mode selection, CI env reproduction, and common failure patterns), use the `$pr-status-triage` skill:

- Skill file: `.agents/skills/pr-status-triage/SKILL.md`

**Use `$pr-status-triage` for automated analysis** - see `.agents/skills/pr-status-triage/SKILL.md` for the full step-by-step workflow.

**CI Analysis Tips:**

- Prioritize CI failures over review comments
- Prioritize blocking jobs first: build, lint, types, then test jobs
- Common fast checks:
- `rust check / build` → Run `cargo fmt -- --check`, then `cargo fmt`
- `lint / build` → Run `pnpm prettier --write <file>` for prettier errors
- test failures → Run the specific failing test path locally

**Run tests in the right mode:**

```bash
# Dev mode (Turbopack)
pnpm test-dev-turbo test/path/to/test.ts

# Prod mode
pnpm test-start-turbo test/path/to/test.ts
```
Use `$pr-status-triage` whenever the user asks about CI failures, PR reviews, or
PR status. It contains the `scripts/pr-status.js` commands, prioritization,
review-thread workflow, and CI reproduction guidance. See
`.agents/skills/pr-status-triage/SKILL.md`.

## GitHub Pull Requests

Expand Down
2 changes: 2 additions & 0 deletions crates/next-api/src/next_server_nft.rs
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,8 @@ fn next_owned_ignores(
rcstr!("**/next/dist/compiled/webpack/*"),
rcstr!("**/node_modules/webpack5/**/*"),
rcstr!("**/next/dist/server/lib/route-resolver*"),
// Upgrade workflows are CLI-only and are not needed by production servers.
rcstr!("**/next/dist/lib/upgrade/**/*"),
// The testmode interceptors bundle reads its HTTP parser WASM with a
// dynamic path, making the tracer include the bundle's whole
// directory. Test proxying is not supported in standalone output, so
Expand Down
101 changes: 101 additions & 0 deletions docs/01-app/02-guides/upgrading/agentic-upgrade.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
title: Complete an agentic Next.js upgrade
description: Agent workflow for repository preflight, codemods, repairs, verification, and delivery of an agentic Next.js upgrade.
# Experimental agent workflow. Not ready for indexing; keep this document marked as draft.
version: draft
---

Each section has a checklist of items to complete. Read line by line and complete
each item before moving on to the next section.

When a security gate stops a command during another task, rerun that command once
as instructed so the original task can continue with a warning. Tell the user
about the finding and ask whether they want to run the upgrade. Continue
independent work while waiting. If approved, complete the upgrade.

For `latest`, the CLI pins npm's latest stable Next.js release as the target.

## 1. Check for duplicates

Before reading another guide or changing files, complete every item:

- [ ] Identify the repository provider from `git remote -v`, then use its CLI or
API to list every open pull request. For GitHub, run
`gh pr list --state open --limit 100 --json number,title,body,url,headRefName`.
- [ ] Inspect the body and diff of likely matches. For GitHub, run
`gh pr diff <number>`.
- [ ] Check local and remote branches and commits for an equivalent upgrade.

Look for unmarked upgrades and these markers:

```md
<!-- next-upgrade: <type>; path="." -->
<!-- next-upgrade: <type>; path="apps/web" -->
```

Do not substitute Git history for the provider lookup. If any check is unavailable,
fails, or finds existing work, stop before changing files and report it. Continue
to step 2 only after every check completes and finds no equivalent work.

## 2. Make a checklist

If the Next.js version is unchanged, make the checklist from the Future Defaults
adoption references in the prompt instead of the version-migration guides below.

- [ ] Read `./codemods.md` and the applicable `./version-<major>.md` files for
every crossed major, or the target major for a same-major upgrade.
- [ ] For version 14 and later, include the **Review migration checklist**
section. For version 13 and below, derive the checklist from the Pages
Router guide under `docs/02-pages/02-guides/upgrading/`.
- [ ] Stop if a required guide is missing.

## 3. Upgrade and repair

If the Next.js version is unchanged, follow the Future Defaults adoption
references in the prompt instead of the codemod steps below.

- [ ] Run the exact command prepared for this upgrade:

```text
<codemod-command>
```

- [ ] Run the codemod once, directly to the exact target. Do not install
intermediate Next.js versions or rerun the codemod for each crossed
major.
- [ ] If a required option is unavailable, correct the tool version or stop.
- [ ] Complete the codemod's manual steps, resolve its review markers, and
complete every applicable migration item. Keep only changes required by
the final target.

## 4. Verify

**This is the most important step.**

- [ ] Review the checklist created in step 2 and verify that every item has been
completed. Iterate until every item is addressed as applied or blocked
with a reason.
- [ ] Run the repository's checks, build when supported, and test affected
runtime behavior.

## 5. Commit and deliver

For adoption without a version change, group commits by the Future Default being
adopted instead of by crossed major.

- [ ] Commit the final diff in ascending major order, with one commit for each
crossed major. Each commit contains that major's surviving final-target
changes. Do not add transitional changes solely to make an intermediate
version work. Each commit message must include its source and target
versions and explain the change as if it were a PR description, so the
commit can later be split into its own PR.
- [ ] With permission, recheck open pull requests for duplicates and publish one
draft PR with the app marker:

```md
<!-- next-upgrade: <type>; path="." -->
<!-- next-upgrade: <type>; path="apps/web" -->
```

- [ ] If the upgrade is incomplete, report completed work, the blocker, and how
to resume.
9 changes: 9 additions & 0 deletions docs/01-app/02-guides/upgrading/version-14.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,12 @@ bun add next@next-14 react@18 react-dom@18 && bun add eslint-config-next@next-14
- The `next/server` import for `ImageResponse` was renamed to `next/og`. A [codemod is available](/docs/app/guides/upgrading/codemods#next-og-import) to safely and automatically rename your imports.
- The `@next/font` package has been fully removed in favor of the built-in `next/font`. A [codemod is available](/docs/app/guides/upgrading/codemods#built-in-next-font) to safely and automatically rename your imports.
- The WASM target for `next-swc` has been removed.

## Review migration checklist

- [ ] The app uses Node.js 18.17 or later and compatible React 18 dependencies and types.
- [ ] Static exports use `output: 'export'` and `next build` instead of `next export`.
- [ ] `ImageResponse` imports use `next/og` instead of `next/server`.
- [ ] Font imports use `next/font` instead of `@next/font`.
- [ ] The app does not rely on the removed `next-swc` WASM target.
- [ ] The app's supported checks and affected runtime behavior pass.
14 changes: 14 additions & 0 deletions docs/01-app/02-guides/upgrading/version-15.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -625,3 +625,17 @@ export function middleware(request: NextRequest) {
// ...
}
```

## Review migration checklist

After applying the changes relevant to your app, review these action items before
you consider the migration complete:

- [ ] React dependencies and types meet the React 19 requirements for the app's router.
- [ ] Async Request API access, including affected helpers and callers, is migrated. Resolve codemod error comments and `UnsafeUnwrapped` casts instead of treating them as completed repairs.
- [ ] Route segment runtime settings no longer use `experimental-edge`.
- [ ] Font imports use `next/font` instead of `@next/font`.
- [ ] Existing `bundlePagesExternals` and `serverComponentsExternalPackages` settings use their documented stable names while preserving package configuration.
- [ ] `NextRequest` `geo` and `ip` consumers use an appropriate source for the actual hosting provider.
- [ ] Apps that relied on automatic Speed Insights instrumentation have an explicit integration if they still need it.
- [ ] The app's supported checks and affected runtime behavior pass. Remaining required failures keep the migration incomplete.
21 changes: 20 additions & 1 deletion docs/01-app/02-guides/upgrading/version-16.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -801,7 +801,9 @@ const nextConfig = {
module.exports = nextConfig
```

If you specify a `quality` prop not included in the `image.qualities` array, the quality will be coerced to the closest value in `images.qualities`. For example, given the configuration above, a `quality` prop of 80, is coerced to 75.
If you specify a `quality` prop not included in the `images.qualities` array, the quality will be coerced to the closest value in `images.qualities`. For example, given the configuration above, a `quality` prop of 80, is coerced to 75.

Review custom image qualities. Add required values to `images.qualities`, and test direct `/_next/image` requests because unsupported `q` values return `400`.

### Local IP Restriction (Breaking change)

Expand Down Expand Up @@ -1246,3 +1248,20 @@ For the full migration path, see [Migrating to Cache Components](/docs/app/guide
### `unstable_rootParams`

The `unstable_rootParams` function has been removed. Use [`next/root-params`](/docs/app/api-reference/functions/next-root-params) instead.

## Review migration checklist

After applying the changes relevant to your app, review these action items before
you consider the migration complete:

- [ ] The app meets the Node.js, TypeScript, and React requirements in this guide.
- [ ] Async Request API access and affected callers are migrated. Resolve codemod error comments and `UnsafeUnwrapped` casts; use React `use` for synchronous Client Components where appropriate.
- [ ] Dynamic metadata image functions and sitemap functions handle their asynchronous parameters, and affected generated URLs work.
- [ ] Development and build scripts agree with the effective bundler configuration, including plugins, loaders, aliases, and affected Sass imports.
- [ ] Middleware and proxy changes preserve the required runtime and routing behavior. Edge middleware is not blindly renamed to proxy.
- [ ] Parallel route slots have appropriate defaults, and direct navigation and reloads retain the intended fallback behavior.
- [ ] Affected image requests succeed under the documented restrictions and defaults while preserving the intended quality.
- [ ] Existing cache API calls follow the documented signatures and semantics. React Compiler and Cache Components are not enabled merely to complete the upgrade.
- [ ] Removed APIs and configuration have no remaining consumers. Runtime configuration replacements preserve server and client visibility and the required evaluation time.
- [ ] Scripts and CI no longer invoke `next lint`; linting runs separately where the app requires it.
- [ ] The app's supported checks and affected runtime behavior pass. A green build alone does not close runtime findings.
3 changes: 3 additions & 0 deletions eslint.cli.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ export default defineConfig([
// tsconfig, not repo code — EVAL.ts files may import modules that only
// resolve inside the sandbox (e.g. @vercel/agent-eval/eval).
'evals/evals/**/*',
'evals/next-upgrade/evals/**/*',
'evals/next-upgrade/results/**/*',
'evals/next-upgrade/shared/**/*',
'examples/**/*',
'test/**/*',
'**/*.d.ts',
Expand Down
3 changes: 3 additions & 0 deletions eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,9 @@ export default defineConfig([
// tsconfig, not repo code — EVAL.ts files may import modules that only
// resolve inside the sandbox (e.g. @vercel/agent-eval/eval).
'evals/evals/**/*',
'evals/next-upgrade/evals/**/*',
'evals/next-upgrade/results/**/*',
'evals/next-upgrade/shared/**/*',
'examples/**/*',
'test/**/*',
'**/*.d.ts',
Expand Down
15 changes: 15 additions & 0 deletions evals/lib/environment.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
const fs = require('node:fs')
const path = require('node:path')

/** Make the repo's existing vc env pull output available to agent-eval. */
function linkEnvironment(root, directory) {
for (const name of ['.env', '.env.local']) {
const source = path.join(root, name)
const destination = path.join(directory, name)
try {
fs.rmSync(destination, { force: true })
if (fs.existsSync(source)) fs.symlinkSync(source, destination)
} catch {}
}
}
module.exports = { linkEnvironment }
24 changes: 24 additions & 0 deletions evals/lib/pack.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
const fs = require('node:fs')
const path = require('node:path')
const { execFileSync } = require('node:child_process')

/** Pack one built workspace package for upload by an eval runner. */
function packPackage(packageDirectory, destination) {
const directory = path.dirname(destination)
fs.mkdirSync(directory, { recursive: true })
const output = execFileSync(
'pnpm',
['pack', '--pack-destination', directory],
{
cwd: packageDirectory,
encoding: 'utf8',
}
)
const produced = output.trim().split('\n').pop()
const source = path.isAbsolute(produced)
? produced
: path.join(directory, produced)
if (source !== destination) fs.renameSync(source, destination)
return destination
}
module.exports = { packPackage }
4 changes: 4 additions & 0 deletions evals/next-upgrade/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.tarballs/
results/
.env
.env.local
46 changes: 46 additions & 0 deletions evals/next-upgrade/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Next.js upgrade evals

This suite extends the existing `@vercel/agent-eval` setup. Fixtures use exact old
Next.js versions, while the separately packed candidate provides the global
`next upgrade` command.

## Run

Use the existing [eval credential setup](../README.md#one-time-setup): `vc link`
and `vc env pull` at the repo root. Both runners share environment-file linking
and package packing. Authentication, sandbox selection, native agents, withheld
assertions, judging and result storage belong to `@vercel/agent-eval`.

```sh
pnpm build-all
pnpm eval:upgrade <fixture-name> --dry
NEXT_UPGRADE_EVAL_EXPERIMENT=codex pnpm eval:upgrade <fixture-name>
```

Omit the experiment filter to run Codex and Claude. `--list` lists fixtures without
packing or making model calls. Run one named fixture at a time. Results use the
framework's normal `results/` layout. Fixtures are added by the feature PRs stacked
above this infrastructure.

## Lifecycle

1. Create one temporary Vercel Sandbox snapshot with the agent CLIs.
2. Upload the fixture and establish its git baseline.
3. Install candidate Next.js and codemod packages separately, route npm and npx
upgrade commands to the candidate CLI, then install app dependencies.
4. Snapshot the prepared fixture and fork each selected agent from it.
5. Run each native agent and judge independently. Agent-eval withholds `EVAL.ts`
and captures transcripts and results as usual.

Package archives use the same fixed, overwritten paths as existing evals. Invalid
fixtures fail before execution, and infrastructure failures are retained in the results.

## Adding feature coverage

Feature PRs add ordinary npm app fixtures with exact dependency versions,
`PROMPT.md`, and `EVAL.ts`. Explicit upgrade scenarios invoke
`npx next@canary upgrade --ai`. Reminder scenarios keep the original task prompt
unchanged so the eval can attribute acknowledgment to the runtime notice. Feature
PRs own browser setup, repository remotes, advisory responses, codemod routing,
grading, and reference or negative controls. Keep graders and reference solutions
withheld, and retain sandbox or authentication failures as failures.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.next/
*.tsbuildinfo
__agent_eval__/
eval-evidence/
node_modules/
next-env.d.ts
!package-lock.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Do not push or create pull requests. Use npm.
Loading
Loading