Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions .agents/skills/create-pr-await-qodo-review/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
---
name: create-pr-await-qodo-review
description: Orchestrate the full PR creation workflow — commit, branch, push, create the PR via gh CLI, then optionally wait for the Qodo bot's automated code review and surface its findings. Use this skill whenever the user asks to "create a PR", "open a PR", "submit a PR", "commit and push and create a PR", or any variation of making a pull request. Also trigger when the user says "let's PR this" or "ship it" in a context where there are uncommitted or unpushed changes, or when they want to wait for / poll / check the Qodo review on a PR.
---

# Create PR and Await Qodo Review

This skill walks through the complete pull request workflow for this repository, from staging changes through to waiting for the automated Qodo code review and surfacing its findings.

The reason this skill exists is that PR creation involves several coordinated steps where getting the details right matters — conventional commit format, structured PR bodies, and the opportunity to catch review feedback before context-switching away. Following this workflow means the PR is ready for human review the moment it lands.

## Workflow

### Step 1: Pre-flight

Run these in parallel to understand what you're working with:

```bash
git status # untracked + modified files
git diff # unstaged changes
git diff --cached # staged changes
git log --oneline -5 # recent commit style
```

Before proceeding, verify there are actual changes to commit. If the working tree is clean, tell the user and stop.

### Step 2: Commit

Stage the relevant files — prefer naming specific files over `git add -A` to avoid accidentally including sensitive files (`.env`, credentials, local settings).

Write a conventional commit message following the project's release flow (see `docs/guides/release-flow.md`). The format is `type(scope): description` or `type: description`.

**Version-bumping types:** `feat` (minor), `fix` (patch), `perf` (patch), `deps` (patch), `revert` (patch)
**Non-bumping types:** `chore`, `docs`, `style`, `refactor`, `test`, `build`, `ci`

End every commit message with:
```
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```

Use a HEREDOC for the commit message to preserve formatting:
```bash
git commit -m "$(cat <<'EOF'
type(scope): short description

Longer explanation of why, not what.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
EOF
)"
```

### Step 3: Branch and push

If currently on `main`, create a feature branch first — never commit directly to main. Branch naming convention: `type/short-description` (e.g., `fix/yaml-parsing`, `ci/harden-workflows`).

```bash
git checkout -b <branch-name> # only if on main
git push -u origin <branch-name>
```

### Step 4: Create the PR

Use `gh pr create` with a structured body. PR titles follow the same conventional commit format (they become the squash-merge commit message on main).

```bash
gh pr create --title "type(scope): description" --body "$(cat <<'EOF'
## Summary
<1-3 bullet points explaining what and why>

## Test plan
[Describe how changes were verified]

🤖 Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```

Capture the returned PR URL — `gh pr create` prints it to stdout.

### Step 5: Wait for the Qodo review (always)

Every PR in this repo gets an automated Qodo code review, and the whole point of this skill is to catch that feedback before you context-switch away — so **always** wait for it. Don't ask the user whether to wait; just tell them you're watching for it and that they're free to keep working in the meantime.

> "PR created at {url}. I'll watch for the Qodo code review and surface its findings as soon as it lands."

Drive the wait with the **Monitor** tool, not a foreground command. The poller streams progress to stderr (which goes only to the monitor's output file) and prints exactly one terminal event to stdout — the full review body when it lands, or a `TIMEOUT:` line if it never does — so each notification is something you'd actually act on, with no log noise in between.

Call the `Monitor` tool with:

- **command**: `python scripts/review_poll.py <pr-url> --interval 30 --timeout 900`
- **description**: something specific like `"Qodo review on PR #123"` (it appears in every notification)
- **persistent**: `true` — the poller ends itself when the review lands or it hits its own `--timeout`, so let it run for the session rather than racing a separate monitor deadline

Then keep working. When the monitor fires, react to what it emitted:

- **Review body** (markdown) → parse out findings. Look for sections like `Bugs`, `Rule violations`, `Action required`, or `Review recommended`. Summarize them for the user, highlighting bugs and actionable items, and offer to fix anything real.
- **`TIMEOUT: ...` line** (or a non-zero exit) → the review didn't post in time. Tell the user, share the PR URL, and offer to re-arm the monitor with a longer `--timeout` or check the PR manually.

A 30s poll interval keeps you clear of GitHub API rate limits, and 900s (15 min) comfortably covers how long Qodo typically takes. Bump `--timeout` for unusually large PRs.

## Script reference

**`scripts/review_poll.py`** — Polls a GitHub PR for the Qodo bot's code review comment.

```
python scripts/review_poll.py <pr-url> [--interval 15] [--timeout 300]
```

- Accepts `https://github.com/owner/repo/pull/123` or `owner/repo#123`
- Shells out to `gh api` (Python standard library only, no third-party dependencies)
- Detects the Qodo bot comment by author login (contains "qodo")
- Distinguishes placeholder ("Looking for bugs? Check back in a few minutes") from the final review (contains `Bugs`, `Rule violations`, `Action required`, etc.)
- Exit 0 = review found (body on stdout), exit 1 = timeout (a `TIMEOUT:` line on stdout + message on stderr), exit 2 = bad args
- Designed for the Monitor tool: only terminal events (the review body, or the `TIMEOUT:` line) go to stdout; per-poll progress goes to stderr so it never spams notifications

## Decision patterns

### When to create a new branch

- On `main` → always create a branch
- On an existing feature branch → stay on it (the user likely has in-progress work)
- Branch already tracks remote and is up to date → just push new commits

### Commit granularity

- One logical change = one commit. Don't bundle unrelated fixes.
- If the user explicitly asks for a single commit covering multiple changes, that's fine.
- If a pre-commit hook fails, fix the issue and create a NEW commit (don't amend — the failed commit didn't happen).

### PR body depth

- Small changes (typos, config tweaks) → brief summary, minimal test plan
- Feature work → explain the why, list affected areas, thorough test plan
- Security/CI changes → explain the threat model or reasoning, link to relevant incidents or docs
39 changes: 39 additions & 0 deletions .agents/skills/create-pr-await-qodo-review/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
[
{
"id": 1,
"name": "full-pr-flow-always-waits",
"description": "User asks to open a PR for finished work — the skill should run the commit/branch/push/create flow and then ALWAYS wait for the Qodo review without asking permission, driving the wait with the Monitor tool.",
"prompt": "The origin-resolver changes on this branch are done and tested — let's open a PR against main.",
"trap": "Model asks the user whether to wait for the Qodo review (it should always wait), runs the poller in the foreground (blocking), or forgets the review wait entirely after creating the PR.",
"assertions": [
{ "id": "1.1", "text": "Follows the commit -> branch (if on main) -> push -> `gh pr create` sequence with a conventional-commit title" },
{ "id": "1.2", "text": "Waits for the Qodo review unconditionally — does NOT ask the user whether to wait" },
{ "id": "1.3", "text": "Drives the wait with the Monitor tool running `scripts/review_poll.py` (persistent), not a blocking foreground command" },
{ "id": "1.4", "text": "States it will surface the Qodo findings once the review lands and that the user can keep working meanwhile" }
]
},
{
"id": 2,
"name": "ship-it-from-main",
"description": "Casual 'ship it' on dirty working tree while on main — must create a branch first (never commit to main), use conventional commit + structured PR body, then wait for Qodo via Monitor.",
"prompt": "ok ship it — branch, push, and raise the pull request",
"trap": "Model commits directly to main, uses a non-conventional commit/PR title, or skips the Qodo review wait.",
"assertions": [
{ "id": "2.1", "text": "Creates a feature branch before committing because the change would otherwise land on main" },
{ "id": "2.2", "text": "Uses conventional-commit format for both the commit message and the PR title" },
{ "id": "2.3", "text": "Arms the Monitor tool on `review_poll.py` to wait for the Qodo review after the PR is created" }
]
},
{
"id": 3,
"name": "check-review-on-existing-pr",
"description": "User asks specifically about the Qodo review on an already-open PR — the skill should poll that PR for the review rather than creating a new one.",
"prompt": "what did qodo flag on PR #128? has the review landed yet?",
"trap": "Model tries to create a new PR, or hits the GitHub API once and reports 'not ready' instead of watching until the review lands.",
"assertions": [
{ "id": "3.1", "text": "Recognizes the PR already exists and does NOT run the commit/branch/create flow" },
{ "id": "3.2", "text": "Uses the Monitor tool with `review_poll.py` against PR #128 to wait for and report the Qodo findings" },
{ "id": "3.3", "text": "Summarizes the Qodo findings (or reports the timeout) rather than dumping the raw comment" }
]
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[
{ "query": "The origin-resolver changes on this branch are done and tested — let's open a PR against main.", "should_trigger": true },
{ "query": "commit what I've got staged and push it up as a pull request", "should_trigger": true },
{ "query": "feature's finished, can you create a PR for these changes and tell me what the qodo review comes back with?", "should_trigger": true },
{ "query": "let's PR this", "should_trigger": true },
{ "query": "ok ship it — branch, push, and raise the pull request", "should_trigger": true },
{ "query": "open a pull request with a proper summary and test plan, then wait for the bot review", "should_trigger": true },
{ "query": "push my work to a new branch and submit a PR", "should_trigger": true },
{ "query": "what did qodo flag on PR #128? has the review landed yet?", "should_trigger": true },
{ "query": "open PR #128 in my browser so I can read through the discussion", "should_trigger": false },
{ "query": "review this diff with me in crit before I commit anything", "should_trigger": false },
{ "query": "rebase my feature branch onto main and help me resolve the conflicts", "should_trigger": false },
{ "query": "what's the status of the CI checks on my current branch?", "should_trigger": false },
{ "query": "just stage and commit these changes locally — don't push or open anything yet", "should_trigger": false }
]
176 changes: 176 additions & 0 deletions .agents/skills/create-pr-await-qodo-review/scripts/review_poll.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
#!/usr/bin/env python3
"""
review_poll.py — Poll a GitHub PR for the Qodo code review comment.

Qodo posts a placeholder comment immediately after PR creation, then edits it
with the full review once analysis is complete. This script polls until the
final review appears and prints it to stdout.

Usage:
python scripts/review_poll.py <pr-url> [--interval 15] [--timeout 300]

Accepts:
https://github.com/owner/repo/pull/123
owner/repo#123

Exit codes:
0 — review found (body printed to stdout)
1 — timeout
2 — argument/usage error

Standard library only — shells out to `gh api`, no third-party dependencies.
"""

import argparse
import json
import re
import subprocess
import sys
import time

PLACEHOLDER_MARKERS = (
"Looking for bugs?",
"Check back in a few minutes",
)

REVIEW_READY_MARKERS = (
"Bugs",
"Rule violations",
"Action required",
"Review recommended",
"Requirement gaps",
"UX Issues",
)

VALID_SLUG = re.compile(r"^[A-Za-z0-9_.-]+$")


def parse_pr(value):
"""Parse a PR reference into (owner, repo, number) or raise ValueError."""
url_match = re.search(r"github\.com/([^/]+)/([^/]+)/pull/(\d+)", value)
if url_match:
owner, repo, number = url_match.groups()
else:
short_match = re.match(r"^([^/]+)/([^#]+)#(\d+)$", value)
if not short_match:
raise ValueError(f"could not parse PR reference: {value}")
owner, repo, number = short_match.groups()

if not VALID_SLUG.match(owner) or not VALID_SLUG.match(repo):
raise ValueError(f"invalid owner/repo characters: {owner}/{repo}")
if not number.isdigit():
raise ValueError(f"invalid PR number: {number}")

return owner, repo, number


def positive_int(value):
ivalue = int(value)
if ivalue < 1:
raise argparse.ArgumentTypeError(f"must be a positive integer (got: {value})")
return ivalue


def parse_args(argv):
parser = argparse.ArgumentParser(
prog="review_poll.py",
description="Poll a GitHub PR for the Qodo code review comment.",
)
parser.add_argument("pr", metavar="pr-url", help="GitHub PR URL or owner/repo#number")
parser.add_argument(
"--interval", type=positive_int, default=15, help="seconds between polls (default: 15)"
)
parser.add_argument(
"--timeout", type=positive_int, default=300, help="max seconds to wait (default: 300)"
)
args = parser.parse_args(argv)

try:
owner, repo, number = parse_pr(args.pr)
except ValueError as err:
parser.error(str(err))

return owner, repo, number, args.interval, args.timeout


def fetch_comments(owner, repo, number):
"""Return a list of {id, author, body} dicts for the PR's issue comments."""
result = subprocess.run(
[
"gh",
"api",
f"repos/{owner}/{repo}/issues/{number}/comments",
"--paginate",
"--jq",
".[] | {id, author: .user.login, body}",
],
capture_output=True,
text=True,
check=True,
)
# --paginate with --jq outputs one JSON object per line (NDJSON).
return [json.loads(line) for line in result.stdout.splitlines() if line.strip()]


def is_placeholder(body):
return any(marker in body for marker in PLACEHOLDER_MARKERS)


def is_review_ready(body):
return not is_placeholder(body) and any(marker in body for marker in REVIEW_READY_MARKERS)


def find_qodo_review_comment(comments):
"""Pick the Qodo "Code Review" comment, falling back to placeholder/last."""
qodo = [c for c in comments if c.get("author") and "qodo" in c["author"].lower()]

code_review = next((c for c in qodo if "Code Review" in c["body"]), None)
if code_review:
return code_review

placeholder = next((c for c in qodo if is_placeholder(c["body"])), None)
if placeholder:
return placeholder

return qodo[-1] if qodo else None


def main():
owner, repo, number, interval, timeout = parse_args(sys.argv[1:])
pr_ref = f"{owner}/{repo}#{number}"

print(f"Polling {pr_ref} every {interval}s (timeout: {timeout}s)...", file=sys.stderr)

start = time.monotonic()
while time.monotonic() - start < timeout:
elapsed = round(time.monotonic() - start)
try:
comments = fetch_comments(owner, repo, number)
qodo = find_qodo_review_comment(comments)

if qodo:
if is_review_ready(qodo["body"]):
print(f"Review ready for {pr_ref}", file=sys.stderr)
print(qodo["body"])
return 0
print(
f"Found Qodo comment but still placeholder... ({elapsed}s elapsed)",
file=sys.stderr,
)
else:
print(f"No Qodo comment yet... ({elapsed}s elapsed)", file=sys.stderr)
except subprocess.CalledProcessError as err:
print(f"API error: {err.stderr.strip()}, retrying...", file=sys.stderr)

time.sleep(interval)

# Emit the timeout on stdout too, not just stderr: when run under the
# Monitor tool only stdout lines become notifications, and a silent exit
# would be indistinguishable from "still polling".
print(f"TIMEOUT: no Qodo review for {pr_ref} after {timeout}s")
print(f"Timeout: no Qodo review found after {timeout}s", file=sys.stderr)
return 1


if __name__ == "__main__":
sys.exit(main())
Loading