diff --git a/README.md b/README.md index f3177aa..29f1a2e 100644 --- a/README.md +++ b/README.md @@ -508,17 +508,22 @@ Custom slash commands are markdown files that define parameterized procedures. T mkdir -p ~/.claude/commands cp commands/review-pr.md ~/.claude/commands/ cp commands/fix-issue.md ~/.claude/commands/ +cp commands/merge-dependabot.md ~/.claude/commands/ ``` ### Review PR -[`commands/review-pr.md`](commands/review-pr.md) -- Reviews a GitHub PR with parallel agents, fixes findings, and pushes. Invoke with `/review-pr 456` where `456` is the PR number. +[`commands/review-pr.md`](commands/review-pr.md) -- Reviews a GitHub PR with parallel agents (pr-review-toolkit, Codex, Gemini), fixes findings, and pushes. Invoke with `/review-pr 456` where `456` is the PR number. ### Fix Issue -[`commands/fix-issue.md`](commands/fix-issue.md) -- Takes a GitHub issue and fully autonomously completes it -- plans, implements, tests, creates a PR, self-reviews with parallel agents, fixes its own findings, and comments on the issue when done. Invoke with `/fix-issue 123` where `123` is the issue number. +[`commands/fix-issue.md`](commands/fix-issue.md) -- Takes a GitHub issue and fully autonomously completes it -- researches, plans, implements, tests, creates a PR, self-reviews with parallel agents, fixes its own findings, and comments on the issue when done. Invoke with `/fix-issue 123` where `123` is the issue number. -Once a workflow is a command, it's not just faster for you -- it's something an agent can run too. You can point `/fix-issue` at 50 issues in parallel across worktrees, run `/review-pr` on every open PR in a repo, or schedule either as part of CI. Commands turn manual workflows into scalable operations. +### Merge Dependabot + +[`commands/merge-dependabot.md`](commands/merge-dependabot.md) -- Evaluates and merges open dependabot PRs for a repo. Audits dependabot config, builds a transitive dependency map, batches overlapping PRs, evaluates each in parallel (build, test, matrix gap analysis), and merges passing PRs sequentially with post-merge re-testing. Invoke with `/merge-dependabot trailofbits/algo`. + +Once a workflow is a command, it's not just faster for you -- it's something an agent can run too. You can point `/fix-issue` at 50 issues in parallel across worktrees, run `/review-pr` on every open PR in a repo, run `/merge-dependabot` across all your repos, or schedule any of them as part of CI. Commands turn manual workflows into scalable operations. ## Writing Skills and Agents diff --git a/commands/fix-issue.md b/commands/fix-issue.md index d9b0623..7f38bb8 100644 --- a/commands/fix-issue.md +++ b/commands/fix-issue.md @@ -1,18 +1,43 @@ # Fix GitHub Issue -@description End-to-end: plan, implement, test, PR, review, fix findings, and comment on a GitHub issue. +@description End-to-end: plan, implement, test, review, fix, push, and PR for a GitHub issue. @arguments $ISSUE_NUMBER: GitHub issue number to fix -Read GitHub Issue #$ISSUE_NUMBER thoroughly. Understand the full -context: problem description, acceptance criteria, linked PRs, -and any discussion. Follow linked issues, referenced PRs, and -external documentation to build complete understanding before -planning. +Read GitHub Issue #$ISSUE_NUMBER from the canonical repo +(`gh issue view $ISSUE_NUMBER --repo `) thoroughly. +Understand the full context: problem description, acceptance +criteria, linked PRs, and any discussion. Follow linked issues, +referenced PRs, and external documentation to build complete +understanding before planning. + +Detect the upstream repository: if a git remote named `upstream` +exists, use it as the canonical repo (fetch from it, base branches +on it, and submit PRs to it). Otherwise, fall back to `origin`. +Resolve the canonical repo's `owner/name` (e.g. from `git remote +get-url upstream`) and store it — use `--repo ` on +every `gh` command (issue views, PR creation, issue comments) to +ensure they target the correct repository. Run +`git fetch ` to ensure you are working with +up-to-date code. Execute every step below sequentially. Do not stop or ask for confirmation at any step. -## 1. Plan +## 1. Research (if needed) + +Before planning, determine if the issue requires external context +you don't already have — unfamiliar APIs, protocols, libraries, +error messages, or domain-specific concepts. If so, use Exa +(`mcp__exa__web_search_exa`) to search for: + +- Official documentation for referenced libraries or APIs +- Known solutions for the error messages or symptoms described +- Implementation patterns used by similar projects + +Skip this step for straightforward bugs where the fix is clear +from the codebase alone. + +## 2. Plan Write a detailed implementation plan to `plan-issue-$ISSUE_NUMBER.md` in the repo root. The plan must: @@ -23,37 +48,162 @@ in the repo root. The plan must: - Call out risks or open questions - Reference relevant code paths by file:line -## 2. Implement +## 3. Create branch -Implement the plan across all necessary files. Follow the -project's CLAUDE.md standards. Keep changes minimal and focused -on the issue requirements -- no speculative features. - -## 3. Build, test, lint - -Run the project's full quality pipeline in this order: - -1. Build (compile/bundle if the project has a build step) -2. Run the full test suite -- iterate on failures until green -3. Add new tests for the changed behavior -4. Run linting, formatting, and type-checking -- fix any issues - -Refer to the project's CLAUDE.md or package.json/Makefile/etc. -for the correct commands. - -## 4. Branch, commit, and push +Create the working branch before writing any code so changes are +never left uncommitted on main. - Determine the branch prefix from the issue type: `fix/` for bugs, `feat/` for features, `refactor/` for refactors, `docs/` for documentation. When ambiguous, use `fix/`. -- Create a branch named `{prefix}issue-$ISSUE_NUMBER` -- Delete the plan file (`plan-issue-$ISSUE_NUMBER.md`) -- it was a +- Create a branch named `{prefix}issue-$ISSUE_NUMBER` based on + the upstream remote's main branch (e.g. `upstream/main` if the + `upstream` remote exists, otherwise `origin/main`) + +## 4. Implement + +Implement the plan across all necessary files. Follow the +project's CLAUDE.md standards. Keep changes minimal and focused +on the issue requirements — no speculative features. + +Add tests for the changed behavior as part of implementation — +tests are code, not a quality gate. + +When stuck during implementation — a confusing error, an +unfamiliar API, or an approach that isn't working — use Exa +to search for solutions rather than spinning. + +## 5. Build, test, lint + +### 5a. Discover project checks (CI is the source of truth) + +Before running anything, read the project's CI configuration to +learn what the project *actually* runs. This takes priority over +the fallback tables below. + +1. **Read CI workflows.** Scan `.github/workflows/` for the main + CI workflow (typically `ci.yml`, `test.yml`, or `build.yml`). + Extract: + - Test commands with feature flags (e.g. + `cargo test --features foo,bar`) + - Lint/format commands with non-default flags + - Any step that runs a command then checks `git diff --exit-code` + — these are **codegen sync checks** (schema generation, + snapshot updates, help text, etc.). Record the command. + - Docs/site build commands (e.g. `make site`, `mkdocs build`) +2. **Read the Makefile** (if present). Cross-reference targets + used in CI — these are the ones that matter. +3. **Read CLAUDE.md** (if present at repo root or `.claude/`). + It may define project-specific quality gates. + +Store the discovered commands. They override the fallback table +for any overlapping step. + +### 5b. Run the quality pipeline + +Detect the project language from manifest files (`Cargo.toml` → +Rust, `pyproject.toml`/`setup.py` → Python, `package.json` → +Node/TypeScript, `go.mod` → Go). A project may use multiple +languages; run checks for each. + +Run checks in this order. For each step, use the CI-discovered +command if one was found; otherwise fall back to the default. + +1. **Build** — compile or bundle +2. **Test** — run the full test suite with the same feature flags + CI uses. Iterate on failures until green. +3. **Lint and format** — fix any issues +4. **Extended checks** — per-language extras (see fallback table) +5. **Codegen sync** — for every codegen check discovered in 5a, + run the command and verify `git diff --exit-code`. If the diff + is non-empty, the generated files are stale — regenerate and + stage them. +6. **Docs build** — if the changes touch documentation files and a + docs build command exists, run it to verify the docs compile. + +### Fallback defaults (when CI config is absent or unclear) + +**Rust** (detected by `Cargo.toml`): + +| step | command | +|--------------|------------------------------------------------| +| build | `cargo build` | +| test | `cargo test` | +| lint | `cargo clippy -- --deny warnings` | +| format | `cargo fmt --check` | +| supply chain | `cargo deny check` (if `deny.toml` exists) | +| careful | `cargo careful test` (if `cargo-careful` installed) | + +**Python** (detected by `pyproject.toml` or `setup.py`): + +| step | command | +|--------------|------------------------------------------------| +| test | `pytest -q` | +| lint | `ruff check` | +| format | `ruff format --check` | +| types | `ty check` (or `mypy` if configured) | +| supply chain | `pip-audit` | + +**Node/TypeScript** (detected by `package.json`): + +| step | command | +|--------------|------------------------------------------------| +| build | per project (`npm run build`, `tsc`, etc.) | +| test | `vitest` (or project test script) | +| lint | `oxlint` (or project lint script) | +| format | `oxfmt --check` (or project format script) | +| types | `tsc --noEmit` | +| supply chain | `pnpm audit --audit-level=moderate` | + +**Go** (detected by `go.mod`): + +| step | command | +|--------------|------------------------------------------------| +| build | `go build ./...` | +| test | `go test ./...` | +| lint | `golangci-lint run` | +| format | `gofmt -l .` | +| vet | `go vet ./...` | + +If a tool is not installed, skip it with a note rather than +failing the pipeline. + +## 6. Self-review + +For docs-only changes, do a focused manual review (verify links, +check prose accuracy, confirm rendering). For code changes, use +`/pr-review-toolkit:review-pr` to run a deep review against the +diff (compare working tree to the upstream main branch). Produce a +list of findings ranked by severity (P1 = blocks merge, +P2 = important, P3 = nice to have). + +## 7. Fix findings + +Address all P1–P3 findings. For each finding, either: + +- **Fix it** — apply the change, or +- **Dismiss it** — explain why it's a false positive or not worth + the churn (e.g. a stylistic disagreement or an impossible edge + case). Document the reasoning inline. + +After addressing all findings, review your own fixes: read the +diff of changes made in this step and verify each fix is correct, +doesn't introduce new issues, and doesn't regress other parts of +the implementation. If you spot a problem, fix it before +proceeding. + +Then re-run the full quality pipeline (build, test, lint). Iterate +until clean. + +## 8. Commit and push + +- Delete the plan file (`plan-issue-$ISSUE_NUMBER.md`) — it was a working artifact and should not be committed - Commit all changes with a conventional commit message referencing the issue - Push the branch -## 5. Create PR +## 9. Create PR Create a PR with: @@ -61,37 +211,15 @@ Create a PR with: - A description that maps changes back to the issue requirements - Link to the issue with "Closes #$ISSUE_NUMBER" (or "Refs" if it doesn't fully close it) +- If an `upstream` remote exists, submit the PR to the upstream + repo using `gh pr create --repo ` -## 6. Self-review +## 10. Comment on issue -Use `/compound-engineering:workflows:review` to perform a full -multi-agent code review of the PR. Produce a list of findings -ranked by severity (P1 = blocks merge, P2 = important, P3 = nice -to have). +Post a summary comment on Issue #$ISSUE_NUMBER in the canonical +repo (`gh issue comment $ISSUE_NUMBER --repo `) +linking to the PR. Include: -## 7. Fix findings - -Address all P1-P3 findings. For each finding, either: - -- **Fix it** -- apply the change, or -- **Dismiss it** -- explain why it's a false positive or not worth - the churn (e.g. a stylistic disagreement or an impossible edge - case). Document the reasoning inline. - -After addressing all findings: - -1. Re-run the full quality pipeline (build, test, lint) -2. Commit the fixes as a separate commit (do not squash into the - original -- preserve review history) -3. Push the branch (regular push, not force-push) -4. Delete any todo files in `todos/` that were created by the - review and are now resolved - -## 8. Comment on issue - -Post a summary comment on Issue #$ISSUE_NUMBER linking to the PR. -Include: - -- What was implemented (1-3 bullet points) +- What was implemented (1–3 bullet points) - Key design decisions - Link to the PR diff --git a/commands/merge-dependabot.md b/commands/merge-dependabot.md new file mode 100644 index 0000000..a6f6912 --- /dev/null +++ b/commands/merge-dependabot.md @@ -0,0 +1,582 @@ +# Merge Dependabot PRs + +@description Evaluate and merge dependabot PRs with parallel builds, dependency-aware batching, and transitive dep analysis. +@arguments $REPO: GitHub org/repo (e.g., trailofbits/algo). $OPTIONS: Optional flags — "--skip-config-audit" skips Phase 0 (use in batch runs where config audit is a separate pass). + +Clone $REPO if not already available locally: + +```bash +gh repo clone $REPO /tmp/depbot-eval-$(echo "$REPO" | tr '/' '-') -- --depth=50 2>/dev/null || \ + (cd /tmp/depbot-eval-$(echo "$REPO" | tr '/' '-') && git fetch origin) +``` + +Work from `/tmp/depbot-eval-{repo-slug}` for all subsequent phases. + +Execute every phase below sequentially. Do not stop or ask for +confirmation at any phase. + +## Turn Budget Management + +If you are running as a background agent with a `max_turns` cap: + +- **At 75% of turns used:** Stop launching new evaluations. Merge + any PRs already evaluated as PASS. Skip Phase 5's detailed + reports — print only the summary table. +- **At 90% of turns used:** Immediately print whatever summary you + have and stop. Do not start new evaluations or re-tests. +- **Prioritize merging over analysis.** If you must choose between + thorough analysis of the last PR and merging already-evaluated + PASS PRs, merge first. + +## Phase 0: Dependabot Config Audit + +If `$OPTIONS` includes `--skip-config-audit`, skip this entire +phase and proceed to Phase 1. + +Detect all package ecosystems present in the repo by checking for +these indicator files: + +| Indicator file(s) | Ecosystem | +|---|---| +| `pyproject.toml` + `uv.lock` | `uv` | +| `pyproject.toml` (no `uv.lock`), `requirements*.txt`, `setup.py`, `setup.cfg` | `pip` | +| `Cargo.toml` | `cargo` | +| `package.json` | `npm` | +| `go.mod` | `gomod` | +| `Gemfile` | `bundler` | +| `Dockerfile`, `docker-compose.yml` | `docker` | +| `.github/workflows/*.yml` | `github-actions` | +| `composer.json` | `composer` | +| `*.csproj`, `*.fsproj` | `nuget` | + +Read `.github/dependabot.yml`. Verify all five conditions: + +1. **Coverage** — every detected ecosystem has a corresponding + `updates` entry with the correct `package-ecosystem` value and + appropriate `directory` (usually `"/"`) +2. **uv vs pip** — if a directory has both `pyproject.toml` and + `uv.lock`, the ecosystem MUST be `uv`, not `pip`. The `pip` + ecosystem does not update `uv.lock`, which causes PRs that + modify `pyproject.toml` but leave `uv.lock` out of sync. + If any entry uses `pip` where `uv` is correct, flag it for + correction. +3. **Schedule** — every entry has `schedule.interval: "weekly"` +4. **Cooldown** — every entry has a `cooldown` block with + `default-days: 7`. This prevents dependabot from flooding the + PR queue with rapid re-attempts after a PR is closed or merged. +5. **Grouped updates** — every entry has a `groups` key with at + least one group using `patterns: ["*"]` or more specific + grouping patterns + +If the file is missing or any condition fails, create a corrective +PR: + +1. `git checkout -b fix/dependabot-config` +2. Write or update `.github/dependabot.yml`. Every `updates` entry + MUST include all four required blocks. Use this template for each + ecosystem entry: + + ```yaml + - package-ecosystem: "{ecosystem}" + directory: "/" + schedule: + interval: "weekly" + cooldown: + default-days: 7 + groups: + {ecosystem}-dependencies: + patterns: + - "*" + ``` + + When updating an existing file, preserve any extra fields already + present (labels, reviewers, open-pull-requests-limit, etc.) and + only add missing blocks. + +3. `git commit -m "chore: update dependabot config for full coverage, weekly schedule, 7-day cooldown, and grouped updates"` +4. `git push origin fix/dependabot-config` +5. `gh pr create --repo $REPO --title "Update dependabot configuration" --body "Adds missing ecosystem coverage, enforces weekly schedule, 7-day cooldown, and grouped updates."` +6. `git checkout main` + +Continue to Phase 1 regardless — this PR is non-blocking. + +## Phase 1: Discovery & Baseline + +### 1a. Fetch dependabot PRs + +```bash +gh pr list --repo $REPO --author "app/dependabot" --state open \ + --json number,title,headRefName,labels,files,mergeable +``` + +If zero PRs are returned, print "No open dependabot PRs for $REPO" +and stop. + +### 1b. Categorize PRs + +For each PR, examine its changed files: + +- **Actions dep** — all changed files are under `.github/workflows/` + or `.github/actions/` +- **Library dep** — everything else (lockfiles, manifests, version + pins, dependency specification files) + +Store the categorized list for later phases. + +### 1c. Baseline build and test + +Verify the main branch is healthy before evaluating any PR. + +1. Check out the default branch: use whatever + `gh repo view --repo $REPO --json defaultBranchRef` reports +2. Discover the build system — follow the same discovery process + described in Phase 3's subagent instructions (read CI workflows + first, then Makefile, then language-specific defaults) +3. Run the build command. If it fails, **stop the entire command** + and report: "Main branch build is broken. Fix main before + processing dependabot PRs." Include the error output. +4. Run the test command. If tests fail, **stop the entire command** + and report: "Main branch tests are failing. Fix main before + processing dependabot PRs." Include which tests fail. +5. Record the baseline: + - Full dependency tree from lockfile(s) (`pip freeze`, + `cargo tree`, `npm ls --all`, `go list -m all`, etc.) + - List of passing tests + - Build output summary + +Store the baseline data — subagents need it for comparison. + +## Phase 2: Dependency Graph Analysis + +### 2a. Build the transitive dependency map + +Parse the repo's lockfile(s) to understand the full dependency +tree: + +| Ecosystem | Lockfile | Tree command | +|---|---|---| +| uv | `uv.lock` | `uv pip freeze` (after `uv sync`) | +| pip | `poetry.lock`, `requirements*.txt` | `pip freeze` | +| cargo | `Cargo.lock` | `cargo tree` | +| npm | `package-lock.json`, `pnpm-lock.yaml` | `npm ls --all` or `pnpm ls --depth=Infinity` | +| gomod | `go.sum` | `go list -m all` | +| bundler | `Gemfile.lock` | `bundle list` | + +For each library dep PR, identify which direct dependency it bumps +(from the PR title and changed files). Look up that package in the +dependency tree to find all its transitive dependents and +dependencies. + +### 2b. Group overlapping PRs into batches + +Two PRs overlap if: +- PR A bumps package X, PR B bumps package Y, and X depends on Y + (or Y depends on X) in the transitive tree +- Both PRs modify the same lockfile section for shared transitive + dependencies + +Group overlapping PRs into **batches**. PRs with no overlaps +remain **independent** work units. + +Actions dep PRs are always independent work units — they don't +interact with library dependency trees. + +### 2c. Sort and queue + +Sort work units in topological order — leaf dependencies first, +core/shared dependencies last. This ensures earlier merges are +less likely to affect later ones. + +If there are more than 5 work units total, process in **waves +of 5**. The first wave starts immediately; subsequent waves start +after the previous wave completes. + +Print the grouping plan before proceeding: +- List each work unit (batch or independent) +- Show which PRs are in each batch and why they were grouped +- Show the evaluation order + +## Phase 3: Parallel Evaluation + +### 3a. Fetch PR branches + +For each work unit, fetch the PR branch into the local repo: + +```bash +git fetch origin pull/{number}/head:pr-{number} +``` + +Do NOT use `wt switch` — shallow clones do not support worktrees +reliably. Use `git checkout` directly when evaluating each PR. + +### 3b. Launch subagents + +Launch up to 5 subagents in parallel using the Task tool. Each +call must use: +- `subagent_type: "general-purpose"` +- The appropriate prompt below (library or actions) + +Send all Task calls in a **single message** for parallel execution. +If more than 5 work units, wait for the current wave to complete +before launching the next. + +Pass each subagent: +- The repo directory path +- The PR number(s) and title(s) +- The baseline dependency tree from Phase 1 +- The repo's build and test commands discovered in Phase 1 + +### Subagent prompt: Library Dep Evaluation + +Use this prompt for each library dep work unit. + +--- + +You are evaluating dependabot PR(s) for merge safety. Work +in the repo directory: {repo_path} + +**Repo:** $REPO +**PR(s) to evaluate:** {pr_numbers_and_titles} +**Baseline dependency tree from main:** + +``` +{baseline_dep_tree} +``` + +**Build command:** {build_command} +**Test command:** {test_command} + +Execute every step. Do not skip steps. Do not ask for confirmation. + +**STEP 1 — Checkout** + +```bash +cd {repo_path} +git fetch origin pull/{number}/head:pr-{number} +git checkout pr-{number} +``` + +For batches (multiple PRs), create a temporary merge branch: + +```bash +cd {repo_path} +git checkout -b test-batch-{batch_id} main +git fetch origin pull/{pr1}/head:pr-{pr1} +git fetch origin pull/{pr2}/head:pr-{pr2} +git merge pr-{pr1} pr-{pr2} --no-edit +``` + +If the merge has conflicts, report FAIL with the conflicting files +and stop. + +**STEP 2 — Transitive dependency analysis** + +Generate the full dependency tree using the same command that +produced the baseline. Compare against the baseline and report: + +``` +DIRECT CHANGES: + - {package}: {old_version} → {new_version} + +TRANSITIVE CHANGES: + - {package}: {old_version} → {new_version} (depended on by: {parent}) + +NEW TRANSITIVE DEPS: + - {package} {version} (pulled in by: {parent}) + +REMOVED TRANSITIVE DEPS: + - {package} {version} + +FLAGS: + - DOWNGRADE: {package} went from {higher} to {lower} + - MAJOR BUMP: {package} crossed a major version boundary +``` + +If there are zero flags and zero new/removed transitive deps, +note "Clean transitive dependency change." + +**STEP 3 — Build** + +Run the build command: {build_command} + +If the build command was not provided (blank), discover it: + +1. Read `.github/workflows/` for build steps +2. Check `Makefile` or `justfile` for a `build` target +3. Language-specific defaults: + +| Manifest | Default | +|---|---| +| `Cargo.toml` | `cargo build` | +| `pyproject.toml` | `uv pip install -e ".[dev]"` | +| `package.json` | `pnpm install && pnpm build` | +| `go.mod` | `go build ./...` | +| `Gemfile` | `bundle install` | + +If the build fails, report FAIL with exact error output and stop. + +**STEP 4 — Test** + +Run the test command: {test_command} + +If the test command was not provided (blank), discover it using the +same approach as Step 3: + +| Manifest | Default | +|---|---| +| `Cargo.toml` | `cargo test` | +| `pyproject.toml` | `pytest -q` | +| `package.json` | `pnpm test` | +| `go.mod` | `go test ./...` | +| `Gemfile` | `bundle exec rspec` or `bundle exec rake test` | + +If tests fail, check whether the same tests also fail on the main +branch baseline. Pre-existing failures do not count against this PR. + +If there are new test failures (pass on main, fail on this PR), +report FAIL with the failing test names and error output. + +**STEP 5 — Build matrix gap analysis** + +Read `.github/workflows/` for `strategy.matrix` blocks. For each +matrix dimension, report what was tested locally vs. what only +runs in CI: + +| Dimension | Example values | Testable locally? | +|---|---|---| +| OS | ubuntu, macos, windows | Current OS only | +| Language version | python 3.9-3.12 | Installed version only | +| Dependency version | numpy 1.x, 2.x | PR's version only | + +Report the matrix gaps and assess risk: +- **HIGH risk:** The dependency is known to have version-specific + behavior (e.g., numpy/scipy ABI, pytorch CUDA builds, native + extensions) and CI tests versions we couldn't test locally +- **LOW risk:** The matrix covers OS variants or formatting + differences unlikely to be affected by a dependency bump + +If there is no matrix strategy in CI, report "No CI matrix — single +configuration build." + +**STEP 6 — Verdict** + +**PASS** — all conditions met: +- Build succeeds +- All tests pass (or only pre-existing failures) +- No transitive dependency flags (downgrades, major bumps) +- No high-risk matrix gaps + +**WARN** — build and tests pass, but concerns exist: +- New transitive dependencies introduced +- Transitive dep crossed a major version boundary +- High-risk matrix gaps +- List each specific concern + +**FAIL** — any of: +- Build fails +- New test failures +- Merge conflicts + +Format the final report: + +``` +## Evaluation Report: PR #{number} — {title} + +**Verdict: {PASS|WARN|FAIL}** + +### Transitive Dependency Analysis +{step 2 output} + +### Build Result +{pass/fail with output if failed} + +### Test Result +{pass/fail with details} + +### Matrix Gap Analysis +{step 5 output} + +### Concerns +{list of concerns, or "None"} +``` + +--- + +### Subagent prompt: Actions Dep Evaluation + +Use this prompt for each GitHub Actions version bump PR. + +--- + +You are evaluating a GitHub Actions version bump for merge safety. +Work in the repo directory: {repo_path} + +**Repo:** $REPO +**PR to evaluate:** #{number} — {title} + +Execute every step. + +**STEP 1 — Checkout** + +```bash +cd {repo_path} +git fetch origin pull/{number}/head:pr-{number} +git checkout pr-{number} +``` + +**STEP 2 — Diff analysis** + +Run `git diff main -- .github/` to see what changed. Identify: +- Which action(s) were bumped +- Old and new versions (or SHA pins) +- Whether this is a patch, minor, or major version bump + +For major version bumps, use Exa (`mcp__exa__web_search_exa`) to +search for breaking changes: +`{action_name} v{old_major} to v{new_major} migration breaking changes` + +**STEP 3 — Workflow validation** + +Run: `actionlint .github/workflows/` + +If `actionlint` is not installed, note this and skip to Step 4. + +Distinguish pre-existing errors (also present on main) from new +errors introduced by the version bump. Only new errors count +against this PR. + +**STEP 4 — Pin verification** + +Check every `uses:` line in changed workflow files. Verify the +SHA-pin format: + +```yaml +# GOOD: +uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + +# BAD (tag only): +uses: actions/checkout@v4 + +# BAD (SHA without version comment): +uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 +``` + +Flag tag-only references as a WARN concern (not FAIL). + +**STEP 5 — Verdict** + +**PASS** — all conditions met: +- actionlint clean (or only pre-existing warnings) +- No breaking changes for major version bumps +- Actions are SHA-pinned with version comments + +**WARN** — concerns exist: +- Tag pin instead of SHA pin +- Major version bump with breaking changes that appear handled + +**FAIL** — any of: +- New actionlint errors +- Major version bump with unhandled breaking changes + +Format the report the same way as library dep evaluations. + +--- + +## Phase 4: Sequential Merge + +Collect all subagent evaluation reports. Process work units in the +dependency order established in Phase 2. + +### 4a. Merge passing PRs + +For each work unit with a **PASS** verdict, in order: + +1. Approve the PR: + ```bash + gh pr review --repo $REPO --approve {number} \ + --body "Automated evaluation: build, tests, and transitive dependency analysis passed." + ``` + +2. Force merge as admin: + ```bash + gh pr merge --repo $REPO --squash --admin {number} + ``` + +3. Verify the merge succeeded (gh pr merge produces no output on + success): + ```bash + gh pr view --repo $REPO {number} --json state + ``` + Confirm `state` is `"MERGED"`. If not, report the error and + skip to the next work unit. + +4. Update main locally: + ```bash + git checkout main && git pull origin main + ``` + +5. **Re-test the next work unit** before merging it. Checkout its + branch and rebase onto updated main: + ```bash + git checkout pr-{next_number} + git merge main --no-edit + ``` + Re-run the build and test commands. If the re-test fails, mark + this work unit as **SKIPPED** with reason: "Passed independent + evaluation but failed after merging prior PRs. Likely conflicts + with: {previously merged PR numbers}." Continue to the next + work unit. + +For **batched** work units, merge each PR in the batch +sequentially using the same approve-then-merge flow. + +### 4b. Handle WARN and FAIL verdicts + +- **WARN** — do not merge. Include in final report with specific + concerns. These need human review. +- **FAIL** — do not merge. Include full error context for diagnosis. + +## Phase 5: Cleanup & Report + +### 5a. Cleanup + +Return to main and delete local PR branches: + +```bash +git checkout main +git branch -D pr-{number} test-batch-{batch_id} # for each evaluated PR/batch +``` + +### 5b. Summary report + +Print a summary table: + +``` +## Dependabot PR Summary for $REPO + +| PR | Title | Type | Verdict | Action | Notes | +|----|-------|------|---------|--------|-------| +``` + +Include every evaluated PR with its verdict and outcome. + +Below the table, print totals: + +``` +**Merged:** {count} +**Skipped (WARN — needs human review):** {count} +**Failed:** {count} +**Skipped (post-merge conflict):** {count} +``` + +If a dependabot config PR was created in Phase 0: + +``` +**Dependabot config PR:** #{number} +``` + +### 5c. Detailed reports for non-merged PRs + +For each WARN, FAIL, or SKIPPED PR, print the full evaluation +report from the subagent so the user has all context needed to +decide or fix the issue. diff --git a/commands/review-pr.md b/commands/review-pr.md index 3398821..484a260 100644 --- a/commands/review-pr.md +++ b/commands/review-pr.md @@ -5,56 +5,261 @@ Read PR #$PR_NUMBER thoroughly using `gh pr view`. Understand the full context: description, linked issues, commit history, and the -diff against the base branch. Check out the PR branch locally. +diff against the base branch. + +Detect the upstream repository: if a git remote named `upstream` +exists, use it as the canonical repo. Otherwise, fall back to +`origin`. Resolve the canonical repo's `owner/name` (e.g. from +`git remote get-url upstream`) and store it — use +`--repo ` on every `gh` command to ensure they target +the correct repository. Run `git fetch ` to +ensure you are working with up-to-date code. + +Check out the PR branch locally. Execute every step below sequentially. Do not stop or ask for confirmation at any step. ## 1. Review -Use `/compound-engineering:workflows:review` to perform a full -multi-agent code review of PR #$PR_NUMBER. Produce a list of -findings ranked by severity (P1 = blocks merge, P2 = important, -P3 = nice to have, P4 = informational). +Run two review passes in parallel, then merge findings. + +### Pass A — pr-review-toolkit agents + +Launch these Task tool agents **in parallel** (single message, +multiple tool calls), each with `subagent_type` from the +pr-review-toolkit plugin. Tell each agent which files changed +(from `git diff --name-only ...HEAD`): + +| agent | focus | +|-------|-------| +| `pr-review-toolkit:code-reviewer` | Code quality, style, project guidelines | +| `pr-review-toolkit:silent-failure-hunter` | Silent failures, swallowed errors, bad fallbacks | +| `pr-review-toolkit:pr-test-analyzer` | Test coverage gaps and missing edge cases | + +### Pass B — external second opinion + +Launch these Task tool agents **in parallel with Pass A** — all +5 agents in a single message, multiple tool calls. Each uses +`subagent_type: general-purpose`. + +**Codex reviewer** — tell the agent to run: + +```bash +codex review --base / \ + -c model='"gpt-5.3-codex"' \ + -c model_reasoning_effort='"xhigh"' +``` + +- `--base` does not accept custom prompts (codex reads + `AGENTS.md` at the repo root if one exists) +- If `gpt-5.3-codex` fails with an auth error, retry with + `gpt-5.2-codex` +- Set `timeout: 600000` on the Bash call +- Tell the agent to summarize findings only — skip + `[thinking]`/`[exec]` blocks and sandbox warnings +- If `codex` is not installed, report and skip + +**Gemini reviewer** — tell the agent to run: + +```bash +git diff /...HEAD > /tmp/pr-review-diff.txt + +# Build prompt file (avoids heredoc shell expansion issues) +{ + echo "Review this diff for code quality, bugs, and improvements." + if [ -f CLAUDE.md ] || [ -f .claude/CLAUDE.md ]; then + echo "" + echo "Project conventions:" + echo "---" + cat CLAUDE.md .claude/CLAUDE.md 2>/dev/null + echo "---" + fi + echo "" + echo "Diff:" + cat /tmp/pr-review-diff.txt +} > /tmp/pr-review-prompt.txt + +# Pipe prompt via stdin to avoid shell metacharacter issues +cat /tmp/pr-review-prompt.txt | gemini -p - \ + -m gemini-3-pro-preview \ + --yolo +``` + +- Uses stdin (`-p -`) instead of heredoc to avoid shell + expansion issues with `$`, backticks, etc. in diffs +- Set `timeout: 600000` on the Bash call +- If `gemini` is not installed, report and skip + +### Merge findings + +Collect results from all 5 sources (3 toolkit agents + Codex + +Gemini). Deduplicate overlapping findings — if multiple sources +flag the same issue, keep the most specific description and note +the consensus. Rank every finding by severity: + +- **P1** — blocks merge (correctness bugs, security issues) +- **P2** — important (missing error handling, test gaps, logic flaws) +- **P3** — nice to have (style, naming, minor simplifications) +- **P4** — informational (observations, suggestions for future work) ## 2. Fix findings -Address all P1-P3 findings. For each finding, either: +Address all P1–P3 findings. For each finding, either: -- **Fix it** -- apply the change, or -- **Dismiss it** -- explain why it's a false positive or not worth +- **Fix it** — apply the change, or +- **Dismiss it** — explain why it's a false positive or not worth the churn (e.g. a stylistic disagreement or an impossible edge case). Document the reasoning inline. -P4 findings are informational -- note them but do not fix unless +When a fix requires external context — unfamiliar library behavior, +unclear API semantics, or an error you don't recognize — use Exa +(`mcp__exa__web_search_exa`) to search for solutions rather than +guessing. + +P4 findings are informational — note them but do not fix unless trivial. +After addressing all findings, review your own fixes: read the +diff of changes made in this step and verify each fix is correct, +doesn't introduce new issues, and doesn't regress other parts of +the PR. If you spot a problem, fix it before proceeding. + ## 3. Verify -Run the project's full quality pipeline: +### 3a. Discover project checks (CI is the source of truth) -1. Build (compile/bundle if the project has a build step) -2. Run the full test suite -- iterate on failures until green -3. Run linting, formatting, and type-checking -- fix any issues +Before running anything, read the project's CI configuration to +learn what the project *actually* runs. This takes priority over +the fallback tables below. -Refer to the project's CLAUDE.md or package.json/Makefile/etc. -for the correct commands. +1. **Read CI workflows.** Scan `.github/workflows/` for the main + CI workflow (typically `ci.yml`, `test.yml`, or `build.yml`). + Extract: + - Test commands with feature flags (e.g. + `cargo test --features foo,bar`) + - Lint/format commands with non-default flags + - Any step that runs a command then checks `git diff --exit-code` + — these are **codegen sync checks** (schema generation, + snapshot updates, help text, etc.). Record the command. + - Docs/site build commands (e.g. `make site`, `mkdocs build`) +2. **Read the Makefile** (if present). Cross-reference targets + used in CI — these are the ones that matter. +3. **Read CLAUDE.md** (if present at repo root or `.claude/`). + It may define project-specific quality gates. + +Store the discovered commands. They override the fallback table +for any overlapping step. + +### 3b. Run the quality pipeline + +Detect the project language from manifest files (`Cargo.toml` → +Rust, `pyproject.toml`/`setup.py` → Python, `package.json` → +Node/TypeScript, `go.mod` → Go). A project may use multiple +languages; run checks for each. + +Run checks in this order. For each step, use the CI-discovered +command if one was found; otherwise fall back to the default. + +1. **Build** — compile or bundle +2. **Test** — run the full test suite with the same feature flags + CI uses. Iterate on failures until green. +3. **Lint and format** — fix any issues +4. **Extended checks** — per-language extras (see fallback table) +5. **Codegen sync** — for every codegen check discovered in 3a, + run the command and verify `git diff --exit-code`. If the diff + is non-empty, the generated files are stale — regenerate and + stage them. +6. **Docs build** — if the PR changes documentation files and a + docs build command exists, run it to verify the docs compile. + +### Fallback defaults (when CI config is absent or unclear) + +**Rust** (detected by `Cargo.toml`): + +| step | command | +|--------------|------------------------------------------------| +| build | `cargo build` | +| test | `cargo test` | +| lint | `cargo clippy -- --deny warnings` | +| format | `cargo fmt --check` | +| supply chain | `cargo deny check` (if `deny.toml` exists) | +| careful | `cargo careful test` (if `cargo-careful` installed) | + +**Python** (detected by `pyproject.toml` or `setup.py`): + +| step | command | +|--------------|------------------------------------------------| +| test | `pytest -q` | +| lint | `ruff check` | +| format | `ruff format --check` | +| types | `ty check` (or `mypy` if configured) | +| supply chain | `pip-audit` | + +**Node/TypeScript** (detected by `package.json`): + +| step | command | +|--------------|------------------------------------------------| +| build | per project (`npm run build`, `tsc`, etc.) | +| test | `vitest` (or project test script) | +| lint | `oxlint` (or project lint script) | +| format | `oxfmt --check` (or project format script) | +| types | `tsc --noEmit` | +| supply chain | `pnpm audit --audit-level=moderate` | + +**Go** (detected by `go.mod`): + +| step | command | +|--------------|------------------------------------------------| +| build | `go build ./...` | +| test | `go test ./...` | +| lint | `golangci-lint run` | +| format | `gofmt -l .` | +| vet | `go vet ./...` | + +If a tool is not installed, skip it with a note rather than +failing the pipeline. ## 4. Commit and push - Commit the fixes as a separate commit (do not squash into the - original -- preserve review history) -- Use commit message: `fix: resolve code review findings for - PR #$PR_NUMBER` + original — preserve review history) +- Write a detailed commit message that covers: + - Subject: `fix: resolve code review findings for PR #$PR_NUMBER` + - Body: list findings by severity, what was fixed vs dismissed + (with brief reasoning), and confirmation that the quality + pipeline passes - Push the branch (regular push, not force-push) - Delete any todo files in `todos/` that were created by the review and are now resolved -## 5. Post summary +## 5. PR comment -Add a comment on PR #$PR_NUMBER summarizing what was done: +Post a review summary as a PR comment using +`gh pr comment $PR_NUMBER --repo `. -- Total findings by severity (e.g. "3 P2, 5 P3") -- How many were fixed vs dismissed (with brief reasoning for - any dismissals) -- Confirmation that the quality pipeline passes +Format the comment body as: + +``` +## Review Summary + +### Findings + +[For each severity level that has findings, list them as a table:] + +| # | Severity | Finding | Resolution | +|---|----------|---------|------------| +| 1 | P1 | [description] | Fixed: [what was done] | +| 2 | P2 | [description] | Dismissed: [reasoning] | +| ... | ... | ... | ... | + +### Verification + +- **Tests**: [pass/fail count] +- **Lint**: [clean/issues] +- **Format**: [clean/issues] + +### Commit + +[commit SHA and subject line] +```