How to review agent-written pull requests

Reviewing human code, you learn where the danger is. The hasty commit at 6pm. The function that grew for two years. The place someone left a comment saying they were not sure about this.

Agent code has none of those tells. It is uniformly tidy, consistently formatted, and confident throughout. The failure modes moved somewhere else, and if your review habits did not move with them you will approve things you should not.

One: check scope before you read the code

The single highest-signal check. Read the task, then read the file list. Ask whether every changed file needed to change.

Agents interpret goals. Asked to fix a date formatting bug, an agent may also normalize timezone handling in three other modules, because it reads that as the real problem. Sometimes it is right. It is still a different change than the one that was reviewed, tested, and scheduled.

Two: verify the APIs exist

Agents produce plausible library calls. A method with the right name and the wrong signature, a config option that was removed two versions ago, a helper that exists in a different package entirely.

Types and tests catch most of this, which is exactly why it survives in the places they do not reach: dynamic config, string-keyed lookups, infrastructure definitions, and SQL. When I see an unfamiliar call in an agent diff, I look it up. Every time. It takes fifteen seconds and it has caught real problems repeatedly.

Three: read the tests as adversarially as the code

This is where I find the most trouble. Four patterns recur:

• Assertions that restate the implementation. The test computes the expected value the same way the code does, so it passes for any behavior.
• Over-mocking. Every dependency stubbed, leaving a test that verifies the mocks were configured correctly.
• Deleted or weakened assertions. The failing check was the problem, so the check went away. Always read removed lines in test files.
• Happy path only. No empty input, no null, no error branch, no boundary.

The fix that scales better than vigilance: require a failing test first for bug fixes, and reject any pull request that reduces assertion count without an explicit reason in the description.

Four: hunt for silent behavior changes

Agents refactor while working, and refactors change behavior at the edges. The specific things worth grepping the diff for:

1. Default values that changed.
2. Error handling turned into a swallowed exception or a default return.
3. Null and empty handling that became a truthiness check.
4. Ordering assumptions introduced or removed.
5. Timeouts, retries, and limits adjusted. These are load-bearing and they are edited casually.

The diff replaced a try block with an optional chain. Cleaner, shorter, and it turned a loud failure into a silent empty result.

Five: security review does not get skipped

Agents reproduce patterns from wherever they learned them, including the insecure ones. Specifically look for string-built SQL, permission checks performed in the client or in a prompt rather than server-side, secrets moved into code or config, expanded CORS, and validation that got relaxed to make a test pass.

For anything touching agent tools or credentials, the standards in prompt injection is not a prompt problem apply to the diff too. Capability changes deserve more scrutiny than logic changes.

Six: make the description do work

Require a template and treat a missing section as a review blocker: what was asked, what changed and why, what was verified and how, what was deliberately left alone, where the reviewer should look first.

A vague or evasive description correlates strongly with a diff that will need rework. It is the cheapest signal available and it takes ten seconds to read.

The checklist, in order

1. Does the diff match the task, and nothing more.
2. Do every new API call and option actually exist.
3. Would the tests fail if the behavior were wrong. Were any assertions removed.
4. Any changed defaults, error paths, null handling, ordering, timeouts, or limits.
5. Any new query building, permission logic, secret, or relaxed validation.
6. Is the diff small enough to have actually been read. If not, send it back to be split.

The structural fixes that beat vigilance

Reviewer attention is finite, so most of the win comes from constraints in the pipeline rather than from trying harder:

• Hard diff size ceiling per agent job type.
• Path allowlists so certain directories are simply off limits.
• Required deterministic checks including tests, types, lint, and your eval suite when behavior code changes.
• Merge-rate tracking per task type, so unproductive categories get retired instead of endlessly tuned.

The full pipeline setup, including sandboxing and token scopes, is in putting AI coding agents in CI. The tooling comparison for what generates these diffs in the first place is in Claude Code vs Cursor.

Agents have genuinely raised my throughput. They raised my review standards at the same time, and that is not a contradiction. It is the price of the speed, and it is worth paying. If the prototype on the other side of that review still needs hardening, the path is in from vibe coding to a production system.

Frequently asked