# Evaluate an AI reviewer with evidence

AntiCode field guide · September 19, 2026

## Code review metrics: measure what happened, not what sounds good

### Write the definition beside the number

A metric becomes useful when two people can reproduce it from the same records. Reviewed pull requests sounds straightforward until one person includes stale reviews, another counts every run, and a third counts only merged changes. Before interpreting a chart, define the event, the population, and the time boundary.

Use a sentence precise enough to expose ambiguity: current reviewed pull requests divided by selected pull requests created during the period. That measures something different from review runs completed during the period. Both can be useful, but switching between them without changing the label makes comparisons unreliable.


### Separate activity from improvement

More review runs may mean broader adoption. They may also mean retries, changing revisions, or an unreliable workflow. More findings may mean better detection, riskier changes, or more noise. Those counts describe recorded activity; they do not establish that software quality improved.

Use a second source of evidence for the outcome you care about. If the question is whether reviews help developers find defects, inspect a sample of findings and the resulting fixes. If the question is whether work moves faster, examine the relevant workflow timestamps and delays. Do not infer either answer from the number of comments alone.


### Keep personal decisions in their proper scope

An acceptance rate needs a denominator and a definition of accepted. In AntiCode, triage decisions are personal. An accepted finding means a particular user considered it relevant; it does not establish that the code was fixed or that the entire team agreed. A high rate may reflect useful results, selective review, or individual habits.

Report how many findings were considered and whether undecided findings remain in the denominator. A rate based on a handful of selected cases should not be presented as a general benchmark. Use the metric to choose questions for investigation, then inspect the cases that produced it.

- State whether the measure counts findings, pull requests, people, or runs.
- Keep the same repository and date selection when comparing periods.
- Distinguish current results from historical or stale reviews.
- Show missing measurements as unavailable instead of inventing a zero.

### Do not rename elapsed time as time saved

A review duration measures how long that recorded process took. It does not measure how long a human would otherwise have spent. A pull request's creation-to-merge interval also includes waiting, discussion, unrelated work, and release decisions. Neither value alone establishes a productivity gain.

DORA defines change lead time around the path from version control to production deployment. That is a different boundary from pull-request time to merge. If your dashboard lacks deployment records, do not relabel its review or merge timestamps as a delivery metric. Use the name that matches the actual events.


### Build a report someone can challenge

AntiCode presents recorded workspace activity and filtered exports, while metrics it cannot derive remain unavailable. Before sharing a report, set the repository, author, date, and status filters deliberately. Include the scope and any visible data limits so another reader can understand what the chart represents.

For a useful weekly discussion, pair one clearly defined metric with a small set of examples. A rising rerun count might lead you to inspect stale revisions or large diffs. A low acceptance rate might lead you to inspect speculative warnings. The report should make those questions easier to ask, not close the conversation with an unsupported claim about engineering performance.


References:
- DORA: Software delivery performance metrics: https://dora.dev/guides/dora-metrics/