Copilot now reports how many review comments get acted on. Read the number carefully
GitHub's usage metrics API has an addressed rate for Copilot code review. A script to compute it per repository, severity and tier, and what the figure can't tell you.
GitHub added a code review section to the repository-level Copilot usage metrics on 5 November 2026. Every repos-1-day row now carries a copilot_code_review object that counts how many of Copilot’s review comments were evaluated and how many of those the developer acted on. Divide the second by the first and you get an addressed rate.
That’s useful, because the existing metric, total_copilot_applied_suggestions, counts only ready-to-apply suggestions that someone applied. It’s also a number that’s easy to over-read. This piece covers what’s in the data, a script that turns it into a table, and four reasons the headline rate can’t be treated as a quality score.
(This is about measuring reviews. For requesting one from a script, see the earlier piece on the review API .)
What the new fields contain
Per GitHub’s changelog and the reference docs, the object has these members:
total_comments_evaluated: Copilot review comments checked for whether the developer acted on them.total_comments_addressed: the evaluated comments that were acted on.total_merged_no_human_change_request_after_review: merged pull requests that Copilot reviewed where nobody requested changes after Copilot’s first review.- Four arrays,
review_comments_by_severity,review_comments_by_review_tier,review_comments_by_comment_typeandreview_comments_by_customization. Each element has one key (severity,review_tier,comment_typeorcustomization) plus the same two counters.
Severity is high, medium or low. Review tier is lite, balanced or max. Comment types include bug, security, performance, accessibility, documentation and custom, the last being your repository’s own coding guidelines. Customization is custom_instruction, skill, mcp_server or none.
The reports come from GET /orgs/{org}/copilot/metrics/reports/repos-1-day?day=YYYY-MM-DD (there’s an enterprise equivalent). The response is a report_day and a list of download_links. Access needs an enterprise or organisation owner, a billing manager, or a custom role with the View Copilot Metrics permission, and the usage metrics policy has to be on.
A script that turns it into a table
The docs don’t say what file format sits behind the download links, so the script below assumes newline-delimited JSON, one repository row per line, and fails loudly if that’s wrong. Download the day’s files first, then pass them in.
import json, sys
from collections import defaultdict
def rate(addressed, evaluated):
return f"{addressed / evaluated:.0%}" if evaluated else "n/a"
repos = defaultdict(lambda: [0, 0])
severity = defaultdict(lambda: [0, 0])
tiers = defaultdict(lambda: [0, 0])
merged_clean = 0
for path in sys.argv[1:]:
for line in open(path):
row = json.loads(line)
cr = row.get("copilot_code_review") or {}
ev = cr.get("total_comments_evaluated", 0)
ad = cr.get("total_comments_addressed", 0)
repos[row.get("repository", "?")][0] += ad
repos[row.get("repository", "?")][1] += ev
merged_clean += cr.get("total_merged_no_human_change_request_after_review", 0)
for e in cr.get("review_comments_by_severity", []):
severity[e["severity"]][0] += e["total_comments_addressed"]
severity[e["severity"]][1] += e["total_comments_evaluated"]
for e in cr.get("review_comments_by_review_tier", []):
tiers[e["review_tier"]][0] += e["total_comments_addressed"]
tiers[e["review_tier"]][1] += e["total_comments_evaluated"]
def show(title, table):
print(title)
for key, (ad, ev) in sorted(table.items()):
print(f" {key:<14}{ad:>4}/{ev:<4} {rate(ad, ev)}")
show("by repository", repos)
show("by severity", severity)
show("by review tier", tiers)
print(f"merged with no change request after review: {merged_clean}")
Two details matter. The script sums counts across days and repositories and only divides at the end. Averaging the daily percentages would let a quiet Sunday with two comments weigh the same as a busy Thursday with sixty. And the repository key name (repository here) is a guess, since the docs excerpt doesn’t show an example row, so check it against a real file.
The invented sample rows below only show the shape of the output. They aren’t real data.
by repository
acme/api 7/10 70%
acme/web 8/40 20%
by severity
high 3/5 60%
low 7/33 21%
medium 5/12 42%
by review tier
balanced 7/10 70%
lite 8/40 20%
merged with no change request after review: 12
Four reasons the rate isn’t a quality score
First, “addressed” is generous. GitHub counts applying the suggestion, asking Copilot to make the fix, editing the code, and replying that the fix will arrive in a later pull request. The last one is a deferral. A team that answers every nit with “follow-up” can post a healthy rate while changing nothing.
Second, the denominator only exists at merge. Comments are evaluated when their pull request merges and count on the merge day, not the day Copilot posted them. A pull request that’s abandoned, or sits open for three weeks, contributes nothing. Comments on bot-opened pull requests (the Copilot cloud agent’s, for instance) are usually not evaluated at all, so the figure describes human-authored work, which is probably what you want but isn’t the whole picture of agent output.
Third, the breakdowns don’t have to add up. GitHub says the severity, tier and comment-type arrays can sum to less than the totals, and a comment can appear under more than one customization entry. Summing the customization array and comparing it with the total is a mistake. Compare rates within one array, and treat a gap between an array’s sum and the total as unclassified comments, not missing data.
Fourth, and this is the one that matters most, acted-on is not the same as right. A low addressed rate can mean noisy comments. It can equally mean good comments that a rushed team ignored. A high rate can mean useful catches, or a review that mostly flags what developers were going to change anyway. Nothing in the object tells those apart.
What to do with it
The severity split is the most informative cut. If high-severity comments are addressed at a rate well above low-severity ones, as in the sample, the reviewer is at least ranking its own findings in an order developers agree with. If high and low sit at the same rate, the severity label isn’t adding information, and it’s worth reading a sample of the high ones by hand.
The tier and customization cuts answer the question teams actually have to decide: is the heavier setting, or the extra instructions, paying for itself? Compare lite against max on the same repository over the same weeks, and none against custom_instruction. The sample size is the constraint. A repository with 40 evaluated comments a week gives rates that move by several points on a handful of pull requests, so pool at least a month before reading anything into a difference.
Pair the rate with something the metric can’t fake. Sample 20 evaluated comments that were not addressed and mark each as noise or as a real miss. If most are noise, the low rate is a tuning problem for your review instructions. If several are real defects that merged anyway, the rate is a process problem, and no setting on the reviewer will fix it.
Then there’s total_merged_no_human_change_request_after_review. It’s a count of pull requests where nobody pushed back after Copilot’s first pass, and it can’t separate “the review was thorough and the code was fine” from “nobody read it”. Track it as a trend and never as a target, because a target turns it into pressure not to request changes.
Start with one repository: download a week of repos-1-day files, run the script, and read the unaddressed high-severity comments before deciding what the rate means for your team.