Upgrade to Vitest 5 without letting the agent weaken your tests to get green
Vitest 5 changes what some passing tests mean, and 'make the suite pass' invites the cheapest fix. A baseline, a test-only rule and a short gate script keep the upgrade honest. About 20 minutes on one project.
Vitest 5 turns the “unawaited assertion” warning into a failure, defaults clearMocks to true, and changes how -t filters match. Most teams will see a handful of red tests after bumping the version and hand them to an agent with the obvious instruction: make the suite pass. There are four cheap ways to do that without fixing anything. Flip clearMocks back to false. Mark the test skip. Loosen an assertion. Edit the production code until the test stops complaining.
All four produce a green run. None of them is an upgrade.
The release (3 September 2026) is a good case for this because its breaking changes are not API renames the compiler catches. They’re changes in what a passing test means, and the places where a test suite quietly depended on the old behaviour are exactly the places a green-at-any-cost fix hides. The setup below takes about 20 minutes: record a baseline on the old version, upgrade with no other changes, let the agent fix tests only, and run a small script that fails the upgrade if the suite got weaker. It works with any coding agent that can read files and run a shell, because every check is a command on the repository.
What actually changed in meaning
The migration guide lists dozens of changes. Most are config renames or browser-mode details. Five alter what an existing node-side suite asserts:
clearMocksis nowtrue, so Vitest callsvi.clearAllMocks()before every test. Call history resets between tests; implementations stay. A suite that accumulated calls on a module-level mock now sees them reset.resolves,rejectsandtoMatchFileSnapshotthat aren’t awaited fail the test. In v4 they were awaited automatically with a warning.-tjoins the describe and test names with>. A filter such as"checkout charges"that worked as a space-separated phrase no longer matches.toThrow('')matches any error, because an empty string is now a substring of every message.expect.pollrejects on timeout instead of settling quietly, and fake timers now mockTemporalwhen it exists.
Add that titles are rendered with pretty-format, so snapshots that embed test names can change, and that vi.mock, vi.unmock and vi.hoisted must now be top level. Vitest 5 also needs Node.js 22.12.0 or later and Vite 6.4.0 or later.
Every item on that list can make a previously passing test fail, and some can make a previously failing assertion pass. The second kind is the one a green run never tells you about.
The demo project
A small checkout module and a suite written the way suites often are. It’s TypeScript with Vitest 4.1.12, and the code is deliberately plain so the failures are easy to read.
// src/checkout.ts
export interface Mailer {
send(to: string, subject: string): Promise<void>;
}
export class PaymentDeclined extends Error {}
export async function charge(cents: number): Promise<string> {
if (cents <= 0) throw new RangeError("amount must be positive");
if (cents > 500_000) throw new PaymentDeclined("over limit");
return `ch_${cents}`;
}
export async function checkout(
mailer: Mailer,
email: string,
cents: number,
): Promise<string> {
const id = await charge(cents);
await mailer.send(email, "Receipt");
return id;
}
export function parseQuantity(input: string): number {
const n = Number(input);
if (!Number.isInteger(n) || n < 1) throw new Error(`bad quantity: ${input}`);
return n;
}
// test/checkout.test.ts
import { describe, expect, it, vi } from "vitest";
import { charge, checkout, parseQuantity, PaymentDeclined } from "../src/checkout";
const send = vi.fn(async () => {});
const mailer = { send };
describe("checkout", () => {
it("charges and sends one receipt", async () => {
await checkout(mailer, "ann@example.com", 1000);
expect(send).toHaveBeenCalledTimes(1);
});
it("sends a receipt per order", async () => {
await checkout(mailer, "bo@example.com", 2000);
expect(send).toHaveBeenCalledTimes(2);
});
it("rejects a zero amount", () => {
expect(charge(0)).rejects.toThrow("positive");
});
it("declines over the limit", async () => {
await expect(charge(600_000)).rejects.toBeInstanceOf(PaymentDeclined);
});
it("rejects a bad quantity", () => {
expect(() => parseQuantity("abc")).toThrow("bad quantity");
});
});
Two tests carry the habits the upgrade targets. The second one only passes because the first one already called the shared mock once, so it asserts a count of 2. The third forgets the await on rejects.
Step 1: record a baseline on the old version
On a branch, before touching package.json, capture two things: the per-test results and a mutation score.
npx vitest run --reporter=json --outputFile=baseline.json
The JSON reporter writes one entry per test, with its full name and status. That’s the list of behaviours the suite claims to check, and it’s what the gate compares against later.
The mutation score is the second anchor. StrykerJS changes the production code in small ways (flips a condition, empties a string) and reruns the tests. A mutant the suite doesn’t notice is a behaviour nothing pins down. A config for the Vitest runner:
{
"testRunner": "vitest",
"mutate": ["src/**/*.ts"],
"reporters": ["clear-text"],
"coverageAnalysis": "perTest"
}
Run npx stryker run with @stryker-mutator/core and @stryker-mutator/vitest-runner installed. On Vitest 4.1.12 it prints, trimmed:
-------------|------------------|----------|-----------|------------|----------|----------|
| % Mutation score | | | | | |
File | total | covered | # killed | # timeout | # survived | # no cov | # errors |
-------------|--------|---------|----------|-----------|------------|----------|----------|
All files | 65.22 | 75.00 | 15 | 0 | 5 | 3 | 0 |
checkout.ts | 65.22 | 75.00 | 15 | 0 | 5 | 3 | 0 |
-------------|--------|---------|----------|-----------|------------|----------|----------|
65.22 is a modest score. That’s fine: the number matters as a floor, not as a goal. Commit the config and note the score in the branch description.
Step 2: upgrade and change nothing else
npm i -D vitest@5.1.0
npx vitest run
Don’t touch a test yet. The failures are the work list, and they’re more useful read raw than summarised. Trimmed:
❯ test/checkout.test.ts (5 tests | 2 failed) 11ms
❯ checkout (5)
× sends a receipt per order 3ms
× rejects a zero amount 3ms
FAIL test/checkout.test.ts > checkout > sends a receipt per order
AssertionError: expected "vi.fn()" to be called 2 times, but got 1 times
FAIL test/checkout.test.ts > checkout > rejects a zero amount
Error: Promise returned by `expect(actual).rejects.toThrow(expected)` was not awaited.
Two failures, one per habit. The first was never a test of the second order at all: it passed because of the first test’s leftover call, and it fails when run alone on Vitest 4 as well (-t "per order" reports one call, not two). The second was an assertion racing the end of the test.
Both are mechanical to fix, which is the point of the next step: the agent should fix these and nothing else.
Step 3: give the agent a narrow job
The prompt (a recommendation, not observed agent behaviour):
The repo was upgraded from Vitest 4 to Vitest 5.1.0. The failing tests are
listed by `npx vitest run`. Fix the tests, and only the tests.
Rules:
- Do not edit anything under src/.
- Do not add skip, todo or only. Do not delete or rename a test.
- Do not add or change vitest.config.* to restore old behaviour
(clearMocks: false, for example). Fix the test so it doesn't depend on it.
- Do not pass --passWithNoTests or change the test script.
- For each failure, say in one sentence which Vitest 5 change caused it
(cite the migration guide section) and what the test was relying on.
- When done, run `python3 upgrade-gate.py baseline.json current.json HEAD`
after writing current.json with the JSON reporter, and report its output.
Two parts of that do the work. The rules close the cheap exits in words the agent can follow. The last bullet asks for the cause of each failure, which turns a mechanical fix into a finding: “this test depended on a call left by the previous test” is a defect report on the suite, not just a patch.
Rules in a prompt are a request, though, not a guarantee. The gate is what enforces them.
Step 4: the gate
About fifty lines of Python. It compares per-test status against the baseline, then scans the diff for the cheap exits. Python is only glue; the app and tests stay TypeScript.
#!/usr/bin/env python3
"""Fail if a test-runner upgrade made the suite weaker.
usage: upgrade-gate.py BASELINE.json CURRENT.json BASE_REF
"""
import json, re, subprocess, sys
def results(path):
out = {}
for f in json.load(open(path))["testResults"]:
for t in f["assertionResults"]:
out[t["fullName"]] = t["status"]
return out
base, cur = results(sys.argv[1]), results(sys.argv[2])
problems = []
for name in sorted(base):
if name not in cur:
problems.append(f"test disappeared: {name}")
elif base[name] == "passed" and cur[name] != "passed":
problems.append(f"{cur[name]}: {name}")
diff = subprocess.run(["git", "diff", "-U0", sys.argv[3]], capture_output=True,
text=True, check=True).stdout
new = subprocess.run(["git", "ls-files", "--others", "--exclude-standard"],
capture_output=True, text=True, check=True).stdout.split()
for path in new:
if path.endswith((".ts", ".js", ".json", ".mts")):
for line in open(path, errors="replace"):
diff += "+" + line
touched = re.findall(r"^\+\+\+ b/(src/\S+)", diff, re.M)
for p in touched:
problems.append(f"production file changed: {p}")
weakeners = r"\.(skip|todo)\b|\bxit\b|\bxdescribe\b|clearMocks:\s*false|--passWithNoTests"
for line in diff.splitlines():
if line.startswith("+") and not line.startswith("+++") and re.search(weakeners, line):
problems.append(f"added line weakens the suite: {line[1:].strip()}")
print(f"baseline {len(base)} tests, current {len(cur)} tests")
for p in problems:
print("FAIL", p)
sys.exit(1 if problems else 0)
The new-files loop matters. git diff ignores untracked files, so a freshly created vitest.config.ts with clearMocks: false would slip past a gate that only read the diff. Reading untracked source and config files as added lines closes that.
With the two honest fixes applied (a fresh expectation of one call, and the missing await), the gate passes:
// the two edits in test/checkout.test.ts
it("sends a receipt per order", async () => {
await checkout(mailer, "bo@example.com", 2000);
expect(send).toHaveBeenCalledTimes(1);
});
it("rejects a zero amount", async () => {
await expect(charge(0)).rejects.toThrow("positive");
});
$ npx vitest run --reporter=json --outputFile=current.json
$ python3 upgrade-gate.py baseline.json current.json HEAD
baseline 5 tests, current 5 tests
$ echo $?
0
Now the lazy version: skip the rejects test, set clearMocks: false in a new config file and soften a production check (cents <= 0 becomes cents < 0). The Vitest run on that tree exits 0. The gate does not:
baseline 5 tests, current 5 tests
FAIL skipped: checkout rejects a zero amount
FAIL production file changed: src/checkout.ts
FAIL added line weakens the suite: it.skip("rejects a zero amount", async () => {
FAIL added line weakens the suite: export default defineConfig({ test: { clearMocks: false } });
Four findings, and the suite was green. That’s the case for running a check the agent can’t talk its way past.
Step 5: rerun the mutation score
After the honest fixes, the same Stryker config on Vitest 5.1.0 prints the same table: 65.22 total, 15 killed, 5 survived, 3 no coverage. The upgrade changed the runner and kept every behaviour the suite could kill.
That equality is the acceptance test. A lower score would mean a fix removed an assertion; a higher one is welcome but not required. Stryker 10.1.1 with @stryker-mutator/vitest-runner 10.1.1 works against Vitest 5.1.0 with the config above. Don’t assume that holds for every version pair: confirm the baseline run is green before trusting the comparison.
The filter that skips everything
One change doesn’t show up as a failure at all. Scripts that run a subset with -t are common in CI and in agent prompts (“run only the checkout tests”).
On Vitest 4.1.12, npx vitest run -t "checkout charges" runs one test and skips four. On 5.1.0 the same command reports Tests 5 skipped (5), because the pattern spans the describe and test names and the separator is now >. The exit code is 0.
A green exit with nothing run is the worst result a filter can give. Search your scripts, CI config and agent instructions for -t , and rewrite the multi-word ones as "checkout > charges" (which runs the one test on 5.1.0), a single-segment pattern or a regex. For a gate, assert the number of tests that ran, which is what comparing against baseline.json does.
Make it yours
The gate doesn’t know about Vitest except through the JSON shape. Any runner that can write per-test names and statuses to a file works with a different results() function: the same idea applies to a .NET test run emitting a results file. Keep the baseline step and the diff scan as they are.
In CI, run the gate as a required check on the upgrade branch, and fetch baseline.json from the commit before the version bump. For a monorepo, run it per package, since a clean score in one package says nothing about the next.
If the suite is big, a full mutation run on every upgrade is slow. Restrict mutate to the modules whose tests the agent touched, using the file list from git diff --name-only. A partial score is still comparable to a partial baseline.
What goes wrong
The gate compares names and statuses, not meaning. An agent can rewrite a test’s body to assert less while keeping its name and status, and the gate will pass it. The mutation score catches that only when the weakened assertion was the one killing a mutant. On a thin suite like the demo (65%), plenty of weakening hides among the survivors, so read the diff of the test files yourself.
The weakener patterns are text matches. describe.skipIf, a conditional early return in a test or a renamed helper won’t match. Treat the list as a net for the common cheats and add to it when review finds a new one.
Browser mode, vitest-browser-* packages and the screenshot and locator changes in the migration guide were not exercised here. The same baseline-and-gate shape applies, but the strict-locator change (browser.locators.exact now defaults to true) is a place where a test can start failing for a good reason and a quick fix can hide a real regression.
And don’t bother with all of this for a ten-test project the upgrade finishes in one commit. Read the diff. The setup earns its time on suites big enough that nobody reads every changed line.
First 15 minutes
Create a branch and run npx vitest run --reporter=json --outputFile=baseline.json on your current version. Install the Stryker packages, add the config above, run it, and write the score in the branch description. Then bump Vitest, run the suite untouched, and read the failures against the five meaning changes above before anyone, human or agent, edits a test.