tokenise by Rosvetic
Development How-to

Codex says it's done. A Stop hook makes it prove it by running your tests first

A short Stop hook that runs the test suite when Codex tries to finish, feeds the failure back as the next prompt, and gives up after three tries. Ten minutes to wire in.

Part of the hooks reading guide

8 min read

A cut-paper arched gate standing open in stacked dark layers, with a teal glow and a winding path behind it

Codex finishes a task, writes a confident summary and stops. The summary says the change is complete. Nothing in it says whether the tests still pass, because nothing made Codex check. A Stop hook does: it runs when Codex tries to end its turn, and it can refuse.

The hook below runs the project’s test command whenever the working tree has changed. If the tests fail, Codex doesn’t stop. The failure output becomes its next prompt, and it carries on fixing. If the tests pass, the turn ends as normal, and a second pass over an unchanged tree costs nothing.

What the Stop event does

Stop is one of the per-turn events in the Codex hooks documentation . It fires when Codex is about to end a turn. A hook can answer with {"decision": "block", "reason": "..."}, and per the docs that doesn’t reject the turn. Codex builds a new continuation prompt from the reason and treats it like a new user prompt.

So the reason string is the whole interface. Whatever the hook writes there is what Codex reads next, which is why the script below puts the tail of the test output in it.

Three details from the docs shape the design:

  • Plain text on stdout is invalid for Stop. Exit 0 with JSON, or exit 2 with the reason on stderr.
  • The input carries stop_hook_active, a boolean meaning this turn was already continued by a Stop hook.
  • No loop limit is documented. A hook that blocks on every failure can keep Codex retrying a test it can’t fix, so the hook needs its own cap.

The Stop matcher is ignored, so the config has no matcher line.

The demo project

A small TypeScript module with a Vitest test is enough to see it work:

// src/price.ts
export function total(items: { cents: number; qty: number }[]): number {
  return items.reduce((sum, i) => sum + i.cents * i.qty, 0);
}

// src/price.test.ts
import { expect, test } from "vitest";
import { total } from "./price";

test("sums line totals", () => {
  expect(total([{ cents: 250, qty: 2 }, { cents: 100, qty: 1 }])).toBe(600);
});

test("empty basket is zero", () => {
  expect(total([])).toBe(0);
});

package.json has "test": "vitest run", and .gitignore lists node_modules and .codex/state. The state directory is where the hook keeps its counters.

The hook script

Save this as .codex/hooks/stop_gate.py. It’s a helper script, so Python keeps it dependency-free.

#!/usr/bin/env python3
"""Codex Stop hook: if the working tree changed, run the tests before Codex may finish."""
import hashlib, json, os, subprocess, sys

MAX_ATTEMPTS = 3
TEST_CMD = os.environ.get("STOP_GATE_CMD", "npm test --silent")
TAIL_LINES = 40

payload = json.load(sys.stdin)
cwd = payload["cwd"]
state_dir = os.path.join(cwd, ".codex", "state")
state_file = os.path.join(state_dir, payload["session_id"] + ".json")


def git(*args):
    return subprocess.run(["git", *args], cwd=cwd, capture_output=True, text=True).stdout


status = git("status", "--porcelain")
if not status.strip():
    sys.exit(0)  # nothing changed this session, nothing to verify

fingerprint = hashlib.sha256((status + git("diff", "HEAD")).encode()).hexdigest()
try:
    state = json.load(open(state_file))
except (OSError, ValueError):
    state = {}

if state.get("passed") == fingerprint:
    sys.exit(0)  # same tree already verified on an earlier turn

if state.get("attempts", 0) >= MAX_ATTEMPTS:
    print(json.dumps({"systemMessage": f"Stop gate: tests still failing after {MAX_ATTEMPTS} attempts, giving up."}))
    sys.exit(0)

result = subprocess.run(TEST_CMD, shell=True, cwd=cwd, capture_output=True, text=True, timeout=300)
os.makedirs(state_dir, exist_ok=True)

if result.returncode == 0:
    json.dump({"passed": fingerprint, "attempts": 0}, open(state_file, "w"))
    sys.exit(0)

state["attempts"] = state.get("attempts", 0) + 1
json.dump(state, open(state_file, "w"))
tail = "\n".join((result.stdout + result.stderr).strip().splitlines()[-TAIL_LINES:])
reason = (
    f"`{TEST_CMD}` failed (attempt {state['attempts']} of {MAX_ATTEMPTS}). "
    "Fix the code or the test, whichever is wrong, and do not delete or skip tests. Output tail:\n" + tail
)
print(json.dumps({"decision": "block", "reason": reason}))

Four decisions in there are worth explaining.

A clean git status means nothing changed, so the hook exits silently. A question-and-answer turn that touched no files never pays for a test run.

The fingerprint is a hash of git status plus git diff HEAD. Stop fires on every turn, and without the fingerprint a passing suite would re-run after each follow-up question. With it, a tree that already passed is skipped.

The attempt counter lives in .codex/state/<session_id>.json. After three failed attempts the hook stops blocking and emits a systemMessage instead, so a test Codex can’t fix doesn’t become an endless loop. A pass resets the counter.

The “do not delete or skip tests” sentence is deliberate. Told only that the suite failed, an agent under pressure can take the cheapest route to green. Saying so in the continuation prompt doesn’t prevent it, but it removes the excuse that nobody mentioned it.

Wire it in

Add this to <repo>/.codex/config.toml:

[[hooks.Stop]]

[[hooks.Stop.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/stop_gate.py"'
timeout = 330
statusMessage = "Running tests before finishing"

The command resolves the script from the git root, because the docs say hooks run with the session’s working directory and repo-local paths should be anchored to the root. The timeout is in seconds and sits just above the script’s own 300-second test limit, so the hook’s timeout never cuts off the test run first.

Then the step people miss. Non-managed hooks don’t run until they’re trusted, and trust is tied to the hook’s hash. Start Codex, open /hooks, review the new Stop hook and trust it. Edit the config later and the hash changes, so the hook goes back to being skipped until it’s trusted again. The project’s .codex/ layer also has to be trusted for repo-level hooks to load at all.

What it does

The script can be exercised without Codex by piping in the JSON that the docs say a Stop hook receives. On a clean tree it exits silently. After a harmless edit with passing tests it exits silently too, and writes the fingerprint:

== change, tests pass
exit=0
{"passed": "1abcb82abb9784b05cba3409b57ac8da98493de5ad9450c27bc306b2bfdfba9b", "attempts": 0}

A second call on the same tree returns in about 40 milliseconds, because the fingerprint matches and the tests don’t run.

Now break the code. Dropping the quantity from the multiplication (i.cents instead of i.cents * i.qty) makes the hook print a block decision. The reason field, trimmed, reads:

`npm test --silent` failed (attempt 1 of 3). Fix the code or the test, whichever is wrong, and do not delete or skip tests. Output tail:
 ❯ src/price.test.ts (2 tests | 1 failed) 9ms
   × sums line totals 8ms
...
 FAIL  src/price.test.ts > sums line totals
AssertionError: expected 350 to be 600 // Object.is equality

- Expected
+ Received

- 600
+ 350

That text, assertion and all, is what Codex receives as its next prompt. On the fourth call with the tree still broken, the output changes to the give-up message:

Stop gate: tests still failing after 3 attempts, giving up.

Everything above was run against the script directly. What wasn’t observed is a live Codex session, since no Codex binary was available. The behaviour of Codex itself (continuing after a block, treating the reason as a prompt) is taken from the documentation. Run one real session on a throwaway branch before trusting it: break a test on purpose and watch whether Codex keeps going.

Make it fit your stack

The test command is the only thing to change. STOP_GATE_CMD overrides the default, so a .NET repo can use dotnet test --nologo -v q, and a monorepo can run only the affected project. For a slow suite, point it at a fast subset (unit tests, a type check) and leave the full run to CI. A gate that takes four minutes per turn gets disabled within a week.

Format and lint checks can join the same command: STOP_GATE_CMD="npm run lint && npm test --silent". The continuation prompt then names whichever step failed.

To get one more safety net against a bad run, add a second hook on PreToolUse that blocks edits to test files. The force-push guard shows the shape of that hook and what happens when it breaks.

Where it goes wrong

The gate only knows what the tests know. If a bug has no test, the hook passes and Codex stops. It’s a check on regressions in covered code, not on whether the change is right. Pair it with the habit of asking for a failing test first, as in the stop-condition approach for /goal .

A hook can fail without blocking. The Codex docs describe hooks as a guardrail, and an error or timeout in the script may let the turn end. A crash from a missing python3 or an unreadable payload ends in a normal stop, with the tests never run. Check that python3 is on the PATH of the machine that runs Codex.

Per-turn cost is real. The first stop after any edit runs the whole command, and every further edit changes the fingerprint and triggers a re-run. A repo with a two-minute suite and an agent that stops every few minutes adds up.

The docs don’t mention codex exec, so don’t assume a scripted run honours the hook. Test it there separately before relying on it in CI. For unattended work the codex exec guide covers the options that matter.

Finally, skip the whole thing for repos with no reliable tests. A flaky suite plus a hook that blocks on failure gives Codex three chances to chase a failure it didn’t cause.

The first ten minutes

Copy the script into .codex/hooks/, add the TOML block, open /hooks and trust the hook. Then ask Codex to change something in a covered file, and watch for the “Running tests before finishing” status message when the gate fires. Break a test on purpose once, so the first time you see the block happen isn’t on a real task.