Check an agent's architecture summary against the import graph before you trust it
Three small scripts turn an AI-written tour of a codebase into claims a machine can check. Fifteen minutes to set up, any agent.
Day one in a repo you didn’t write, and the agent’s tour reads beautifully. “The entry point wires up a core client, which delegates to utility modules and throws typed errors.” Every sentence sounds right. Some of them aren’t, and nothing in the prose tells you which.
The fix is to stop reading the tour as prose. Ask the agent to write its architecture notes as a list of one-line claims about which file imports which, then run a script that checks every claim against the real import graph. Wrong claims show up as FAIL lines in seconds, and the notes that survive are ones you can lean on.
Below are three scripts (about 110 lines of Python in total), a prompt that gets claims in a checkable form, and a worked run against a real open-source library. The technique needs an agent that can read files and run a shell, and nothing else, so it works the same in Claude Code, Codex, Cursor, Copilot or a plain chat window with the files pasted in. The graph builder here handles TypeScript; the idea carries to any language where imports are greppable.
Why a summary needs a check
An agent summarising a codebase is compressing thousands of lines it read partially, in an order it chose. It fills gaps with what’s typical for the kind of project it thinks it’s looking at. A file named retry-timing.ts gets described as if it were used by everything that retries, because that’s the usual shape.
The mistakes are almost never wild. They’re plausible: a dependency that points the wrong way, a module described as shared that has exactly one caller, a “circular dependency” that the compiler erases before it ever runs. Those are the details that change how you edit the code, and they’re the ones a skim won’t catch.
Imports are a good thing to check because they’re facts. A file either imports another or doesn’t, and a 40-line script can answer that for the whole tree. Everything else the agent says about design (why, intent, risk) stays opinion, but the structural claims underneath it can be made falsifiable.
Step 1: build the graph
The target here is ky 2.2.0, a small MIT-licensed HTTP client written in TypeScript (the repository head was commit 3541888, 9 October 2026). Clone any TypeScript repo you like. Save this as depgraph.py:
#!/usr/bin/env python3
"""Build the internal import graph of a TypeScript source tree. Usage: depgraph.py SRC_DIR [--verbatim] > graph.json"""
import json, re, sys
from pathlib import Path
verbatim = "--verbatim" in sys.argv # verbatimModuleSyntax keeps `import {type A} from` as a runtime import
root = Path(sys.argv[1]).resolve()
files = {p.relative_to(root).as_posix(): p for p in root.rglob("*.ts") if not p.name.endswith(".d.ts")}
# import ... from './x.js' / export ... from './x.js' (multi-line safe)
pattern = re.compile(r"""(?:^|\n)\s*(import|export)\s+(type\s+)?([^;'"]*?)\s*from\s*['"](\.[^'"]+)['"]""", re.S)
def resolve(src, spec):
base = (Path(src).parent / spec).as_posix()
base = re.sub(r"\.js$", "", base)
parts = []
for seg in base.split("/"):
if seg == "..": parts.pop()
elif seg != ".": parts.append(seg)
cand = "/".join(parts)
for c in (cand + ".ts", cand + "/index.ts"):
if c in files: return c
return None
def all_type_only(clause):
names = re.findall(r"[{,]\s*(type\s+)?[\w$]+", clause)
return bool(names) and all(n.strip() for n in names)
edges = {}
for rel, p in sorted(files.items()):
text = p.read_text(encoding="utf-8")
for kind, tkw, clause, spec in pattern.findall(text):
dst = resolve(rel, spec)
if not dst: continue
type_only = bool(tkw) or ("{" in clause and all_type_only(clause) and not verbatim)
e = edges.setdefault((rel, dst), {"value": False})
if not type_only: e["value"] = True
out = {"files": sorted(files), "edges": [{"from": a, "to": b, "value": v["value"]} for (a, b), v in sorted(edges.items())]}
json.dump(out, sys.stdout, indent=1)
Run python3 depgraph.py ky/source > graph.json. Each edge records whether the import survives compilation (value: true) or is erased as type-only. That split matters, and it’s the reason this is a regex script and not a grep import: a diagram that mixes the two will show cycles that don’t exist at runtime.
The script is deliberately small. It ignores require, dynamic import() and path aliases from tsconfig, so on a repo that uses those, extend resolve first or reach for a real tool. dependency-cruiser is the established one for JavaScript and TypeScript; since v16.0.0 it detects inline import { type X } as type-only and ships an example rule, no-circular-at-runtime, that only flags cycles made entirely of runtime imports. Read its configuration before you adopt it, but for a first-day check, 40 lines you can read beat a config you can’t.
Step 2: look at the shape yourself
Before any claim gets checked, print the shape. Save report.py:
#!/usr/bin/env python3
"""Summarise graph.json: fan-in, fan-out, folder-to-folder edges, cycles, entry points."""
import json, sys
from collections import defaultdict
g = json.load(open(sys.argv[1]))
if "--value-only" in sys.argv: g["edges"] = [e for e in g["edges"] if e["value"]]
out, inn = defaultdict(set), defaultdict(set)
for e in g["edges"]:
out[e["from"]].add(e["to"]); inn[e["to"]].add(e["from"])
print("file in out")
for f in sorted(g["files"], key=lambda f: (-len(inn[f]), f)):
print(f"{f:<30}{len(inn[f]):>3}{len(out[f]):>5}")
folders = defaultdict(int)
d = lambda p: p.rsplit("/", 1)[0] if "/" in p else "."
for e in g["edges"]:
if d(e["from"]) != d(e["to"]): folders[(d(e["from"]), d(e["to"]))] += 1
print("\nfolder -> folder (edge count)")
for (a, b), n in sorted(folders.items()): print(f" {a} -> {b}: {n}")
# cycles (Tarjan)
idx, low, stack, on, sccs, c = {}, {}, [], set(), [], [0]
def sc(v):
idx[v] = low[v] = c[0]; c[0] += 1; stack.append(v); on.add(v)
for w in out[v]:
if w not in idx: sc(w); low[v] = min(low[v], low[w])
elif w in on: low[v] = min(low[v], idx[w])
if low[v] == idx[v]:
comp = []
while True:
w = stack.pop(); on.discard(w); comp.append(w)
if w == v: break
if len(comp) > 1: sccs.append(sorted(comp))
for f in g["files"]:
if f not in idx: sc(f)
print("\ncycles:", sccs or "none")
print("no importers:", [f for f in g["files"] if not inn[f]])
Run it twice, once on everything and once with --value-only. The top of the runtime-only table for ky looks like this (trimmed to the first rows):
file in out
core/constants.ts 6 0
errors/KyError.ts 6 0
utils/is.ts 6 0
errors/TimeoutError.ts 4 1
errors/ForceRetryError.ts 3 3
errors/HTTPError.ts 3 1
Fan-in is where to look first. The files at the top (constants, the base error class, a type-guard helper) are the ones where a careless edit breaks the most other files, so they’re also the ones whose description in the agent’s tour should be precise.
The folder table is where the first surprise turns up:
folder -> folder (edge count)
. -> core: 2
. -> errors: 7
. -> utils: 2
core -> errors: 6
core -> utils: 9
errors -> utils: 2
utils -> core: 4
utils -> errors: 7
core imports from utils nine times, and utils imports from core four times. So at the folder level there’s no clean “core on top of utils” layering, whatever a summary says. Both directions exist at runtime. (At file level there’s no cycle, which the last lines confirm.)
Now the cycle check, with and without type-only imports:
# all imports, types included
cycles: [['core/Ky.ts', 'core/constants.ts', 'errors/ForceRetryError.ts', ... 16 files]]
# runtime imports only (--value-only)
cycles: none
Counting type imports, 16 files sit in one strongly connected cluster. Counting only what survives compilation, there are none. An agent that reports “circular dependencies between the types and the core” is describing something true in the type system and harmless at runtime. An agent that reports “no cycles” is also half right. Neither is a claim to repeat in a design doc until it says which graph it means.
Step 3: turn the tour into claims
Now the prompt. Paste this into whichever agent you use, from the repo root:
Read the code under source/ and write architecture notes as a claims file, not prose.
One claim per line, using only these forms and repo-relative paths under source/:
imports A B A has a runtime import of B (not type-only)
imports-type A B A imports B in any way, including type-only
never-imports A B A has no import of B at all
importers B A1 A2 ... the runtime importers of B are exactly these files
leaf A A imports nothing from this project
Write 20 to 30 claims about the structure you consider most important for someone
about to change this code. Do not run any import-analysis tool. After the list, add a
short paragraph on anything structural you could not express in these forms.
Three details in that prompt carry the weight. Fixed forms mean a machine can read the answer. “Do not run any analysis tool” stops the agent from simply querying the graph and handing the check a perfect score, which would test nothing. And the closing paragraph keeps the claims that don’t fit (design intent, naming conventions) visible as unchecked opinion instead of letting them vanish.
The claims file for a real run would come from the agent. The example below is hand-written in the same format, with two lines made wrong on purpose so you can see what a failure looks like. Save it as claims.txt and save the checker as checkclaims.py:
#!/usr/bin/env python3
"""Check architecture claims against graph.json. Usage: checkclaims.py graph.json claims.txt
Claim forms (one per line, # comments allowed):
imports A B A has a runtime (value) import of B
imports-type A B A has any import of B, including type-only
never-imports A B A has no import of B, value or type
importers B A1 A2 ... the runtime importers of B are exactly these files
leaf A A imports nothing from the project
"""
import json, sys
g = json.load(open(sys.argv[1]))
files = set(g["files"])
val = {(e["from"], e["to"]) for e in g["edges"] if e["value"]}
anyi = {(e["from"], e["to"]) for e in g["edges"]}
bad = 0
for n, line in enumerate(open(sys.argv[2]), 1):
line = line.split("#")[0].strip()
if not line: continue
op, *a = line.split()
missing = [x for x in a if x not in files]
if missing:
print(f"line {n}: FAIL {line}\n no such file: {missing}"); bad += 1; continue
if op == "imports": ok, why = (a[0], a[1]) in val, ""
elif op == "imports-type": ok, why = (a[0], a[1]) in anyi, ""
elif op == "never-imports": ok, why = (a[0], a[1]) not in anyi, ""
elif op == "leaf": ok, why = not any(f == a[0] for f, _ in anyi), ""
elif op == "importers":
actual = sorted(f for f, t in val if t == a[0]); ok = actual == sorted(a[1:])
why = f"actual: {actual}"
else: ok, why = False, "unknown claim form"
print(f"line {n}: {'ok ' if ok else 'FAIL'} {line}" + (f"\n {why}" if not ok and why else ""))
bad += not ok
sys.exit(1 if bad else 0)
# Claims lifted from an architecture summary of source/
imports index.ts core/Ky.ts
imports index.ts utils/merge.ts
imports-type types/options.ts types/hooks.ts
leaf errors/KyError.ts
never-imports utils/is.ts core/Ky.ts
importers core/retry-timing.ts core/Ky.ts
importers utils/delay.ts core/Ky.ts
never-imports errors/HTTPError.ts utils/body.ts
imports core/Ky.ts errors/HTTPError.ts
imports utils/merge.ts types/options.ts
imports core/Ky.ts utils/hooks.ts
Run python3 checkclaims.py graph.json claims.txt and it prints:
line 2: ok imports index.ts core/Ky.ts
line 3: ok imports index.ts utils/merge.ts
line 4: ok imports-type types/options.ts types/hooks.ts
line 5: ok leaf errors/KyError.ts
line 6: ok never-imports utils/is.ts core/Ky.ts
line 7: ok importers core/retry-timing.ts core/Ky.ts
line 8: ok importers utils/delay.ts core/Ky.ts
line 9: ok never-imports errors/HTTPError.ts utils/body.ts
line 10: ok imports core/Ky.ts errors/HTTPError.ts
line 11: FAIL imports utils/merge.ts types/options.ts
line 12: FAIL imports core/Ky.ts utils/hooks.ts
no such file: ['utils/hooks.ts']
The exit code is 1 when anything fails, so the script slots into a pre-commit step or a CI job if you keep the claims file in the repo.
Look at the two failures, because they’re the two kinds you’ll meet. Line 11 is a half-truth: utils/merge.ts does mention types/options.ts, but only through import type, which disappears at compile time. In a prose summary “merge depends on the options types” reads as correct. As a runtime claim it’s false, and the difference decides whether you can move the file without touching the module’s load order. Line 12 is a file that doesn’t exist: a plausible name for a plausible helper. Prose can’t be caught doing that. A path in a claim can.
Step 4: loop the failures back
Send the FAIL lines back to the agent with the corrected facts, and ask it to rewrite only the notes that depended on those claims:
These claims failed against the real import graph (output below). Revise the architecture
notes where they relied on them. Keep every claim that passed. For each claim you change,
say what you had assumed.
<paste the FAIL lines>
The last sentence is the useful part. “I assumed utils/hooks.ts existed because hook merging looked like a utility” tells you how the agent builds its model, and which of its other beliefs probably rest on the same habit. Do one round of this. A second round mostly tests how well the agent repairs the checker’s complaints, not how well it understands the code.
When the claims file passes, commit it next to the notes, and the notes now come with a test. After refactors, rerun the checker: a failing line tells you that a described dependency has changed, which is the one signal that stops architecture docs rotting quietly.
The compiler setting that changes the graph
The value-only graph rests on one rule: type-only imports are erased. That depends on a TypeScript setting. The same source compiled with TypeScript 5.9.3 shows the difference. With verbatimModuleSyntax off, the compiled JavaScript’s relative imports between files match the 52 runtime edges from depgraph.py exactly. With verbatimModuleSyntax on, the compiled output has seven more: an import like import {type Options} from './options.js' is kept as import {} from './options.js', which still loads the module.
The --verbatim flag in depgraph.py models that. Run python3 depgraph.py ky/source --verbatim > graph.json for a project that sets the option (check its tsconfig.json and anything it extends) and the runtime graph gets those seven edges, so “no importers” shrinks and a claim like importers lists more files. The runtime cycle check on ky stays at none either way. If you aren’t sure which mode a repo uses, run tsc once into a temporary folder and compare the import lines in the output with the graph. That comparison is how the numbers above were checked, and it takes a minute.
Make it yours
For a C# solution, swap the graph builder and keep the other two scripts. Project references come from dotnet list <project> reference, and namespace-level using edges can be pulled with a Roslyn analyzer or even a regex over using lines. Write them out as the same files and edges JSON and the claim checker works unchanged.
For a monorepo, claim at package level. Read each package.json, treat internal dependencies as edges, and the same forms (imports, never-imports, importers) check statements like “nothing outside apps/ depends on apps/web”.
To make the check a standing guard, add a rule to your agent’s context file: structural claims in design notes go in claims.txt and must pass checkclaims.py. The rule is one sentence and works as plain text for any agent that reads a context file.
What goes wrong
The graph only covers what the regex can see. Dynamic imports, barrel files that re-export with export *, path aliases and generated code are all invisible to a 40-line script, so a claim can fail because the checker is blind, not because the agent is wrong. When a FAIL looks suspicious, open the file and read the import line before you correct the notes.
Imports aren’t behaviour. Two files with no import between them can still be coupled through a shared global, a message bus or a config key. A passing claims file means the agent’s picture of the dependencies is right. It says nothing about whether its picture of what the code does is right, so keep reading the code that matters.
The agent can also game it. Given the checker, a model will happily write only claims it’s sure of, which is why the prompt asks for the ones that matter for a change and forbids running analysis tools. Treat the closing paragraph of unchecked remarks as a to-do list for your own reading.
Don’t bother with any of this on a repo under a few dozen files. You can read the imports faster than you can set up the check.
First 15 minutes
Pick a repo you need to learn. Run depgraph.py on its source folder, then report.py with and without --value-only, and read the folder table and the top of the fan-in list before the agent says anything. Then paste the claims prompt into your agent of choice, save its answer as claims.txt, and run checkclaims.py. Every FAIL line is a thing the tour got wrong that you’d otherwise have found by breaking something.