Debugging When the AI Is Out of Ideas

At 11:20 UTC on 18 November 2025, a large part of the internet started returning errors. Cloudflare, which sits in front of millions of websites, was failing.

The strange part was the pattern. The network would fail, then recover, then fail again, roughly every five minutes. That’s not how a normal bug behaves. It looked like an attack. And then Cloudflare’s status page, which is hosted completely outside Cloudflare’s own systems, went down too. Some of the team, very reasonably, concluded that someone was attacking both.

They were wrong. Cloudflare’s own postmortem says the outage “was not caused, directly or indirectly, by a cyber attack.” The status page going down was a coincidence.

The real cause was a database permissions change made at 11:05. It caused a query to return duplicate rows, which roughly doubled the size of a configuration file used by Cloudflare’s bot detection. The proxy software had a hard limit of 200 entries for that file (it normally used about 60). When the oversized file arrived, the Rust code hit an unwrap() on an error value and crashed.

And the five-minute rhythm? The file was rebuilt every five minutes by a query on a database cluster that was being updated one part at a time. If the query ran on an updated part, you got a bad file. If not, a good one. So the network flickered between broken and fine until every part was updated, and then it stayed broken.

For the first stretch of the incident, the team focused on a different service that looked like the cause. Main traffic was restored at 14:30, after they stopped the bad file from being generated and pushed out a known-good one. Everything was back by 17:06.

This is a team of very good engineers with excellent tools. The symptoms still pointed them in the wrong direction for a while. That’s what debugging is like. The error tells you where it hurts, not where the problem is.

Now imagine handling that by pasting the error message into a chat window.

The doom loop

You know the loop, because everyone who has used an AI coding tool has been in it.

Something breaks. You paste the error into the AI.
The AI says “I see the issue!” and changes some code.
A different error appears. You paste that one.
The AI changes more code. The first error comes back.
Forty minutes later, you have twelve changes you don’t understand, the original bug, and two new ones.

Karpathy described this honestly in the post that named vibe coding. When the AI couldn’t fix a bug, he would “work around it or ask for random changes until it goes away.” That’s fine for a weekend toy. It’s a disaster for anything with users.

Why the AI gets stuck

It’s not that the AI is bad at debugging. It’s that you’ve given it the worst possible input.

An error message is a symptom, not a cause. The AI only sees where it hurts. It can’t see your data, your environment variables, your database or what happened five minutes earlier.
It’s built to answer. Ask “why is this broken?” and you’ll get a confident reason, even with too little information to know.
Each guess changes the code. Now the bug you’re chasing is in a slightly different program than the one that failed. Guesses stack up.
You stop building a picture. You’re relaying messages instead of thinking. When the AI runs out of ideas, so do you.

There’s evidence for that last point. In Anthropic’s January 2026 study of developers learning a new library, the researchers grouped people by how they used the AI. The group that relied on the AI to debug repeatedly scored lowest on the follow-up quiz, about 24%, compared with 67% for the group that coded by hand. The groups were small (four people in that one), so treat it as a signal, not a law. But it matches what I see: people who debug by relay don’t get better at debugging.

And in Stack Overflow’s 2025 survey, 45% of developers said debugging AI-generated code takes more time. The fix isn’t to stop using AI. It’s to stop using it as a slot machine.

The method: reproduce, minimise, hypothesise, test

Debugging is the scientific method applied to a program. It sounds grand. In practice it’s four steps, repeated.

The doom loop changes code to make the error go away. The debugging loop changes your understanding until the cause is obvious.

1. Reproduce

You can’t fix what you can’t make happen. Before anything else, find exact steps that trigger the bug every time, and ideally turn them into a script or a test.

“Sometimes the total is wrong” is not a reproduction. “Run node scripts/weekly-report.ts with the data from last Monday and the total is $0.00” is.

If you can’t reproduce it, that’s information too. It usually means the bug depends on something you haven’t noticed yet: specific data, timing, a time zone, an environment variable, which server handled the request. (Cloudflare’s “which part of the cluster ran the query” is a perfect example.)

2. Minimise

Now make the reproduction as small as possible. Remove code, data and steps one at a time. Does it still fail? Keep removing.

The goal is the smallest program that still shows the bug. Two things happen along the way:

Often, removing one particular thing makes the bug vanish. That thing is your best clue.
The smaller the program, the fewer places the bug can hide. A 15-line reproduction is often its own answer.

3. Hypothesise

Write down one specific guess about the cause. Not “something’s wrong with the database.” Something you could prove wrong:

“The total is 0 because the loop finishes after the function has already returned.”

Writing it down matters. It stops you from changing three things at once, and it gives you something concrete to test.

4. Test

Design one check that would disprove the guess if it’s wrong. Add a log line, set a breakpoint, change one input. Then run it.

If the guess is wrong, cross it off and go back to step 3 with what you learned. If it survives, fix the cause, then add a test that would have caught it.

A good habit: keep a short debug log in a scratch file. Each line: what you guessed, what you checked, what happened. When you get stuck, reading it back often shows the assumption you never tested.

Reading a stack trace

When a JavaScript program crashes, it prints a stack trace: the chain of function calls that led to the crash, most recent first. Here’s a real one from Node.js (paths shortened):

TypeError: Cannot read properties of undefined (reading 'discountPercent')
    at summary (file:///app/src/services/orders.ts:6:67)
    at file:///app/src/services/orders.ts:7:1
    at ModuleJob.run (node:internal/modules/esm/module_job:345:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:665:26)

How to read it:

The first line is the error type and message. Read it slowly. Cannot read properties of undefined (reading 'discountPercent') does not mean discountPercent is missing. It means the thing before the dot is undefined. In customer.discountPercent, it’s customer that’s empty.
Find the first frame in your own code. Skip anything starting with node:internal or living in node_modules. Here it’s orders.ts, line 6, column 67.
Go there, and then ask “why?” The crash line is where the problem became visible, not always where it started. Why was customer undefined? Was the order loaded without its customer? Was the customer deleted? That’s your hypothesis.

For async code, traces can be shorter than you’d like, because the original caller may be gone by the time the error is thrown. That’s when logging earns its keep.

Logging vs a debugger

Both are ways of seeing what the program is actually doing, instead of what you think it’s doing.

Logging is adding print statements. It’s fast, works everywhere (including production), and is great for timing and order problems. Make your logs easy to find and read:

console.log("[weeklyTotal] day=%s orders=%d", day, orders.length);
console.log("[applyDiscount] input", JSON.stringify({ amountCents, discount }));

The label in brackets lets you search the output. JSON.stringify shows you the exact value, including whether "10" is a string or 10 is a number. Remove debugging logs when you’re done, or turn them into proper structured logs if they’d help next time.

A debugger pauses the program on a line so you can inspect every variable, step forward one line at a time, and see the call stack. It’s better when you don’t know what to log yet, or when the state is complex.

node --inspect-brk src/scripts/weekly-report.ts
# then open chrome://inspect in Chrome, or attach VS Code's debugger

Or just add debugger; on a line and run under VS Code’s “JavaScript Debug Terminal.” Learn conditional breakpoints too (“pause only when order.id === 'abc'”). They’re how you catch the one bad record among ten thousand.

My rule of thumb: logs for “in what order did things happen?”, a debugger for “what exactly is in this object right now?”

Binary search with git bisect

Sometimes you don’t know what’s wrong, but you know when it was right. It worked in last month’s release. It’s broken now. There are 200 commits in between.

You don’t need to read 200 commits. You need about 8, because each check cuts the range in half. git bisect does this for you.

git bisect start
git bisect bad                 # the current commit is broken
git bisect good v1.4.0         # this tag was fine
# git checks out a commit halfway between. Test it, then tell git:
git bisect good                # or: git bisect bad
# ...repeat until git prints "<hash> is the first bad commit"
git bisect reset               # go back to where you started

Even better, if you have a command that fails when the bug is present, git can run the whole search on its own:

git bisect start HEAD v1.4.0
git bisect run node --test src/reports/weekly.test.ts

git bisect run treats exit code 0 as “good,” most other codes from 1 to 127 as “bad,” and 125 as “can’t test this commit, skip it.” When it finishes, it prints the commit that introduced the bug, with its message and the files it touched.

This is the most underused tool in git. It turns “I have no idea” into “this exact change, by this person, on this day,” which is a much better question to bring to a teammate or an AI. It also works the same way for code you didn’t write, which matters more and more.

Cloudflare’s recovery followed the same logic at a bigger scale. They found what had changed (a newly generated file), went back to the last version known to be good, and the network recovered.

Rubber ducking (and the AI as a duck)

“Rubber duck debugging” was popularised by The Pragmatic Programmer (Andrew Hunt and David Thomas, 1999): explain your code, line by line, out loud, to a rubber duck on your desk. Halfway through explaining what a line “obviously” does, you notice it doesn’t.

It works because explaining forces you to state assumptions you’d otherwise skip.

An AI makes a surprisingly good duck, if you tell it not to fix anything:

I’m debugging a problem and I want to explain it to you. Don’t suggest fixes yet. After I explain, ask me up to five questions about things I haven’t checked or assumptions I’m making.

You do the explaining. It does the poking. You stay the one building the picture.

When to bring the AI back in: give it a minimal repro

None of this means “debug without AI.” It means give the AI what a good senior engineer would need. After you’ve reproduced and minimised, your prompt looks completely different:

Goal: weeklyTotal() should return the sum of all orders for the given days. Expected: 4000 for the input below. Actual: 0. Minimal repro (runs with node repro.ts, Node 22): (15 lines of code) Already ruled out: the data. fetchOrders returns 2 orders per day; I logged it. Observation: the [return] log prints before the [loop] logs. Question: why does the function return before the loop finishes?

This prompt has a clear expected-vs-actual, code the AI can actually run in its head, what you’ve already eliminated, and the clue you found. With this, almost any current model will give you the right answer on the first try. And you’ll understand the answer, because you did the work that makes it make sense.

A worked example: the $0.00 weekly report

This one is fictional, but it’s one of the most common async bugs in JavaScript, and AI tools write it regularly.

After a refactor, the weekly revenue email says $0.00. Nothing crashes. No error to paste.

Reproduce. Run the report script locally against last week’s data. Total: 0. Good, it’s reproducible.

Bisect. The report was right in last month’s release. git bisect run with the report’s test finds the first bad commit: “refactor: compute weekly total from daily queries.”

Minimise. Strip the commit’s code down to the core, with the database replaced by a fake function:

// repro.ts (run with: node repro.ts, Node 22+)
type Order = { id: string; amountCents: number };

async function fetchOrders(day: string): Promise<Order[]> {
  return [{ id: `${day}-1`, amountCents: 1200 }, { id: `${day}-2`, amountCents: 800 }];
}

async function weeklyTotal(days: string[]): Promise<number> {
  let total = 0;
  days.forEach(async (day) => {
    const orders = await fetchOrders(day);
    console.log("[loop]", day, "orders:", orders.length);
    for (const o of orders) total += o.amountCents;
  });
  console.log("[return] total:", total);
  return total;
}

console.log("result:", await weeklyTotal(["mon", "tue"]));

Still returns 0. The bug survived minimising, with no database involved. That rules out a whole category of guesses.

Hypothesis 1: fetchOrders returns nothing. Test: the [loop] log. It prints orders: 2 for each day. Disproved.

Hypothesis 2: the function returns before the loop runs. Test: look at the order of the logs:

[return] total: 0
[loop] mon orders: 2
[loop] tue orders: 2
result: 0

[return] prints first. Hypothesis survives.

Cause. forEach calls each callback but ignores what it returns. An async callback returns a promise, so forEach starts each one and moves straight on. The function returns total while it’s still 0, and the additions happen afterwards, to a number nobody reads.

Fix. Use something that waits. If the days can be fetched in parallel:

async function weeklyTotal(days: string[]): Promise<number> {
  const perDay = await Promise.all(
    days.map(async (day) => {
      const orders = await fetchOrders(day);
      return orders.reduce((sum, o) => sum + o.amountCents, 0);
    }),
  );
  return perDay.reduce((sum, t) => sum + t, 0);
}

Or, if they should run one at a time, a plain for...of loop, where await really does pause:

async function weeklyTotal(days: string[]): Promise<number> {
  let total = 0;
  for (const day of days) {
    const orders = await fetchOrders(day);
    for (const o of orders) total += o.amountCents;
  }
  return total;
}

Both return 4000. Then add a test with known data that checks the total, so git bisect run has something to use next time. A linter rule such as typescript-eslint’s no-misused-promises can catch this pattern before it ships.

Notice what didn’t help: there was no error message. The doom loop has nothing to paste. The method didn’t need one.

A debugging practice routine (with AI)

Timebox the AI. Give it two tries on a bug. If the second fix doesn’t work, stop and switch to the method. Don’t let it make change number three on top of two failed guesses. Undo them first (git stash or git checkout .).
Reproduce before you prompt. Never ask the AI to fix something you can’t trigger on demand.
Write your hypothesis before you read the AI’s. Then compare. Over time, yours will get better.
Bisect once, on purpose. Next time something breaks between releases, use git bisect even if you think you know the cause. It’s a skill you want ready before the crisis.
Read one postmortem a month. Cloudflare, GitHub and others publish detailed write-ups. Each one is a free lesson in how experienced teams reason when symptoms mislead.
Ask “why” one more time. After the fix, ask: why did this happen, and why didn’t anything catch it? The second answer usually leads to a test or a check that prevents the next one.

Checklist: before you ask the AI to fix a bug

I can reproduce it on demand, with exact steps or a script.
I’ve shrunk the reproduction as far as I reasonably can.
I know what I expected and what actually happened.
I’ve read the stack trace and found the first frame in my own code.
I’ve written down at least one hypothesis and how I tested it.
If it used to work, I’ve tried git bisect.
I’ve undone any earlier failed “fixes” so I’m debugging the original code.

Key ideas

The error shows where it hurts, not where the problem is. Even Cloudflare’s engineers first suspected an attack.
Pasting errors back is a slot machine. It works for small bugs and fails exactly when it matters.
Debugging is a loop: reproduce, minimise, hypothesise, test. Change your understanding, not just the code.
Read stack traces from the top, then find the first frame in your own code, then ask why.
Logs for order and timing, debuggers for state.
git bisect finds the breaking commit in a handful of steps, even in code you didn’t write.
Bring the AI back with a minimal repro, expected vs actual, and what you’ve ruled out. Then it becomes genuinely good at this.

Sources: Cloudflare, “Cloudflare outage on November 18, 2025” (postmortem). Andrej Karpathy’s vibe coding post on X (2 February 2025). Anthropic, “How AI assistance impacts the formation of coding skills” (29 January 2026). Stack Overflow, 2025 Developer Survey, AI section. Git documentation, git bisect. Node.js docs, Debugging guide. typescript-eslint, no-misused-promises. Andrew Hunt and David Thomas, The Pragmatic Programmer (1999).