Chapter 06
Debugging with AI
Using AI to find causes rather than paper over symptoms: reproduce first, give evidence, work through hypotheses, fix the root cause with a regression test and keep using the real debugger.
Reproduce before you ask
Also called repro, make it fail
- What
- Finding the exact steps, input and environment that make the bug happen every time, before asking anyone, human or AI, to fix it.
- Why
- Without a reliable repro nobody can prove a fix worked. With one, it becomes a failing test.
Watch out
Pasting "it doesn't work" and a screenshot, and getting a confident guess back.
Minimal reproduction
Also called MRE, minimal reproducible example
- What
- The smallest piece of code and data that still shows the bug, with everything unrelated removed.
- Why
- Shrinking the problem often reveals the cause on its own, and a small repro fits in the context window without noise.
Hypotheses, not fixes
Also called root cause analysis, scientific debugging
- What
- Asking for a ranked list of possible causes and how to test each one, then running the tests, instead of asking for a fix straight away.
- Why
- Each hypothesis is checked with evidence: a log line, a breakpoint, a changed input. That finds the real cause and teaches you the system.
Here is the failing test and the full error.
List 3 possible causes, most likely first, and how
to confirm each. Do not change code yet.Watch out
Accepting the first fix that makes the error message go away without knowing why it appeared.
Paste the evidence
Also called stack trace, logs, error output
- What
- Giving the tool the full error, stack trace, logs, input and what you already tried, instead of a summary in your words.
- Why
- The details you think are irrelevant often hold the answer. Complete evidence saves several rounds of guessing.
@terminal why did the last command fail?
/fix # propose a fix for selected code
#selection # attach the failing codeSymptom patches
Also called band-aid fix, swallowed exception
- What
- Changes that hide a bug instead of fixing it: an empty catch block, a null check that skips the work, a retry loop, a disabled lint rule or a longer timeout.
- Why
- AI tools reach for these because they make the error disappear. The bug is still there and now fails silently somewhere else.
- Example
- A test fails with "undefined is not a function". The patch adds
if (fn)before the call. The real bug is an import that returns undefined.
Root cause and regression test
Also called regression test, five whys
- What
- Fixing the cause you proved, then keeping the failing test from your repro so the same bug cannot return unnoticed.
- Why
- A bug fixed without a test tends to come back. The test also documents what went wrong for the next person.
Watch out
Declaring it fixed because the symptom stopped: if you did not find the cause, it is not fixed.
Use the real debugger
Also called breakpoints, step through, print debugging
- What
- Setting breakpoints, stepping through code and inspecting variables in your editor's debugger, or adding targeted log lines, to see what the program actually does.
- Why
- The model predicts what code probably does; the debugger shows what it does. When they disagree, the debugger is right.
Watch out
Going back and forth with the tool for an hour when five minutes with a breakpoint would have shown the wrong value.
Go deeper
- Debugging: The 9 Indispensable RulesPaid · booksefism.com/resources/debugging-9-indispensable-rules
- How to Ask Good Questions (Julia Evans)Free · guidesefism.com/resources/julia-evans-asking-good-questions
- The Missing Semester of Your CS EducationFree · coursesefism.com/resources/the-missing-semester-of-your-cs-education