Chapter 06
Debugging with AI
Using AI to find causes rather than paper over symptoms: reproduce first, give evidence, work through hypotheses, fix the root cause with a regression test and keep using the real debugger.
Reproduce before you ask
Also called repro, make it fail
- What
- Finding the exact steps, input and environment that make the bug happen every time, before asking anyone, human or AI, to fix it.
- Why
- Without a reliable repro nobody can prove a fix worked. With one, it becomes a failing test.
Watch out
Pasting "it doesn't work" and a screenshot, and getting a confident guess back.
Minimal reproduction
Also called MRE, minimal reproducible example
- What
- The smallest piece of code and data that still shows the bug, with everything unrelated removed.
- Why
- Shrinking the problem often reveals the cause on its own, and a small repro fits in the context window without noise.
Hypotheses, not fixes
Also called root cause analysis, scientific debugging
- What
- Asking for a ranked list of possible causes and how to test each one, then running the tests, instead of asking for a fix straight away.
- Why
- Each hypothesis is checked with evidence: a log line, a breakpoint, a changed input. That finds the real cause and teaches you the system.
# Debug mode: the agent forms hypotheses, adds
# temporary log statements, asks you to reproduce,
# then fixes the confirmed cause. Pick it in the
# mode menu (Cmd/Ctrl+.). Remove leftover logs.Watch out
Accepting the first fix that makes the error message go away without knowing why it appeared.
Paste the evidence
Also called stack trace, logs, error output
- What
- Giving the tool the full error, stack trace, logs, input and what you already tried, instead of a summary in your words.
- Why
- The details you think are irrelevant often hold the answer. Complete evidence saves several rounds of guessing.
@Terminals # attach terminal output to chat
Cmd/Ctrl+Shift+L # add the failing code selectionSymptom patches
Also called band-aid fix, swallowed exception
- What
- Changes that hide a bug instead of fixing it: an empty catch block, a null check that skips the work, a retry loop, a disabled lint rule or a longer timeout.
- Why
- AI tools reach for these because they make the error disappear. The bug is still there and now fails silently somewhere else.
- Example
- A test fails with "undefined is not a function". The patch adds
if (fn)before the call. The real bug is an import that returns undefined.
Root cause and regression test
Also called regression test, five whys
- What
- Fixing the cause you proved, then keeping the failing test from your repro so the same bug cannot return unnoticed.
- Why
- A bug fixed without a test tends to come back. The test also documents what went wrong for the next person.
Watch out
Declaring it fixed because the symptom stopped: if you did not find the cause, it is not fixed.
Use the real debugger
Also called breakpoints, step through, print debugging
- What
- Setting breakpoints, stepping through code and inspecting variables in your editor's debugger, or adding targeted log lines, to see what the program actually does.
- Why
- The model predicts what code probably does; the debugger shows what it does. When they disagree, the debugger is right.
Watch out
Going back and forth with the tool for an hour when five minutes with a breakpoint would have shown the wrong value.
Go deeper
- Debugging: The 9 Indispensable RulesPaid · booksefism.com/resources/debugging-9-indispensable-rules
- How to Ask Good Questions (Julia Evans)Free · guidesefism.com/resources/julia-evans-asking-good-questions
- The Missing Semester of Your CS EducationFree · coursesefism.com/resources/the-missing-semester-of-your-cs-education