Stage 1 · Lesson 2 of 6
Hallucinations and the context window
A is a confident answer that is simply wrong: a function, command-line flag, file path or package that does not exist, or an outdated detail from before the model was trained. The is everything the model can see at once: your messages, its replies, the files it read and command output, measured in .
It has a fixed size. The model knows nothing outside it, and when it fills up the tool summarises or drops older parts, and answers get worse. These two ideas explain most bad results: the model guessed because the fact was not in its context, or the context was full of noise.
Common trapkeeping one long chat for unrelated tasks and then trusting details about a library version newer than the model without opening its docs.
A question to check yourself, an AI practice session and 2 cheat sheet entries come when you add the roadmap to your account. Add it
Resources for this stage
Start with
- VideoFreeDeep Dive into Large Language Models like ChatGPT
Andrej Karpathy · Intermediate
A clear, free explanation of how language models are trained and why they hallucinate; watch the first hour