Deep Dive into Large Language Models like ChatGPT
Three and a half hours that replace a shelf of blog posts. Karpathy walks the whole training stack, pretraining through post training, and explains why models hallucinate, why they are bad at counting letters, and what a token really is. If you build on top of models, this is the clearest explanation of what is underneath.
Video
AI & Machine Learning
Andrej Karpathy
3.5 hours
Intermediate
Free
Three and a half hours that replace a shelf of blog posts. Karpathy walks the whole training stack, pretraining through post training, and explains why models hallucinate, why they are bad at counting letters, and what a token really is. If you build on top of models, this is the clearest explanation of what is underneath.
Ready to start?
Opens on Andrej Karpathy in a new tab.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Every session is free; a few slots open each week.
Follow along
New writing, resources and project ideas land here first.