Deep Dive into Large Language Models like ChatGPT

What the lecture covers

The video (3 hours 31 minutes, released in February 2025, with timestamped chapters and about 9.7 million YouTube views as of September 2026) follows the order in which a model is actually built:

• Pretraining data: where the text comes from, shown by scrolling through a real web dataset (FineWeb) on screen.
• Tokenisation: byte pair encoding demonstrated live, which explains many odd model behaviours.
• The neural network: what goes in and what comes out, with a 3D visualisation of a transformer, then inference and a GPT-2 training run.
• Post-training: how a base model that only continues text is turned into an assistant using conversation data and chat templates.
• LLM psychology: hallucination, tool use such as web search and code execution, the difference between knowledge in the weights and information in the context window, and why models struggle with spelling and counting.
• Reinforcement learning: how models learn to reason by practising on problems, and why this is a different stage from imitating human answers.
• What comes next: multimodality, longer-running agentic tasks and how to keep track of new models.

Who it is for

No machine learning background is needed. It is aimed at anyone who uses these tools, but engineers get the most out of it because it explains failure modes they meet every day. If you have ever wondered why a model confidently makes up a citation, or why it does arithmetic better when asked to show its working, this answers it.

It is not a coding tutorial. You will not write a model yourself. For that, Karpathy's Neural Networks: Zero to Hero series builds everything in Python.

How to watch it well

• Split it into three sittings: pretraining, post-training, then the reinforcement learning and future sections.
• Keep a tokeniser tool open and try your own sentences when the tokenisation section starts.
• After each sitting, write down two things you will do differently when prompting or building with a model. For example, putting the facts a model needs directly in the prompt rather than relying on its memory.
• Rewatch the hallucination and tool use chapters before you build any feature that relies on model answers being correct.

Where to go after it

• Using models in products: read about retrieval augmented generation and evaluation, since this video explains why both are needed.
• Building models: Neural Networks: Zero to Hero, then the Let's build GPT lecture in that series.
• Karpathy's follow-up video on how he uses LLMs day to day is a practical companion if you want applied tips rather than internals.

Common questions

Is it still current? The fundamentals it explains (pretraining, tokenisation, post-training, reinforcement learning) have not changed. Specific model names and leaderboards it mentions will date faster.

Do I need maths? No. There are no equations to follow; the ideas are explained with visual demos and plain language.

Is three and a half hours worth it? If you use LLMs at work, yes. It replaces a lot of scattered reading, and the hallucination and tokenisation sections in particular change how most people prompt.

Best for: anyone who works with LLMs and wants to understand what is happening underneath without doing a machine learning course.