Stage 1 of 8 · 8 lessons · ~3 weeks
How LLMs work: tokens and context
What a model does when it answers: text becomes tokens and the model predicts one token at a time inside a fixed context window. Tokens are both your bill and your limit, so learn to decide what earns a place in the context.
Start: What a model does: predicting the next tokenLessons· one lesson for each topic, in this order
- 1What a model does: predicting the next tokenLesson
- 2Tokens and tokenisationLesson
- 3The context window as a budgetLesson
- 4Token pricing and rate limitsLesson
- 5Temperature and samplingLesson
- 6Why models hallucinateLesson
- 7Reasoning models and effort settingsOptionalLesson
- 8Choosing a model: quality, cost, latencyLesson
Start with· follow one of these
- CoursePaidGrokking Modern AI Fundamentals
Design Gurus · Beginner · $72 one-time
Structured grounding in models, transformers and embeddings before you build on them
- VideoFreeDeep Dive into Large Language Models like ChatGPT
Andrej Karpathy · Intermediate
Free: Karpathy on tokens, training and why models hallucinate, in one sitting
Keep handy
Also worth your time
Input vs output tokens and a step by step cost estimate you can reuse for any feature
- Neural Networks: Zero to HeroGo deeper
Build a small GPT from scratch if you want to see the machinery for yourself
Before moving on: Take one real prompt from a feature you want to build, count its input and output tokens, work out its cost per 1,000 requests on two different models, and write down which parts of the context actually earn their place.
You do not need to train a model to build on one, but you do need a correct mental model of tokens and context. Every token you send costs money and pushes something else out, so treat the context window as a budget from day one and leave the attention maths until you have shipped something.