Stage 1 · Lesson 5 of 8
Temperature and sampling
At each step the model gives a probability for every possible next token, and is how it picks one. Temperature changes how adventurous that pick is. Low values, like 0 to 0.3, make the most likely token win almost every time.
That suits extraction, and code. Higher values, like 0.8 to 1, spread the choices out, which suits brainstorming. Top-p is another setting: it limits the pick to the smallest group of tokens whose probabilities add up to p. Change one setting at a time, and judge the effect on real inputs, not on a single run.
Common trapbelieving temperature 0 makes output fully repeatable: batching (the provider running many requests together) and hardware can still change results slightly, and some reasoning models ignore or fix these settings.
A question to check yourself, an AI practice session and 1 cheat sheet entry come with the full roadmap. Unlock
Resources for this stage
Start with· follow one of these
- CoursePaidGrokking Modern AI Fundamentals
Design Gurus · Beginner · $72 one-time
Structured grounding in models, transformers and embeddings before you build on them
- VideoFreeDeep Dive into Large Language Models like ChatGPT
Andrej Karpathy · Intermediate
Free: Karpathy on tokens, training and why models hallucinate, in one sitting
Keep handy
Also worth your time
Input vs output tokens and a step by step cost estimate you can reuse for any feature
- Neural Networks: Zero to HeroGo deeper
Build a small GPT from scratch if you want to see the machinery for yourself