Stage 1 · Lesson 2 of 8
Tokens and tokenisation
Models do not read letters or whole words. They read , which are of text taken from a fixed list of roughly 100,000 to 200,000 entries. In English a token is about four characters on average, or three quarters of a word.
So 1,000 words is roughly 1,300 tokens. Code, numbers, and most non-English languages break into many more tokens per word. Every limit and every bill in this track is counted in tokens.
Learn to estimate them, and check your guess with the provider's token counter or a tokenizer library (a tool that splits text the way the model does). Tokenisation also explains some odd failures. A model that sees "strawberry" as two or three chunks can miscount its letters.
A question to check yourself, an AI practice session and 1 cheat sheet entry come with the full roadmap. Unlock
Resources for this stage
Start with· follow one of these
- CoursePaidGrokking Modern AI Fundamentals
Design Gurus · Beginner · $72 one-time
Structured grounding in models, transformers and embeddings before you build on them
- VideoFreeDeep Dive into Large Language Models like ChatGPT
Andrej Karpathy · Intermediate
Free: Karpathy on tokens, training and why models hallucinate, in one sitting
Keep handy
Also worth your time
Input vs output tokens and a step by step cost estimate you can reuse for any feature
- Neural Networks: Zero to HeroGo deeper
Build a small GPT from scratch if you want to see the machinery for yourself