Chapter 01
Start from the task
Describe the job before you look at models: its task family, its requirements and what a wrong answer costs. Then build a shortlist from leaderboards and decide with your own data.
Task families
Also called LLM use cases, task types
- What
- Five common kinds of LLM work: chat, coding, research, data analysis and workflow automation. Each stresses a model differently, from tone and speed in chat to exact output and tool calls in automation.
- Why
- Naming the family tells you which strengths to test and which to ignore, before you compare a single model.
- Example
- A WhatsApp order bot is chat plus workflow automation: it needs a friendly tone, fast replies and reliable tool calls to update a sheet.
Model requirements card
Also called model requirements, six requirement lines
- What
- Six lines written before any testing: quality (and the cost of a wrong answer), speed, volume, context size, tools and privacy.
- Why
- These six answers rule most models in or out, and stop you choosing the most famous model by habit.
Watch out
Skipping the privacy line until the end, then finding the chosen provider cannot be used for this data.
Which model for which job
Also called model decision chart, model selection chart
- What
- A starting point for each kind of job: which tier to test first and when to step up or down. Always confirm it with your own golden set.
- Why
- It saves the first round of guessing and reminds you that most jobs do not need the top tier.
Job Start with Step when
Tagging, routing, FAQ small tier rules slip: middle
Extraction to JSON small + strict JSON messy docs: middle
Chat and tutoring middle tier easy turns: small
Coding and agents top, or middle+high long tasks: top
Research, long reports top + long context data too big: RAG
Data analysis middle + code tool many steps: top
Private or offline data open weights no GPUs: open host
Nightly bulk jobs any tier + batch passes: go smallerWatch out
Treating the chart as an answer: it only tells you where to start testing.
The cheapest model that is good enough
Also called good enough model, step down
- What
- The model to ship is the cheapest and fastest one that clears your quality bar for this task, not the smartest one available.
- Why
- Proving a feature on a strong model, then stepping down while your tests still pass, often cuts cost by ten times or more.
Watch out
Stepping down without a fixed test set, so you cannot tell when quality quietly dropped.
Model tiers
Also called model family, model sizes, parameters
- What
- Providers sell a family in sizes: a top tier (most capable, slowest), a middle workhorse and a small, cheap, fast tier. Open-weight families come in sizes measured in parameters, such as 27B.
- Why
- Asking which tier a task needs lasts longer than memorising names, which change every few months.
- Example
- As of October 2026: Claude Opus, Sonnet and Haiku; GPT-6 Astra, Sol and Luna; Gemini Pro, Flash and Flash-Lite.
Leaderboards
Also called Arena, LMArena, Artificial Analysis, model rankings
- What
- Public rankings of models. Arena (
https://arena.ai/leaderboard) ranks from blind human votes; Artificial Analysis (https://artificialanalysis.ai/) runs its own tests and measures speed and price. - Why
- They turn hundreds of models into a shortlist of three in minutes.
Watch out
Letting a leaderboard decide: it measures other people's tasks, not your documents, language or tools.
Dated snapshots
Also called as of date, stale prices
- What
- Every table of model names, prices or context windows is a photograph of one day. Write the date and the source link on it.
- Why
- Models, prices and retirements change every few weeks, so an undated table quietly becomes wrong.
Watch out
Copying prices from a blog post or screenshot instead of the provider's own pricing page.