Sefism Choosing an LLM and AI Stack Cheat Sheet

What this cheat sheet is

The Sefism Choosing an LLM and AI stack cheat sheet is a quick-revision companion for picking a model, a deployment and the tools around it. It is written for developers, students and founders about to build something with a large language model, and it follows the same order as the Choosing an LLM and AI stack roadmap.

It has 10 chapters and 75 concepts. Each concept is a short card: what it is in plain words, why it matters and, for most, the mistake people make with it. Each chapter ends with up to three library resources to go deeper, and a glossary at the end covers about 50 smaller terms.

Chapter by chapter

1. Start from the task

Seven concepts: task families, the model requirements card, a decision chart for which model fits which job, the cheapest model that is good enough, model tiers, leaderboards and dated snapshots. The point of the chapter is that the job decides the model, and leaderboards only build the shortlist.

2. Reasoning and quality

Seven concepts: reasoning models, reasoning tokens, effort and thinking settings, compounding errors over many steps, instruction following, price plus the cost of errors, and what benchmarks can tell you. It gives you the simple sum for when a bigger model is worth paying for.

3. Speed and cost

Eight concepts: tokens and tokenizers, input and output price, cost per task, latency, small models, batch processing, prompt caching and hidden costs. The core idea: prices are per million tokens, but your bill is per task.

4. Snapshot: prices and context, October 2026

Six cards with prices and context windows for the main Claude, OpenAI, Gemini and DeepSeek models and for open-weight models, read on 7 October 2026 from each provider's own page at standard rates, plus where to refresh the snapshot. Treat it as dated: these numbers change often, and the sheet says to review it every quarter.

5. Context and memory

Six concepts: the context window, maximum output, lost in the middle, chat history and memory, retrieval-augmented generation (RAG), and the choice between RAG and a long context.

6. Tools and agents

Five concepts: function calling, structured output, MCP support, tool choice and agent reliability over many steps. What to check before you trust a model with an agent.

7. Privacy and deployment

The largest decision chapter, with ten concepts: data retention and training, open-weight models, model licences, self-hosting, quantisation, data residency, data processing agreements, redaction before prompts, deployment options and the open weights or API checklist.

8. The rest of the stack

Eleven concepts covering every layer around the model: hosting, OpenAI-compatible APIs, embeddings models, the MTEB leaderboard, document extraction, OCR, web extraction, LLM frameworks, vector stores, LLM observability and a stack picker by layer with a sensible default for each.

9. Run your own comparison

Six concepts: the golden test set, the scoring rubric, LLM judges, a three-model comparison, the decision record and error analysis. This is where the decision is made.

10. Production choices

Nine concepts for keeping the choice right after launch: model routing, cascades, fallback models, LLM gateways, vendor lock-in, pinned model versions, model deprecation, the quarterly model review and production monitoring.

Rules of thumb

The sheet closes with ten rules, among them: write the six requirement lines before you open a leaderboard; use leaderboards to pick three candidates and your own golden set to choose one; compare models on cost per task, not price per million tokens; try a different effort level before a different model; check whether the data fits in the prompt before you build retrieval; read the licence on the model card before building on an open-weight model; keep every model call behind one function so a switch is a config change; and date every price table.

How to use it with the roadmap

• After each stage, read the matching chapter and see whether you can explain every concept without looking. The chapters line up with the roadmap's stages, with the dated price snapshot added as chapter 4.
• Before you pick a model for a new feature, run through chapters 1, 3 and 7: the requirements card, the cost arithmetic and the deployment checklist.
• When designing the stack, keep chapter 8's stack picker open and fill in one choice and one alternative per layer.
• At every quarterly review, refresh the snapshot in chapter 4 from the providers' pages and re-run the comparison in chapter 9.

How it works on Sefism

Everything on Sefism is free to learn from. The public cheat sheet page shows every chapter title, with the first chapters open to read. Keeping the sheet uses credits and opens every chapter in your member area; if you add the Choosing an LLM and AI stack roadmap to your dashboard, its cheat sheet comes with it, and the first roadmap you add is free. New members get starting credits, and if you have a question the sheet does not answer, you can send it to Tauseef.

Related on Sefism

• Arena and Artificial Analysis for chapters 1 to 3, and the MTEB leaderboard for choosing an embeddings model.
• Docling and vLLM for document extraction and self-hosting.
• Cheat sheets: LLM APIs, AI evals and GenAI app architecture.
• Related roadmaps: AI engineering and AI agents and MCP.

FAQ

Is this a substitute for the roadmap? No. The roadmap is the plan, with hours, milestones and experiments on your own feature; the sheet is what you revise from and keep open when you decide.

Are the prices current? They were read on 7 October 2026 from each provider's own page. Prices and models change often, so check the provider's page before relying on a number; chapter 4 lists where to look.

What is cost per task? The tokens one real request uses, input plus output plus any reasoning tokens, multiplied by their prices, minus any batch or caching discount. It is the number to compare models on, not the price per million tokens.

When should I choose an open-weight model? When privacy, residency or licence terms rule out a cloud API, or when volume makes self-hosting cheaper and your team can run it. Chapter 7's checklist helps you decide.