AI engineering cheat sheet (Python)

1. 1How LLMs work: tokens and contextWhat a model does when it answers: text becomes tokens, the model predicts one token at a time inside a fixed context window, and every token costs money and time.9 concepts

   1. 1Large language modelA neural network, almost always a transformer, trained on huge amounts of text to predict the next token.
   2. 2Tokens and tokenisationModels read and write tokens: chunks of text from a fixed vocabulary, often word pieces.
   3. 3The context windowThe maximum number of tokens a model handles in one request, counting the system prompt, history, tool definitions, retrieved documents and the output.
   4. 4Token pricingProviders charge per million input tokens and per million output tokens, with output several times more expensive.
   5. 5Rate limitsCaps on requests per minute and on input and output tokens per minute for an account or workspace.
   6. 6Temperature and samplingThe model outputs a probability for every possible next token and sampling picks one.
   7. 7HallucinationA fluent, confident answer that is false or unsupported, such as an invented citation, API, number or policy.
   8. 8Reasoning models and effortModels that generate internal reasoning tokens before the final answer.
   9. 9Choosing a modelPicking a model by measuring quality on your own eval set against cost and latency.

   9 in this chapterOpen this chapter
2. 2Prompting and model APIsCalling a model well: the request shape, system prompts, clear instructions and examples, streaming, handling failures and treating prompts as code.9 concepts

   1. 1Chat messages APIThe core request: a model id, a maximum output length, an optional system prompt and a list of user and assistant messages.
   2. 2System promptInstructions sent separately from the conversation that set the role, rules, tone, output format and background for every turn.
   3. 3Clear, specific instructionsStating the task, audience, constraints, what good output looks like and the exact format, as you would brief a capable colleague with no context.
   4. 4Few-shot examplesIncluding a handful of worked input and output examples in the prompt so the model copies their format, style and judgement.
   5. 5Streaming responsesReceiving output piece by piece as it is generated, over server-sent events, instead of waiting for the whole response.
   6. 6Timeouts, retries and backoffSetting a timeout on every call and retrying transient failures (429, 5xx, overloaded, dropped connections) with exponential backoff and jitter.
   7. 7Hosted vs local modelsHosted models run behind a provider API.
   8. 8Prompts in version controlTreating prompts as code: stored in the repository or a prompt registry, reviewed, versioned and linked to the eval results each version produced.
   9. 9Open-weight modelsModels whose trained weights are published for download, each under its own licence.

   9 in this chapterOpen this chapter
3. RAG: embeddings, chunking and retrievalGiving a model your own knowledge: embed and chunk documents, retrieve the best passages with vector and keyword search, then answer only from them with citations.11 conceptsOpens with credits
4. Evals: measure before you add moreHow you know a change helped: read real outputs, build a small labelled test set, check with code first, use a judge model only once it agrees with you, and run it all in CI.8 conceptsOpens with credits
5. Tool calling and structured outputLetting a model act and return data your code can trust: tool definitions, the tool calling loop, schemas, validation and errors the model can recover from.7 conceptsOpens with credits
6. MCP: Model Context ProtocolThe open standard for plugging tools and data into AI applications: who talks to whom, what a server exposes, how it connects and how to build one.5 conceptsOpens with credits
7. Agents and durable executionWhen to let the model drive, and how to keep it safe and affordable: the agent loop, context engineering, hard limits, checkpoints, human approval and multiple agents.7 conceptsOpens with credits
8. Running AI systems in productionWhat it takes to run an LLM feature for real users: tracing, guardrails and injection defence, caching, routing and fallbacks, latency and cost budgets, and when to fine-tune or self-host.10 conceptsOpens with credits
9. Put it together: ship a production GenAI appHow the parts of a real GenAI app fit together, where each lives in the repo, and what must be true before launch. The GenAI app architecture cheat sheet goes deeper.4 conceptsOpens with credits
10. 10Glossary, A to ZEvery concept in the sheet and the smaller terms it uses, in one list.184 terms

Build products on LLMs and generative AI.