LLM APIs cheat sheet (OpenAI)

1. 1How LLM APIs workAn LLM API takes text (and sometimes images) in and returns generated text, billed by the token. These ideas shape every request you will write.6 concepts

   1. 1TokensModels read and write tokens: chunks of text that are often a word or part of one.
   2. 2Context windowThe maximum number of tokens a model can take in one request, counting the system prompt, the conversation history, tool definitions, documents and the output.
   3. 3The Responses APIOpenAI’s current main endpoint.
   4. 4Your first requestCreate a client (it reads the API key from an environment variable) and send a prompt.
   5. 5Reading token usageEvery response reports how many input and output tokens it used (and cached tokens, where relevant).
   6. 6Choosing a modelProviders offer tiers: large models for hard reasoning and coding, mid-size for most product work, small fast ones for classification and high volume.

   6 in this chapterOpen this chapter
2. 2Prompts, roles and samplingHow you structure a request: who says what, which instructions persist, how history is kept, and which knobs change the style and length of the answer.8 concepts

   1. 1Messages and rolesA conversation is a list of turns, each with a role: the user, and the model (called assistant, or model in Gemini).
   2. 2System promptsStanding instructions that apply to the whole conversation: role, tone, rules, output format and context about the product.
   3. 3Keeping conversation stateEither resend the full history yourself each turn, or use a provider feature that keeps it for you where one exists.
   4. 4Clear, structured instructionsState the task, the audience, the constraints and the output format explicitly, and separate instructions from data with headings or tags such as <document>.
   5. 5Few-shot examplesIncluding a handful of example inputs and ideal outputs in the prompt so the model copies the pattern.
   6. 6Temperature and samplingTemperature controls randomness when picking the next token: low values give focused, repeatable answers, higher values more varied ones.
   7. 7Reasoning and effortReasoning models think before answering, using extra hidden output tokens.
   8. 8Output limits and stop reasonsYou cap output length, and each response says why it stopped: finished naturally, hit the limit, called a tool or was declined.

   8 in this chapterOpen this chapter
3. Streaming, structured output and imagesGetting output into a shape your product can use: streamed for chat interfaces, schema-checked JSON for code, and images as input when text is not enough.6 conceptsOpens with credits
4. Tool calling and agentsTool calling lets the model ask your code to do things: look up an order, run a query, send an email. Your code runs the tool and sends the result back, in a loop, until the model answers.7 conceptsOpens with credits
5. Embeddings and retrieval (RAG)Retrieval-augmented generation finds the most relevant pieces of your own data and puts them in the prompt, so answers are grounded in current, private information the model was never trained on.7 conceptsOpens with credits
6. Cost, caching, batching and reliabilityWhat changes when real traffic arrives: repeated prompts should hit the cache, bulk jobs should go through batch APIs, and every call needs retries, timeouts and a budget.8 conceptsOpens with credits
7. Evals and observabilityPrompts and models change behaviour in ways unit tests do not catch. Evals turn "it seems better" into a number, and tracing shows what actually happened in production.7 conceptsOpens with credits
8. Security and safetyLLM features add new attack surface: text can carry instructions, output can carry payloads, and API keys are valuable. Design as if every input and output is untrusted.7 conceptsOpens with credits
9. 9Glossary, A to ZEvery concept in the sheet and the smaller terms it uses, in one list.78 terms

Building on OpenAI, Anthropic and Gemini APIs: prompts, streaming, structured output, tools, RAG, caching, cost, evals and safety.