Tauseef FayyazA to Z
Glossary
Every concept in this sheet, plus the smaller terms it uses. Select a concept to open its card.
C
- Chat messages API
- The core request: a model id, a maximum output length, an optional system prompt and a list of user and assistant messages.
- Choosing a model
- Picking a model by measuring quality on your own eval set against cost and latency.
- Clear, specific instructions
- Stating the task, audience, constraints, what good output looks like and the exact format, as you would brief a capable colleague with no context.
F
- Few-shot examples
- Including a handful of worked input and output examples in the prompt so the model copies their format, style and judgement.
H
- Hallucination
- A fluent, confident answer that is false or unsupported, such as an invented citation, API, number or policy.
- Hosted vs local models
- Hosted models run behind a provider API.
L
- Large language model
- A neural network, almost always a transformer, trained on huge amounts of text to predict the next token.
O
- Open-weight models
- Models whose trained weights are published for download, each under its own licence.
P
- Prompts in version control
- Treating prompts as code: stored in the repository or a prompt registry, reviewed, versioned and linked to the eval results each version produced.
R
- Rate limits
- Caps on requests per minute and on input and output tokens per minute for an account or workspace.
- Reasoning models and effort
- Models that generate internal reasoning tokens before the final answer.
S
- Streaming responses
- Receiving output piece by piece as it is generated, over server-sent events, instead of waiting for the whole response.
- System prompt
- Instructions sent separately from the conversation that set the role, rules, tone, output format and background for every turn.
T
- Temperature and sampling
- The model outputs a probability for every possible next token and sampling picks one.
- The context window
- The maximum number of tokens a model handles in one request, counting the system prompt, history, tool definitions, retrieved documents and the output.
- Timeouts, retries and backoff
- Setting a timeout on every call and retrying transient failures (429, 5xx, overloaded, dropped connections) with exponential backoff and jitter.
- Token pricing
- Providers charge per million input tokens and per million output tokens, with output several times more expensive.
- Tokens and tokenisation
- Models read and write tokens: chunks of text from a fixed vocabulary, often word pieces.