Chapter 2 · Practice 4 of 4
Order prompt parts so the stable prefix can be cached
Why it matters
Prompt caching works on an exact prefix, so the order you choose also decides your cost and latency. OpenAI's guide says to put reusable content at the start of the prompt to benefit from caching.
A cache-friendly order looks like this:
- tool definitions and system instructions;
- long reference material that rarely changes;
- few-shot examples;
- the conversation so far;
- the new user input.
This fits with the long-context advice above: the long, stable material sits early, and the specific question sits at the end.