Chapter 5 · Practice 2 of 4
Put stable content first so prompt caching can reuse it
From Cost and latency
Why it matters
Providers can cache a repeated prompt prefix and bill cache hits at a fraction of the normal input price, with faster responses too. To get those hits, you have to order the prompt for it.
Caching matches on an exact prefix, so put the stable parts first:
- tool definitions
- system prompt
- long reference documents
- the conversation so far
- the new user message
Anthropic's docs describe a default cache lifetime of five minutes, refreshed on each hit, with the cache keyed on everything up to a marked breakpoint. Check your provider's usage fields for cached tokens to confirm hits.