Prompt caching

Many requests in a feature begin with the same long prefix: the system prompt, tool definitions, few-shot examples, a reference document, or the earlier turns of a conversation. With prompt caching, the provider keeps the computed state of that prefix and reuses it for later requests that begin the same way. With Anthropic you mark cache breakpoints with cache_control; other providers cache automatically or offer their own controls.

The savings can be large. On Claude, cache reads cost a small fraction of the normal input price, while writing to the cache costs somewhat more than normal input, so caching pays off when a prefix is reused several times within the cache lifetime. Latency drops too, because the cached part isn't processed again.

Caching is an exact prefix match, and that drives how you structure prompts. Put stable content first and anything that changes per request, such as timestamps, user names, or the newest message, after the last breakpoint. A single changed character early in the prompt invalidates everything after it. Prefixes below a minimum length don't cache at all, and caches are per model. Confirm it's working by checking the cache-read token count in each response's usage data.