Prompt caching, explained: how to cut LLM API bills by 90%

Prompt caching has quietly become the highest-leverage cost optimization in production AI, yet adoption remains low because the mechanics are poorly understood. This guide covers the fundamentals.

The core idea

APIs charge dramatically less for input tokens they have already processed recently. If your requests share a long common prefix — system prompt, tool definitions, few-shot examples — that prefix can be cached and re-billed at 10% or less of the normal rate.

The rules that matter

• ▸Caches match on exact prefixes: one changed byte at position zero invalidates everything after it
• ▸Put stable content first, volatile content last
• ▸TTLs are short (typically 5 minutes), so steady traffic keeps caches warm
• ▸Cache writes cost slightly more than normal input; breakeven is usually 2 requests

A practical prefix layout

[system prompt]        <- never changes, cache this
[tool definitions]     <- changes on deploy only
[few-shot examples]    <- changes rarely
[conversation history] <- append-only, cache-friendly
[current user input]   <- always new, keep last

Teams that restructure prompts around caching routinely report 70-90% reductions in input token spend, with latency improvements arriving as a side effect.