Prompt caching: when it slashes your bill (and when it doesn't)

By LW Forge — maintainer of LLM Scout · Updated July 27, 2026

Prompt caching is the rare discount that's often invisible unless you build for it on purpose: two products with identical token counts can land on bills that differ by 5-10x, depending only on whether their prompt structure lets the provider recognize a repeated prefix. This piece walks through how OpenAI and Anthropic actually price caching, a worked example with real numbers, and the three situations where it quietly saves nothing.

How each provider actually charges for it

OpenAI's caching is automatic and free to opt into: any prompt of 1,024 tokens or more is checked for a matching prefix from a recent request, and matched tokens are billed at roughly a tenth of the normal input rate — no code changes, no explicit markers. Cache entries live for roughly the length of a coffee break, on the order of 30 minutes of inactivity before eviction, longer under sustained traffic.

Anthropic's version is explicit: you mark cache breakpoints in your prompt yourself, choosing exactly where the reusable prefix ends. Writing a prompt into the cache costs a premium over normal input price (roughly 1.25x); reading it back on a later request costs about a tenth of the input rate — the same order of magnitude as OpenAI's discount. The lifetime is a choice: a default 5-minute window, or a longer 1-hour option at a higher write premium, so you can match the cache to how bursty your real traffic is.

The two designs converge on similar economics — write once, read many times, for about 10% of the input price on repeat reads — but the engineering effort differs. OpenAI's discount is easy to capture by accident; Anthropic's requires you to decide, explicitly, what's stable enough to cache.

A worked example

Take an assistant with an 8,000-token system prompt (instructions, tool schemas, a few examples) serving 500 requests a day, dense enough that requests land within the cache window.

Without caching, at GPT-5.6 Terra's list rate for input tokens, repeating that system prompt on every call adds up to roughly 8,000 × 500 × 30 tokens a month — a few hundred dollars a month, just for text that never changes. Check the current per-token rate in the OpenAI cost calculator to see today's version of that number.

With caching active, only the first request of each cache window pays the write cost (negligible at this size); the rest read the prefix at roughly a tenth of the input rate — cutting that portion of the bill by around 90%, with zero change to output quality, because nothing about the model's answer changed. Only how the input was priced did.

Run the Claude cost calculator with your own token counts to see the full-price baseline, then apply this discount by hand to estimate the cached version.

The rule of thumb that holds up

If your prompt has a stable prefix longer than about 2,000 tokens and your traffic arrives faster than the cache expires, caching is close to a free win — implement it and move on. Below that size, the write premium (on Anthropic) or the sheer smallness of the saving (on both providers) usually isn't worth the added complexity of managing breakpoints.

When caching quietly does nothing

Three situations neutralize the discount:

1. Sparse traffic. A request every ten or fifteen minutes on OpenAI, or past the TTL on Anthropic, means every request is a cold write. You pay the write premium (or full price) on effectively every call and never collect the read discount.
2. A prefix that isn't actually stable. Any change to the cached portion — a different date stamp, a personalized greeting, a retrieved document that varies — invalidates the match. The fix is ordering: put static instructions and examples first, and push per-request variables (the user's message, retrieved context, the current time) to the end of the prompt, after the cache breakpoint.
3. Short prefixes. A 300-token system prompt caches fine but saves pennies; the win scales with prefix length, and a short one isn't worth the engineering attention.

What caching doesn't touch

Caching only discounts input tokens that hit the cache — it does nothing for output tokens, which are billed at full rate regardless, and does nothing for the parts of your input that change every request. For a chatbot, that means the win applies to the system prompt and, if you cache it deliberately, retrieved context — but not to the user's message or the model's reply. See how much a chatbot actually costs to run for the fuller cost breakdown this discount slots into, and Claude Sonnet 5 vs GPT-5.6 for how the two providers' caching mechanics compare head to head.

Check your own numbers

The math above is a template, not a universal answer — your real savings depend on your prefix length, your requests per minute, and which provider's cache lifetime matches your traffic pattern. Measure your actual system prompt length in the token calculator, then compare full-price and cached estimates side by side using the GPT vs Claude comparison.