Prompt Caching Across OpenAI, Claude, and Gemini

If your prompts share a long, stable prefix — a big system instruction, a tool catalog, a document you keep asking about — prompt caching lets the provider skip re-processing that prefix and bill it at a steep discount. All three of OpenAI, Anthropic, and Google offer this, and the payoff is the same everywhere: lower input cost and faster time-to-first-token on the cached portion. What differs is who decides what gets cached. OpenAI caches automatically by prefix. Anthropic makes you mark cache points explicitly. Gemini caches implicitly by default and also exposes an explicit cache object. This article maps the three control surfaces, the usage fields each returns, and how to structure a prompt so the cache actually hits.

Last checked: 2026-07-15. Per-model minimums, TTLs, and pricing multipliers change; confirm against each provider's linked pricing and caching pages before you rely on a specific number.

The one rule all three share: cache the prefix, vary the suffix

Every provider caches a prefix of the prompt and matches on exact bytes. The practical consequence is identical across vendors: put the parts that stay the same first, and the parts that change last. A system prompt, few-shot examples, tool definitions, and a large reference document belong at the top; the user's current question belongs at the bottom. Reorder those and you move the boundary between "cacheable" and "not," and a single early edit — even a timestamp injected into the system message — invalidates everything after it. This is the highest-leverage change you can make, and it is free.

Caching also has a minimum size. Short prompts are not worth caching and providers simply do not, so the discount only appears once the cached prefix crosses a threshold that varies by model. Below it, you pay the normal input rate and see zero cached tokens.

OpenAI: automatic, prefix-based, nudged by prompt_cache_key

OpenAI caching is automatic — you do not opt in. Caching is enabled automatically for prompts that are 1024 tokens or longer, and matching works off the prompt prefix. When a request reuses a prefix that is already cached, the reused tokens bill at the discounted cached-input rate and appear in the usage object as cached_tokens: under usage.prompt_tokens_details on Chat Completions, and under usage.input_tokens_details on the Responses API. Requests below the threshold report cached_tokens as zero.

You do not control caching directly, but you can influence routing with prompt_cache_key. Requests that share a long common prefix but come from different users or paths may land on different cache backends; setting the same prompt_cache_key on related requests is combined with the prefix hash to route them together and improve hit rates. On newer models this key is what makes matching reliable, and OpenAI suggests keeping traffic per key modest rather than firehosing thousands of requests per minute through a single key.

Retention is controllable. Newer models accept prompt_cache_options with a ttl (30 minutes is the supported minimum lifetime and default), while earlier models use prompt_cache_retention, whose in-memory entries generally stay active for a few minutes of inactivity and can be extended up to 24 hours. On the newest models, tokens written to cache are reported separately as cache_write_tokens and priced at 1.25× the uncached input rate; older models do not add a write fee. A gateway passes prompt_cache_key and prompt_cache_retention straight through to the upstream, so you keep the same knobs behind one base URL.

Anthropic: a cache_control marker, automatic or explicit

Anthropic caches only what a cache_control marker covers, but you do not have to hand-place every breakpoint. There are two modes. Automatic caching adds a single cache_control field at the top level of the request; the system applies the breakpoint to the last cacheable block and moves it forward as the conversation grows, which is the low-effort choice for multi-turn chats. Explicit breakpoints put cache_control directly on individual content blocks for fine-grained control over exactly what is cached. The example below uses an explicit breakpoint on a stable playbook:

import anthropic
client = anthropic.Anthropic(base_url="https://your-gateway.example.com", api_key="AVE_GATEWAY_API_KEY")

client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=[
        {"type": "text", "text": "You are a contract analyst."},
        {
            "type": "text",
            "text": LONG_STATIC_PLAYBOOK,        # the expensive, stable prefix
            "cache_control": {"type": "ephemeral"},
        },
    ],
    messages=[{"role": "user", "content": "Summarize the indemnity clause."}],
)

You may define up to 4 cache breakpoints (the automatic top-level marker consumes one of the four), which lets you cache sections that change at different frequencies — a rarely-edited system prompt behind one breakpoint, a per-session document behind another. The minimum cacheable length depends on the model (many current models, including Claude Opus 4.8 and Sonnet 5, cache from 1,024 tokens; some tiers require 2,048 or 4,096), and a prompt shorter than the minimum is silently processed without caching — no error, just no discount.

The usage accounting is explicit and worth reading carefully. Responses return three separate counts: cache_creation_input_tokens (tokens written to a new cache entry), cache_read_input_tokens (tokens served from cache), and input_tokens (everything after the last breakpoint). Total input is the sum of the three, and if both cache counts are zero, nothing cached. The default cache lifetime is 5 minutes, refreshed for free every time the entry is used, so an actively hit cache effectively stays warm; a "ttl": "1h" option extends it at higher write cost. The pricing multipliers are public: a 5-minute cache write costs 1.25× base input, a 1-hour write costs 2×, and every cache read costs 0.1× base input. That 10× read discount is the reason to bother.

Gemini: implicit by default, explicit when you need control

Google's default is the least work: implicit caching is enabled automatically for Gemini 2.5 and newer models, and cost savings are passed on automatically when a request hits cache. You structure the prompt with common content first, send similar-prefix requests close together in time, and read the cached-token count from the usage metadata. Minimums apply here too — current Flash and Pro tiers begin caching in the low thousands of tokens, and the newer generation requires a larger prefix.

When you need guaranteed caching and explicit lifetime control, Gemini's generateContent API offers explicit context caching through a CachedContent object: you upload the content once, receive a handle, set a TTL, and reference the handle on subsequent calls. Explicit caching guarantees the discount for content you know you will reuse (a long PDF, a video, a fixed instruction set) rather than leaving it to the implicit heuristic. The cached token count surfaces in usageMetadata (as cachedContentTokenCount on generateContent), which a gateway normalizes into the same cached-token field it reports for the other providers.

What a gateway can and cannot normalize

Behind one OpenAI-compatible endpoint, a gateway forwards the caching controls each provider understands and folds the disparate usage fields into a single accounting shape — cached_tokens for reads, plus cache-creation counts — so a cached request appears the same way in your usage logs regardless of upstream. The gateway also applies its own configurable billing ratios: a cache-read ratio that bills cached input below the normal rate, and a cache-creation ratio (defaulting to 1.25×) for cache writes, published per model on the public pricing page.

Be explicit about the limits, though. The three providers do not expose the same knobs, so normalization is lossy in both directions:

• Control does not map one-to-one. OpenAI caches by prefix with no notion of explicit breakpoints, while Anthropic requires a cache_control marker (automatic or up to four explicit) and has no OpenAI-style implicit prefix cache; a prompt_cache_key means nothing to Gemini. Passthrough forwards what an upstream supports and ignores what it does not.
• TTL semantics differ. OpenAI's ttl is a minimum lifetime, Anthropic's is a fixed 5-minute (or 1-hour) window refreshed on use, and Gemini's explicit TTL is set per cache object. "One hour" does not mean the same thing on all three.
• Usage vocabulary differs. Anthropic separates 5-minute and 1-hour cache creation; OpenAI separates read and write only on newer models; Gemini reports a single cached-content count. The gateway maps these into common fields but cannot invent a distinction a provider never reported.

Treat the gateway as the place that gives you one billing view and one passthrough path, not as a layer that makes three different caching designs behave identically. When in doubt, read the raw usage the gateway logs and reconcile it against the provider's own docs.

Making the cache actually hit

Four habits decide whether caching helps or quietly does nothing:

1. Front-load the stable content. Instructions, tools, and reference material first; user turn last. This is the single rule that matters most.
2. Keep the prefix byte-stable. No timestamps, request IDs, or shuffled tool orderings in the cached region. One changed byte early invalidates the rest.
3. Send related requests close in time. Caches expire; batching similar work within the TTL window keeps entries warm. An idle cache is a cold cache.
4. Verify with usage, not vibes. Confirm cached_tokens (or the provider's equivalent) is non-zero before claiming a saving. Caching that you assume but never measure is a common source of imaginary cost reductions.

Prompt caching is the highest-return item on the broader menu of cost work in reducing LLM API costs, and it pairs naturally with LLM context window management: the same stable prefix you cache is often the context you would otherwise re-send and re-bill on every turn.

FAQ

Does caching reduce output token cost too?

No. Caching applies to input (prompt) tokens only — the prefix the provider can reuse. Output tokens are generated fresh every call and billed at the normal output rate regardless of cache state. If your cost is dominated by long generations rather than long prompts, caching moves the wrong number; look at output length and model choice instead.

If both cache-read and cache-write counts are zero, did caching fail?

It means nothing cached for that request — usually because the prefix was below the model's minimum cacheable size, the prefix changed since the last call, or (for explicit schemes) you did not mark a breakpoint. Providers do not error on an un-cacheable prompt; they just bill it at the standard input rate. Read the usage fields to confirm rather than assuming a hit.

Is a cache hit guaranteed if I resend the exact same prefix?

No. Caches are best-effort and time-bounded. Entries expire after their TTL, and a hit also depends on routing — which is why OpenAI exposes prompt_cache_key to steer related requests to the same backend. Design for caching as an optimization that usually helps, not a guarantee you can bill against.

Can one gateway request use OpenAI, Anthropic, and Gemini caching interchangeably?

You can send caching controls through one endpoint, but each is only understood by its own upstream: cache_control reaches Anthropic, prompt_cache_key reaches OpenAI, an explicit CachedContent handle reaches Gemini. The gateway normalizes the reported usage into common fields; it cannot make one provider honor another's caching directive.

Measure a real before/after

Pick your most repetitive prompt — the one with the biggest fixed prefix — and run it twice through your gateway within a minute. On the first call the prefix is written to cache; on the second it should read from cache. Check your usage logs and confirm the second request reports non-zero cached tokens and a lower input charge than the first. That before/after pair, on your own traffic, tells you the real saving far better than any published multiplier.

References

• OpenAI: Prompt caching
• Anthropic: Prompt caching
• Google Gemini: Context caching
• OpenAI API pricing
• Anthropic pricing