Every request sends the same 10,000-token system prompt. Same coding standards. Same few-shot examples. Same tool definitions. You pay full input price for every single one. At 500 requests per day with GPT-5.5 at $5/M input, that’s $25 per day —$750 per month —just for content the model has already processed 500 times before.
Prompt caching drops that to $75 per month. Same content. Same output quality. One code change. Prompt caching is one of the 12 strategies covered in our cost optimization walkthrough —and it delivers the highest ROI for the smallest code change.
This article covers how caching works across all four major providers, the exact code to implement it, and how to diagnose whether your workload is cache-friendly. This is the authority reference for prompt caching across the TokSpan blog —other articles link here for full implementation details.
How Prompt Caching Works (and Why It’s Not a Silver Bullet)
When you send a prompt to an LLM, the model processes every token —even the ones it’s seen hundreds of times before. The computation is repeated for every request. Cache those tokens, and the model skips the redundant computation.
The mechanics: You mark a portion of your prompt as cacheable. The provider hashes that portion. On subsequent requests with the same prefix, the provider retrieves the cached computation instead of re-running it. You pay a reduced rate for cached tokens —or, with some providers, no inference cost at all for the cached portion.
What can be cached: System prompts (the most common and highest-ROI use case). Tool definitions —they’re identical across requests. Few-shot examples. Static document context —product documentation, knowledge base articles, coding standards. Conversation history up to the last message —the history is identical until the newest user message.
What can’t be cached: The user’s new message —it’s at the end of the prompt and unique per request. Non-deterministic prefixes —anything that changes between requests. Content shorter than the provider’s minimum cache length (typically 1,024 tokens).
The catch: Caches expire. TTLs range from 5 minutes (Anthropic, OpenAI) to configurable hours (Google). If your request interval exceeds the TTL, the cache is cold and you pay full price. Caches are also provider-specific and model-specific —switching from Opus to Sonnet invalidates the cache.
Provider-by-Provider Implementation
Anthropic —90% off, best-in-class, explicit control.
Anthropic offers the best caching economics in the industry: 90% off cached input tokens. Anthropic’s prompt caching documentation covers cache breakpoints, TTL behavior, and pricing in detail. Cache writes carry a small premium over the base input price (see pricing breakdown below). Break-even: approximately 1.3 total requests per cache write —meaning just 2 requests sharing a cached prefix already saves ~32.5%. For most production workloads, you’ll get dozens. For a complete walkthrough of setting up the Anthropic SDK with caching, see our Claude API developer guide.
import anthropic
client = anthropic.Anthropic(
base_url="https://api.tokspan.com/anthropic",
api_key="ts-your-key-here"
)
response = client.messages.create(
model="claude-opus-4-8",
max_tokens=1000,
system=[
{
"type": "text",
"text": "You are a code reviewer. Here are our 10,000-token coding standards...",
"cache_control": {"type": "ephemeral"} # Cache this
}
],
messages=[{"role": "user", "content": "Review this PR."}]
)Cache breakpoints. You can place multiple cache_control blocks throughout your messages. Each marks a point where the prefix up to that point is cached. Strategic placement: cache the system prompt (always). Cache the conversation history excluding the last user message (the history is static; the new user message is dynamic). Cache tool definitions (they rarely change).
Minimum cacheable length: 4,096 tokens for Opus 4.5+ (including Opus 4.8/4.7/4.6/4.5) and Haiku 4.5; 2,048 tokens for Sonnet 4.6. Prompts shorter than this won’t be cached —the overhead of cache management exceeds the computation savings.
OpenAI —50% off, automatic, zero-effort.
OpenAI’s prompt caching is automatic for prompts longer than 1,024 tokens. No code changes. No cache_control blocks. The provider detects repeated prefixes and applies the discount silently. The tradeoff: 50% discount vs. Anthropic’s 90%, and no guarantee of a cache hit —the provider decides when to cache.
# No special code needed. OpenAI automatically caches repeated prefixes.
response = client.chat.completions.create(
model="gpt-5.5",
messages=[
{"role": "system", "content": "Your 5,000-token system prompt here..."},
{"role": "user", "content": "User message here."}
]
)
# If the system prompt was cached, you pay 50% less for those input tokens.
# Check response.usage for cache status.Google Gemini —context caching, explicit, configurable TTL.
Google’s approach is different: you create a “cached content” resource with an explicit TTL —Google’s context caching docs cover setup and pricing. TTLs range from minutes to 24 hours. You reference the cached content in your requests. The cache persists across multiple requests and users. Best for: document Q&A with static content that multiple users query against.
DeepSeek —$0.0036/M cache hits, automatic, cheapest absolute price.
DeepSeek’s cache-hit price of $0.0036 per million tokens is absurdly cheap —40x cheaper than their already-cheap base rate. Caching is automatic (similar to OpenAI). No explicit control. But at this price, the economics are favorable for virtually any workload with repeated content.
Cache Pricing: The Real Savings
| Provider | Cache Write | Cache Read | vs. Base Input | TTL | Automatic? |
|---|---|---|---|---|---|
| Anthropic | 1.25x base | 10% of base | 90% off | ~5 min | No (explicit) |
| OpenAI | — | 50% of base | 50% off | 5–60 min | Yes |
| Google Gemini | Varies by TTL | Varies by TTL | Up to 75% off | Configurable | No (explicit) |
| DeepSeek | — | $0.0036/M | ~97% off | ~5 min | Yes |
Selection guide. Beyond raw pricing, each provider’s caching implementation has architectural tradeoffs. Anthropic gives you explicit control and the deepest discount —but you pay a 25% cache-write premium and must manage breakpoint placement yourself. OpenAI is fire-and-forget: zero code, 50% savings, no guarantee of cache hits. Google’s configurable TTL is unique: set a 24-hour cache on your product documentation and serve thousands of users against one cached resource. DeepSeek’s absolute price floor ($0.0036 per million cached tokens) makes cache-hit-rate optimization largely irrelevant —at that price, even a 10% hit rate saves money.
| Provider | Min Cache Length | Effort | Best For | Caveat |
|---|---|---|---|---|
| Anthropic | 4,096 tokens (Opus 4.5+, Haiku 4.5); 2,048 (Sonnet 4.6) | Add 3 lines | System prompts, tool defs, few-shot | 1.25x write cost; 5-min TTL |
| OpenAI | 1,024 tokens | None | Any workload, lazy savings | No hit guarantee; only 50% off |
| Google Gemini | Varies by model | Create resource | Document Q&A, long-TTL needs | Higher cost for longer TTLs |
| DeepSeek | ~1,024 tokens | None | High-volume, price-sensitive | Automatic only; less predictable |
Savings calculator. A workload with 500 requests/day, 10,000-token cached system prompt, 500-token user messages, using Claude Opus at $5/M input:
- Without caching: 500 ×(10,000/1,000,000 ×$5) = $25/day on cached content alone
- With caching: 500 ×(10,000/1,000,000 ×$0.50) = $2.50/day on cached content
- Annual savings: $8,212 —$821 on that content. One code change.
The hidden cache write cost. Anthropic charges 1.25x the base input price for cache writes. The breakeven is approximately 1.3 total requests per cache write —meaning even a single cache read (2 requests sharing the same prompt) saves ~32.5% vs. no caching. For system prompts and tool definitions that are identical across all requests, this is never a concern. For content where most requests are unique (no cache reads), you pay the 25% write premium with zero offsetting savings. Monitor your cache hit rate. Target >80%.
Is Your Workload Cache-Friendly?
Best workloads for caching:
- Chatbots with long system prompts —the prompt is identical across all users and conversations. Cache hit rate: near 100%.
- RAG applications with static document context —the knowledge base content is the same for every query against it. Cache hit rate: high, depending on how many distinct documents you query.
- Few-shot prompted tasks —your examples are identical across requests. Cache them.
- Multi-turn conversations with long history —the history up to the last message is static. Cache everything except the final user turn.
- Coding agents with tool definitions —tool schemas are static. Cache them.
Worst workloads for caching:
- One-off prompts —every request is unique. No repeated prefixes. Cache hit rate: 0%.
- Unique documents per request —if every query is against a different document, there’s nothing to cache.
- Very short prompts (<1,024 tokens) —below the minimum cacheable length for most providers.
- High-randomness generation —if every response is creative and unconstrained, the output isn’t cacheable (caching is about input tokens).
Simple diagnostic. Log the first 2,000 characters of every API request for one day. Count how many are identical. If >50% of your requests share a common prefix longer than 1,000 tokens, prompt caching will save you substantial money. If <20%, the savings won’t justify the implementation effort for explicit caching (though automatic caching still provides passive savings).
Cache Hit Rate Diagnostic: A 5-Minute Script
Before touching production code, verify your workload is cache-friendly. Run this script against your last 1,000 API requests. It hashes the cacheable portion of each request and reports your duplication rate —the percentage of requests that share a prefix with at least one other request.
import hashlib
import json
from collections import Counter
# Load your request log —adapt the path and format to your setup
with open("api_requests.jsonl") as f:
requests = [json.loads(line) for line in f]
def get_cacheable_prefix(req):
"""Extract the static portion of each request.
For most applications this is the system prompt + any messages
before the final user turn."""
messages = req.get("messages", [])
prefix = messages[:-1] if len(messages) > 1 else messages
return json.dumps(prefix, sort_keys=True)
prefixes = [get_cacheable_prefix(r) for r in requests]
hashes = [hashlib.md5(p.encode()).hexdigest() for p in prefixes]
counts = Counter(hashes)
total = len(requests)
unique = len(counts)
most_common = counts.most_common(1)[0] if counts else (None, 0)
dup_rate = (total - unique) / total * 100 if total else 0
print(f"Total requests analyzed: {total}")
print(f"Unique prefixes: {unique}")
print(f"Most-repeated prefix: {most_common[1]} occurrences")
print(f"Duplication rate: {dup_rate:.1f}%")
if dup_rate > 70:
print("=> Cache-friendly. Implement explicit caching —high ROI.")
elif dup_rate > 40:
print("=> Borderline. Caching helps, but audit write costs first.")
else:
print("=> Not cache-friendly. Skip caching; use other optimizations.")What the numbers mean for your bill. A 70% duplication rate on 500 daily requests with a 10,000-token system prompt at $5/M input means 350 requests benefit from caching. With Anthropic’s 90% discount, cached tokens cost $0.50/M instead of $5/M —saving $15.75/day on that block alone. With OpenAI’s automatic 50% discount, you save $8.75/day with zero code changes. Even a 40% duplication rate matters: 200 requests/day at 10,000 tokens saves $9/day on Anthropic.
No access to request logs? Your LLM provider’s dashboard shows total request count and average tokens per request. Cross-reference with your application’s MAU. If you serve 100 daily active users with a fixed 5,000-token system prompt, you have 100 identical prefixes. If each user uploads a unique 20-page document per session, your duplication rate rounds to zero. The dashboard tells you which bucket you are in without scraping a single log line.
When Prompt Caching Costs You Money
Most of the time, prompt caching saves money. But under the wrong conditions, it does the opposite —you pay more and get nothing back. Here are the scenarios where you should think twice before adding cache controls.
Anthropic’s write premium on low-hit-rate workloads. Anthropic charges 1.25x the base input price for every cache write. If your cache hit rate falls below 20%, the write premium exceeds the read discount. A 10,000-token cached block at $5/M base: cache write costs $0.0625 (1.25x), cache read costs $0.005 (0.1x). You need approximately 1.3 total requests per cache write to break even —meaning even a single cache read (2 total requests sharing the same prompt) saves ~32.5%. A support bot that rotates its system prompt every 3 requests (1 write + 2 reads) saves ~52% vs. no caching. Monitor cache_read_input_tokens vs cache_creation_input_tokens in Anthropic’s usage response. If the read-to-write ratio is under 0.5:1 (fewer than 1 read per 2 writes), remove the cache block.
Streaming-heavy, one-shot workloads. Real-time transcription, one-off code generation, creative writing —every prompt is unique by design. Adding cache_control blocks to a stream of unique prompts adds ~20 tokens per block to every request and never returns a cache hit. OpenAI’s automatic caching skips these silently (no hit, no charge). But Anthropic’s explicit cache_control blocks always incur the write cost on first appearance. On unique workloads, every request is a first appearance —you pay the 25% premium on every call with zero offsetting savings.
Prompts that fall just under the minimum threshold. Anthropic requires 4,096 tokens for Opus 4.5+ and Haiku 4.5, and 2,048 tokens for Sonnet 4.6. OpenAI requires 1,024 tokens. A 1,500-token system prompt with a cache_control block on Opus costs the same as one without —the provider silently ignores the cache directive because it falls below the 4,096-token minimum. One cache_control block adds ~20 tokens. At 5 million requests/month and $5/M input, those 20 wasted tokens cost $500/month. Check your actual token count (not your character count) with the provider’s tokenizer before adding cache controls.
Quick rule. Only invest in explicit caching when: (a) your cacheable prefix exceeds 1,024 tokens, (b) it appears in over 50% of requests, and (c) your median request interval is under the provider’s TTL. If any condition fails, let automatic caching handle it or skip caching entirely and pick another strategy from the cost optimization playbook.
Caching Strategy Guide
The hierarchy. Anthropic for maximum savings on explicit, high-frequency caches. DeepSeek for the cheapest absolute cache-read price on high-volume workloads. Google for long-TTL document caches (up to 24 hours). OpenAI for zero-effort automatic savings.
The multi-provider cache strategy. Cache your static content on the provider with the best cache economics (Anthropic, 90% off). Route cache-friendly traffic to that provider. Route unique requests to the provider with the best base pricing for your use case. This is an advanced optimization —implement basic caching first, tune routing later.
Platform-level caching. Some aggregation platforms layer their own caching on top of provider caching. The platform caches responses at the API gateway level —identical requests served from the platform cache never reach the provider. For high-volume applications with repetitive query patterns, this doubles the savings. TokSpan’s prompt caching documentation covers platform-level caching setup, including cache-hit analytics and per-provider savings tracking.
FAQ
How much can I actually save with prompt caching?
50–90% on input tokens, depending on provider and workload. With Anthropic (90% off) and 80% of input tokens cached: effective input cost drops ~72%. On a $1,000/month input spend, that’s $280/month —$720/month savings.
Do I need to change my code?
OpenAI and DeepSeek: no, caching is automatic. Anthropic: yes, add cache_control blocks (3 lines of code). Google: yes, create cached content resources. The implementation effort is proportional to the savings —Anthropic requires the most code but offers the highest discount.
How long do caches last?
Anthropic: ~5 minutes. OpenAI: 5–60 minutes (variable, not guaranteed). Google: configurable TTL (minutes to 24 hours, with higher cost for longer TTLs). DeepSeek: similar to OpenAI. For applications with request intervals under 5 minutes, caching works automatically. For less frequent access patterns, only Google’s configurable TTL helps.
Can I cache across different models from the same provider?
Generally no. Cache is model-specific. Switching from Opus to Sonnet invalidates the cache. Pin to specific model versions in production to maximize cache hit rates.
What happens if my cache expires mid-conversation?
The provider reverts to full pricing for the affected tokens —no error, no interruption, just higher cost for that one request. The user experience is unaffected. Cache expiration is a cost event, not a reliability event. Low risk, high reward.
Prompt caching delivers up to 90% savings with almost no code —a rare free lunch in infrastructure. But free lunches have a history of getting priced in. As caching becomes standard across every provider, the real question is whether today’s discounts will survive the next pricing cycle, or whether the savings get silently baked into higher base rates while the marketing stays the same.
If the discount window does close, the question is not whether caching was worth implementing —the savings from even six months of 90%-off input tokens more than justify the three lines of code it took to enable. Set up prompt caching on TokSpan and layer platform-level caching on top of every provider’s built-in discount while the economics still favor it this heavily.