Claude API Prompt Caching Cost Reduction: Chunk Size Experiments

Prompt caching cut one production team’s Claude API costs by 60%. That’s not a marketing claim — it’s a documented outcome from a DEV Community case study on real workloads. But that number hides something important: chunk size determines whether you hit 60% savings or 6%.

The discussion around Claude API prompt caching has been running hot in developer communities because caching is genuinely powerful, but most teams are leaving money on the table. They enable caching, see some savings, and move on. The ones who dig into chunk sizing are the ones getting the headline numbers.

This is a breakdown of what the data shows, why chunk size is the critical variable, and how to structure your caching strategy for 2026 production workloads.

In brief: Prompt caching on the Claude API can reduce input token costs by up to 90% on cached segments, but actual savings depend heavily on how you structure cacheable content blocks. Teams that engineer for cache reuse — not just enable caching — see the biggest cost deltas.

1. Claude’s prompt caching requires a minimum 1,024 tokens to activate on Haiku and Sonnet 3.5, and 2,048 tokens for Opus models.
2. Cache writes cost more than standard input tokens (25% premium), so cache hit rate must be high enough to justify the write cost — making chunk size and reuse frequency the two levers that matter most.
3. The 60% cost reduction documented in production came from combining caching with batching, not from caching alone.

Why Prompt Caching Became a Cost Priority

Claude API pricing charges per input token on every request. For applications with large, stable system prompts — think RAG pipelines, legal document review tools, or code assistants with lengthy context — you’re paying to re-send the same tokens hundreds or thousands of times per day.

Anthropic introduced prompt caching on Claude in mid-2024. The mechanism: mark specific content blocks with cache_control: {"type": "ephemeral"}, and Claude stores a KV snapshot of that prefix. Subsequent requests that hit that exact cache point pay roughly 10% of the standard input token price — a 90% reduction on cached tokens, according to Anthropic’s official API documentation.

Cache entries live for 5 minutes by default, with a TTL that resets on each hit. That’s a meaningful constraint. An application firing requests every 30 seconds will sustain cache hits. One making hourly batch calls will not — at least not without architectural changes.

The conversation has shifted from “does caching work?” to “how do you structure prompts to maximize cache hit rates?” That’s the right question. And chunk size is the answer most teams are still working out.

The Minimum Token Threshold Changes Everything

Cache activation isn’t free or automatic. Per Anthropic’s documentation, the minimum cacheable block is 1,024 tokens for Claude Haiku and Sonnet 3.5, and 2,048 tokens for Opus. If your system prompt is 800 tokens, caching doesn’t activate at all. You’re paying the standard rate.

This creates a real design constraint. Teams with lean system prompts can’t just flip a switch. They either need to restructure prompt architecture or consolidate multiple prompt components into a single cacheable block.

The DEV Community production case study showed that chunking worked best when teams front-loaded stable content — system instructions, domain knowledge, document context — into a single large block at the top of the prompt. Dynamic content (user queries, session variables) went at the end, outside the cached segment. That structure maximizes the token count inside the cache boundary and keeps the cache key stable across requests.

Cache Write Costs vs. Cache Hit Economics

Cache writes are priced at 125% of standard input token cost per Anthropic’s pricing documentation. That’s the friction point most analyses skip.

A cache miss doesn’t just fail to save money — it costs more than a standard request. Write a 2,000-token block, pay 125% on those tokens. If that cache entry only gets hit once before expiring, you’ve spent more than you would have without caching.

The break-even math: if a cached block costs 1.25x to write, you need at least 2 cache hits per write cycle to come out ahead. At 3+ hits per 5-minute window, savings compound quickly toward that 90% reduction on the cached segment. High-frequency applications — chatbots, live coding assistants, real-time document QA — hit this threshold easily. Low-frequency or batch workflows don’t, unless you explicitly architect around it.

Chunk Size Experimentation: What the Data Suggests

The chunk size question doesn’t have one universal answer. It depends on your traffic pattern. But the production data points to a few consistent findings.

Larger chunks work better when content is stable. A 4,000-token system prompt that never changes is a better candidate than four 1,000-token blocks that each change slightly. Every character difference in a block creates a new cache key — you’re not getting a hit, you’re writing a new entry.

Granular chunking increases write overhead. Break a 6,000-token prompt into six separate 1,000-token blocks, and each one requires its own cache write. Miss rate multiplies. Savings erode.

The sweet spot in documented experiments runs 2,000–5,000 tokens per cached block, sized to keep content stable and hit rates high. That range clears the minimum threshold on all Claude models, stays well within the 32K limit on cached prompts, and tends to align with real prompt structures — full system prompt plus static reference document.

Comparing Caching Strategies by Workload Type

The batch processing row matters. Caching actively hurts low-frequency workloads. The DEV Community case study achieved its 60% figure by pairing caching with the Batch API for asynchronous processing — not by applying caching indiscriminately.

This approach can also fail when teams apply caching to prompts that seem stable but contain subtle dynamic elements — timestamps, session IDs, or frequently rotated examples embedded in what looks like static context. Each variation breaks the cache key. You end up paying write costs on every request with almost zero hits.

Three Scenarios Worth Modeling

Scenario 1: You’re running a customer support chatbot with a 3,000-token system prompt.

Cache it. Every call. At even 50 requests per hour, the write cost amortizes within the first 5-minute window and savings compound on every subsequent hit. Structure the entire system prompt as one block. Put user messages and conversation history after it.

Scenario 2: You’re doing nightly document analysis across 10,000 documents.

Don’t cache. Use the Batch API. Your request frequency is too low for cache TTL to work in your favor. Batching gives you a 50% cost discount on input tokens without the write overhead. The DEV Community production analysis confirmed this split approach — caching for interactive workloads, batching for async jobs — as the architecture that reached the 60% aggregate reduction.

Scenario 3: You’re building a RAG application where retrieved context changes per query.

Cache the system prompt and any static domain knowledge. Don’t cache the retrieved chunks — they change too often to maintain cache key stability. This hybrid approach captures savings on the stable portion, often 40–60% of total prompt tokens in RAG setups, while staying honest about what can’t be cached.

What the Numbers Actually Mean

The bottom line on Claude API prompt caching in 2025:

Key Takeaways

• The 90% per-token reduction is real — but only on cache hits. Write cost makes misses expensive.
• Large, stable blocks consistently outperform granular chunking across every documented workload type.
• The 60% production figure requires combining caching with batching. Neither alone gets you there.
• Batch API is the correct tool for low-frequency workloads — not a fallback when caching fails.
• Profile which prompt segments are actually stable before you architect anything.

Looking ahead into late 2026, Anthropic has been extending cache TTL options and expanding model support. Longer TTLs would shift the economics for medium-frequency workloads considerably — a 30-minute cache window would move batch processing from “not recommended” into genuinely viable territory.

The teams seeing the best outcomes right now aren’t just enabling caching. They’re profiling which prompt segments are stable, measuring hit rates per cache block, and splitting workloads across caching and batching based on request frequency.

What’s your current cache hit rate on production workloads? If you haven’t instrumented it yet, that’s the first place to look.

References

1. Prompt caching - Claude API Docs
2. Claude API Cost Optimization: Caching, Batching, and 60% Token Reduction in Production - DEV Communi

Photo by Bernd 📷 Dittrich on Unsplash