Your Prompt Cache Isn't Working and You're Paying Full Price

Bottom Line

• Under ~1024 tokens, the cached prefix silently does not cache. No error, no warning.
• Maximum 4 cache_control breakpoints per request. More and it stops behaving.
• Caches are per-model. Change models, start cold.
• Everything before the breakpoint must be byte-identical. One timestamp ruins it.
• Check usage, not your intuition. cache_read_input_tokens is the only number that settles it.

First, Get the Actual Numbers

Do not reason about this. Read it off the response. Every response carries a usage object:

"usage": {
  "input_tokens": 41,
  "cache_creation_input_tokens": 3187,
  "cache_read_input_tokens": 0,
  "output_tokens": 216
}

Three states, three meanings:

The third row is the one people report as “caching doesn’t work,” and it is the informative case: it means the request never qualified in the first place.

Cause One: The Prefix Is Too Short

This is the big one. The cacheable prefix has to reach roughly 1,024 tokens. Below that, the cache silently declines to store anything. You get no error, no warning field, and no indication that the cache_control block you carefully added did nothing at all.

A system prompt that feels substantial when you read it — a couple of paragraphs of instructions, a few rules — is often 300 to 600 tokens. It looks cacheable. It is not.

Count before you debug anything else:

count = client.messages.count_tokens(
    model="claude-sonnet-5",
    system=SYSTEM_PROMPT,
    messages=[{"role": "user", "content": "x"}],
)
print(count.input_tokens)

If that is under about 1,024, you have found your bug and none of the rest of this page applies.

The fix is not to pad the prompt with filler. It is to move more real content in front of the breakpoint: tool definitions, schemas, reference documents, examples, coding standards, the API surface the model needs. Content that is genuinely stable across calls and genuinely useful. Most applications have plenty of it sitting after the breakpoint for no reason.

Cause Two: Too Many Breakpoints

There is a limit of four cache_control breakpoints per request. Exceeding it does not produce the behavior you want.

Four is more than almost anyone needs. A sensible layout is:

1. Tool definitions
2. The stable system prompt
3. Long-lived context (documents, schemas)

Then everything variable — the user’s actual message, retrieved chunks, timestamps — goes after, uncached. If you find yourself wanting a fifth breakpoint, the structure is probably wrong rather than the limit.

Cause Three: The Prefix Is Not Identical

Cache matching is on an exact prefix. Anything that varies before a breakpoint means a different prefix, which means a miss, every single time, forever.

The classics:

• A timestamp in the system prompt. "Current date: 2026-08-19T14:22:07Z" at the top. Every call is unique. Move it into the user message.
• A session or user id in the preamble. Same problem.
• Tool definitions built from a dict whose iteration order is not stable across processes.
• Trailing whitespace that varies with how the string was assembled.
• A changed model. Caches are per-model; a fallback to a different model reads cold.

A quick way to catch all of these at once: hash your prefix and log it.

import hashlib, json
prefix_repr = json.dumps({"tools": tools, "system": system}, sort_keys=True)
print(hashlib.sha256(prefix_repr.encode()).hexdigest()[:12])

Two consecutive calls that should share a cache must print the same hash. If they do not, you now know precisely which component is moving.

The Expensive Version of This Bug

Caching failures are worth chasing because of how they compound in agent loops rather than how they look in a single call.

An agent turn resends the entire accumulated conversation. If the front of that is cached, you pay a fraction for it. If it is not, you pay full input price for the whole history on every turn, and the history grows monotonically. A twenty-turn session with a broken cache does not cost twenty times a single call — it costs closer to the sum of a growing series.

Retries make it worse. A model refusal or an error that triggers a retry pays a cold cache on the retry, so the failure path is more expensive per attempt than the success path. If something in your setup retries aggressively, the bill is dominated by the calls that did not work.

The Verification Loop

Once you have made a change, prove it rather than assuming:

for i in range(3):
    r = client.messages.create(...)   # identical prefix each time
    u = r.usage
    print(i, u.cache_creation_input_tokens, u.cache_read_input_tokens)

Expected output is a write on the first call and reads afterwards:

0 3187 0
1 0 3187
2 0 3187

If call 1 shows another creation instead of a read, your prefix is changing between calls — go back to the hash check. If everything stays at zero, go back to the length check.

Related

• Fix the claude -p $1,800 API Bill Trap
• Why Claude Code’s Context Fills Up