Cutting our Claude API bill 10x with prompt caching

Our AI support assistant at SealMyIdea runs a system prompt, a full knowledge base, and 13 tool definitions on every single turn of every conversation. Without caching, that's thousands of tokens re-billed on every message — even when nothing in the context has changed.

What actually gets cached

Anthropic's prompt caching lets you mark a prefix of the request as cacheable. We cache three things, in this order:

1. The system prompt (rules, tone, escalation policy)
2. The knowledge base snapshot (39 articles, serialized)
3. The 13 tool definitions (JSON Schema, converted from Zod)

Only the user's actual message and the running conversation history sit outside the cached prefix.

The TTL tradeoff

We use a 1-hour cache TTL. Shorter TTLs mean more cache misses during quiet periods; longer TTLs risk serving a stale knowledge base after an admin edits an article — so every knowledge-base write explicitly invalidates the cache instead of waiting for TTL expiry.

The number that mattered

Across 578 measured production conversations, blended cost landed at $0.016 per conversation — roughly a 10x reduction from the uncached baseline. The lesson: prompt caching pays off fastest when your system prompt and tools are large relative to the actual user message, which is almost always true for a tool-calling agent.