Hi @valentinoPereira, great question.
Each chat request, each tool call & each follow‑up message is its own API call to the model, and for every one of those calls we have to send the full chat context so the agent has everything it needs. We cannot just send “only the diff.”
Prompt caching does not change what we send, it changes how the reused part is billed:
- The first time you send a big chunk of context, those tokens are billed as a cache write (slightly more than normal input).
- When you reuse the exact same context on later calls, those tokens are billed as cache reads (much cheaper than normal input).
- Anything new you add on each step (your latest message, new tool results, etc.) is still billed as normal input.
- Output tokens are always billed as normal output, regardless of caching,
Using Opus 4.6 numbers per 1M tokens (standard context tier):
- Input: $5.00
- Output: $25.00
- Cache write: $6.25
- Cache read: $0.50
Simple example (including output)
Say you have a 1M‑token context that you reuse across 20 steps (tool calls + replies), and each response is about 100k (0.1M) of output:
Without caching
- Context input per step: 1.0M × $5.00 = $5.00
- Output per step: 0.1M × $25.00 = $2.50
- Total per step: $7.50
- 20 steps: 20 × $7.50 = $150
With caching
-
1st step:
- Cache write for 1.0M context tokens: 1.0M × $6.25 = $6.25
- Output: 0.1M × $25.00 = $2.50
- Step 1 total: $8.75
-
Next 19 steps:
- Cache read for 1.0M context tokens: 1.0M × $0.50 = $0.50
- Output: 0.1M × $25.00 = $2.50
- Per step: $3.00
- 19 steps: 19 × $3.00 = $57.00
Total with caching over 20 steps:
- Step 1: $8.75
- Steps 2–20: $57.00
- Overall: $65.75
So for these 20 steps you go from $150 without caching down to $65.75 with caching. Most of that saving comes from those 1M context tokens switching from $5.00 per 1M (input) to $0.50 per 1M (cache reads) on every step after the first. That is why cache reads and writes appear as separate, chargeable items: they are different price classes for the same context tokens that still have to be sent on every request.
Overall caching reduces processing cost up to 90% while giving same output quality and full context for request like processing all tokens as input would do.
This is a feature we use on your behalf to reduce token cost within each chat thread and for all providers who use caching. Note that some providers include cache write into Input tokens and only charge for cache read while some like Anthropic separate cache writes. In all cases we show the precise token consumed in Usage report as provided by AI provider sent back with AI response.