AI Agent Cost & Tools HubPrompt caching explained
In one sentence: if the start of your prompt is the same every time, the provider can remember it and charge you much less to read it again.
Why it matters for agents
An AI agent sends the same long instructions, tool descriptions and conversation history on every step. Without caching you pay full price for that repeated text each time. With caching, the repeated part is billed at the cache-read price, which is often 90% cheaper. For example, Claude Sonnet 5.5 costs $2 per million normal input tokens but $0.1 per million cached tokens.
How it works
- Put everything that doesn't change (system prompt, tool definitions, reference documents) at the start of the prompt.
- Put the changing parts (the user's question, the latest tool result) at the end.
- The provider caches the matching start. Some providers do this automatically (e.g. OpenAI, DeepSeek, Gemini); others need you to mark it (e.g. Anthropic's
cache_control). - Caches expire after a few minutes without use, so caching helps most when calls come in regularly.
Some providers also charge a little extra to write the cache the first time. Our calculator and the table below only count cache reads, so treat the savings as a best case.
How much you can save
Same example agent as in our monthly cost guide: 500 tasks/day, 6 steps per task, 3,000 input and 400 output tokens per step. "With caching" assumes 70% of input tokens are cache hits.
| Model | No caching | 70% cached | Saving |
|---|---|---|---|
| Claude Haiku 5.5 | $45.00 | $27.99 | 38% |
| DeepSeek V4.1 Flash | $124 | $68.63 | 45% |
| Mistral Large 4 | $265 | $146 | 45% |
| Gemini 3.8 Flash | $338 | $211 | 37% |
| GPT-6.1 Terra | $450 | $270 | 40% |
| Grok 4.7 | $756 | $473 | 38% |
| GPT-6.1 Sol | $900 | $541 | 40% |
| Claude Sonnet 5.5 | $900 | $541 | 40% |
| Gemini 4 Argon | $900 | $541 | 40% |
| Claude Opus 5.5 | $1,800 | $1,082 | 40% |
The saving is lower when output tokens are a big part of the bill, because caching only discounts input.
When caching does not help
- Every prompt is different from the first word (e.g. you put a timestamp at the top).
- Very short prompts. Many providers only cache prompts above about 1,000 tokens.
- Calls are hours apart, so the cache has expired.
More ways to cut spend: How to reduce LLM token costs.
Try it with your own numbers: the free AI agent cost calculator shows the monthly cost for 10 popular models in seconds.
Want a step-by-step method? The AI Agent Cost Playbook (29-page PDF + spreadsheet, $29 + tax) walks you through cutting your bill in one afternoon. Free sample.
Prices are list prices per million tokens from our pricing table (last refresh 2026-11-01); check providers before you commit. This guide was written with help from AI tools and checked by a human.