Prompt Caching

Anthropic prompt caching cuts the bill on cache-eligible input tokens by ~90% on reads (cache hits) and +25% on writes (cache creation). Codeep enables it automatically for every Anthropic-protocol request — no config, no opt-in. In a typical agent session (5+ tool-call iterations against the same system prompt + skills catalog), this is a 60-90% net reduction on input billing.

★

Caching ships on in 2.0.2+. If you upgraded from 1.x, no action needed — every Anthropic call already cached. Verify with /costafter a few iterations (look for the “Prompt caching” section).

What gets cached

Two cache breakpoints per agent request, placed on the largest stable content blocks. The order matches Anthropic's cache prefix matching — we mark the boundary at the end of each cacheable region, and everything up to that point gets cached together.

Cache region	Includes	Typical size
`System prompt block`	Agent system prompt + project rules + project intelligence + skills catalog + active personality addendum	5-15K tokens (depends on project size + active skills)
`Tools array`	Every built-in tool definition + MCP server tools surfaced under <server>__<tool> namespace	2-8K tokens (depends on how many MCP servers are registered)

⚠

Anthropic requires ≥ 1024 input tokensin a cache block for it to actually cache (smaller blocks are silently skipped, no error). Tiny single-turn chats won't benefit. Agent runs and substantial system prompts always do.

Pricing math

Anthropic bills cache reads at 0.1× the base input rate and cache writes at 1.25×. So the first request in a window pays a small premium to populate the cache; every subsequent request inside the 5-minute TTL pays almost nothing on the cached portion.

// For Claude Opus 5 at $5/1M input rate

uncached_input  = N tokens × $5/1M             // baseline
cache_creation  = N tokens × $5/1M × 1.25     // first request (slight premium)
cache_read      = N tokens × $5/1M × 0.10     // every subsequent request (90% off)

// Realistic agent loop, 8 iterations, 10K-token system block + tools:
// Iteration 1: 10K write   →  10K × $5/1M × 1.25 = $0.0625
// Iterations 2-8: 10K read ×7 →  70K × $5/1M × 0.10 = $0.0350
// TOTAL: $0.0975  (vs $0.40 without caching → 76% savings)

How to verify savings

Run any agent task, then check /cost in the same session. After the second iteration, the “Prompt caching” section appears with token breakdown and dollar savings:

$ /cost
 
## Session Cost
Requests: 8  ·  Input: 142K  ·  Output: 6.2K  ·  Total: 148K
Estimated cost: $0.31
### Prompt caching
Cache reads: 105K tokens (billed at 0.1× input rate)
Cache writes: 12K tokens (billed at 1.25× input rate)
Estimated savings vs no caching: $0.45

OpenRouter routing

When you route to an Anthropic model via OpenRouter (e.g. anthropic/claude-opus-4), OpenRouter forwards the cache_control markers to Anthropic upstream — so caching still works through OpenRouter for Anthropic-served requests. Your OpenRouter invoice reflects the discounted billing.

Non-Anthropic models (GPT, Llama, Gemini through OpenRouter) don't use these markers. OpenAI applies automatic prefix caching server-side with no opt-in needed; other providers handle caching on their end too. Either way, no code change in Codeep — the cost report uses usage.cost from OpenRouter (authoritative per-call USD), so what you see is what you pay.

Cache TTL and warm-up

Detail	Behaviour
`TTL`	5 minutes from last cache read. Each hit refreshes.
`Warm-up cost`	First request pays 1.25× on the cached portion (cache_creation tokens).
`Cache breakage`	Any change to the cached prefix invalidates the cache — adding a new MCP server, changing the active personality, or editing project rules mid-session forces a fresh cache write on the next request.
`Cold start`	First request after a 5+ minute gap re-cycles cache creation cost.

★

For long sessions with intermittent activity, the cache often expires between iterations. You still save when you're working in bursts. For continuous agent loops (10+ iterations back-to-back), caching is near-pure win.

Other providers

OpenAI-compatible providers (OpenAI direct, Z.AI, DeepSeek, MiniMax, Ollama) don't expose explicit cache markers in their API. Codeep doesn't send cache_control to them — they apply automatic prefix caching server-side at their own discretion. Cost accounting works the same: /costuses each provider's reported token counts.

Ollama (local) doesn't bill, so caching is irrelevant for cost — but model inference can benefit from KV cache reuse, which Ollama handles internally.

Advanced: inspecting cache behaviour

Codeep's token tracker exposes per-call cache breakdown. If you're building integrations or debugging cost surprises, the relevant fields on every recorded API call are:

interface TokenUsage {
  promptTokens: number;        // total input including cached
  completionTokens: number;
  totalTokens: number;
  cacheCreationTokens?: number; // tokens written to cache this call
  cacheReadTokens?: number;     // tokens read from cache this call
}

The getCacheStats() helper in utils/tokenTracker.ts aggregates across all in-session calls and computes estimated savings (cache-read savings minus cache-write premium). Same logic feeds the /cost output.

Providers Project Rules