#Reuse stable prefixes

Prompt caching reuses the stable part of a request (system prompt, tool schemas, conversation prefix) so multi-step tool loops do not re-pay full input cost every turn.

Most providers either cache automatically or need only a sticky session key. Anthropic is the exception: it needs explicit cache breakpoints (or a gateway that inserts them).

#Provider matrix

What crabcode does today for each routing path:

Anthropic is the unfriendly one. Without cache_control (or Gateway caching: auto), Claude traffic does not cache-read. That was the main gap crabcode fixed.

#What gets marked (direct Anthropic)

Hybrid of OpenCode auto + Grok Build placement:

1. Last tool — tool schemas are large and stable across a tool loop
2. Last system block — instructions / project context
3. Transcript tip — last markable content block (skips thinking / redacted_thinking)
4. Previous user tip — where the prior request ended, so turns past the 20-block lookback still hit cache

At most 4 breakpoints; the 4th slot stays free when tools or previous-user are missing so gateways can auto-mark.

Breakpoints are applied after message regrouping so adjacent tool_use / tool_result blocks stay valid.

#xAI Grok Build (cli-chat-proxy)

In addition to sticky prompt_cache_key = session_id, crabcode stamps Grok Build–style affinity headers:

Parent-cached aux (e.g. max-steps text-only summary): keeps parent prompt_cache_key + session/conv so the conversation prefix can reuse the main turn's KV cache; assigns a fresh aux-… req id.

Subagents use the child session id on purpose. They have a different system prompt and tool set, so parent-prefix reuse would miss and can pollute sticky routing. Isolation beats a false shared key.

#Prefix stability (anti-bust)

• Empty system/user rows are dropped before the wire (they pad the sticky prefix).
• Images use hysteresis: compact only above a trigger (~6 MiB total data-urls), then reclaim to a lower target (~3 MiB) so later turns stay cache-warm instead of re-evicting every step.

#How to verify it works

Run with logs enabled and watch app.log for [prompt-cache] lines:

crabcode --emit-logs

Logging writes to app.log in the working directory when --emit-logs is set.

#Direct Anthropic

[prompt-cache] anthropic input=… output=… cache_read=… cache_creation=… total_input=… hit_pct=…

#AI Gateway / OpenAI-compatible

[prompt-cache] openai-compatible prompt=… completion=… cached_tokens=… cache_read=… cache_creation=… hit_pct=…

#OpenAI / xAI Responses

[prompt-cache] openai-responses input=… output=… cached_tokens=… hit_pct=…

#Healthy multi-step session

Notes:

• Anthropic's input_tokens is non-cached only. Total input ≈ input + cache_read + cache_creation.
• Gateway may surface cache as prompt_tokens_details.cached_tokens and/or forwarded Anthropic fields — crabcode logs both when present.
• No dashboard required: stream finals carry usage; crabcode logs them when logging is enabled.

#What we intentionally do not do

#Related

• AI Gateway automatic caching: Vercel docs
• Anthropic prompt caching: provider docs for cache_control / ephemeral