Response Caching
Exact-match response caching on nRouter: how hits are matched and metered, the x-nr-response-cache header, and how to bypass or disable the cache.
nRouter response caching eliminates redundant provider latency and spend by serving byte-identical non-streaming requests directly from edge cache at cache-read input rates. Stored responses return in single-digit milliseconds, settling at the model's exact upstream cache-read rate while preserving full telemetry across request headers and analytics.
How Matching Works
Matching is deterministic and byte-identical:
| Component | Match Behavior |
|---|---|
| Model & Parameters | model, temperature, top_p, max_tokens, seed, and tools must match byte-for-byte. |
| Payload Content | messages, prompt, or input strings must be identical down to whitespace. |
| Stripped Directives | Directives (nrouter_fallbacks, nrouter_guardrails) are stripped before hashing. |
Applies to buffered requests (stream: false). Provider errors and policy refusals are never cached, incurring $0 spend on blocked requests. Cache hits settle at upstream cache-read input rates with 0% platform markup.
Telemetry & Bypass Controls
| Scope | Mechanism | Telemetry Result |
|---|---|---|
| Cache Hit | Automatic match against edge cache. | Emits x-nr-response-cache: hit. |
| Cache Miss | First call stores verified response. | Emits x-nr-response-cache: miss. |
| Per-Request Bypass | Pass "nrouter_cache": false in body. | Emits x-nr-response-cache: bypass. |
| Org-Wide Disable | Toggle under Router Settings. | Bypasses cache across all keys. |
Next Steps
Frequently asked questions
Is the cache semantic? Will a similar prompt hit?
No, matching is strictly byte-identical; any reworded prompt forwards directly to the provider.
Is a cache hit free?
No, hits settle at the model's upstream cache-read rate reported in x-nr-request-cost.
How do I know a response came from the cache?
Inspect the x-nr-response-cache response header: hit, miss, or bypass.
Does caching apply to streaming requests?
No, default caching applies exclusively to buffered non-streaming requests.