Use Cases
AI use cases on one managed gateway.
RAG pipelines, AI agents, support bots, coding assistants, document processors, and voice apps behind one key.
Six use cases, one gateway.
Pick a workload to see which nRouter capabilities apply and how requests flow.
RAG pipelines
Route embeddings and chat synthesis through one key, cache exact repeat queries, and track cost per stage.AI agents
Keep multi-step runs alive with provider failover, cap spend per agent with 402 budgets, and log every call.Customer support bots
Redact customer PII and block prompt injection inline, version tone in templates, and cap bot spend.Code generation
Pick fast models for autocomplete and reasoning models for refactors, block leaked secrets, cap each seat.Document processing
Scope keys to vision-capable models, pace batch loops against rate limits, and track cost per job.Voice & realtime AI
Stream conversational turns to your speech layer with mid-call failover and turn-level logs.
What every use case shares.
The workloads differ, but the gateway controls underneath stay the same.
One key, no provider config
Use one nRouter virtual key and the OpenAI-compatible endpoint across the catalog with no separate SDKs.
Explore virtual keysGuardrails and budgets, always on
Inline PII redaction, prompt-injection defense, and hard HTTP 402 budgets apply to every workload.
Explore guardrailsCost and logs you can attribute
Every call logs exact list-price cost, model, latency, and tokens across stages, agents, jobs, or seats.
Explore observabilityOne client setup.
Point standard OpenAI clients at the gateway endpoint with one virtual key.
pip install openai| 1 | # Cache: enabled (org default). Pass nrouter_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 |
|
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NROUTER_API_KEY"], |
| 7 | base_url="https://api.nrouter.ai/v1", |
| 8 | ) |
| 9 |
|
| 10 | response = client.chat.completions.create( |
| 11 | model="gpt-5.4-mini", |
| 12 | temperature=1, |
| 13 | max_completion_tokens=1024, |
| 14 | messages=[ |
| 15 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 16 | ], |
| 17 | extra_body={ |
| 18 | # "nrouter_cache": False, # Uncomment to skip cache |
| 19 | }, |
| 20 | ) |
| 21 |
|
| 22 | print(response.choices[0].message.content) |
FAQs
How does nRouter support multiple AI workloads behind a single key?
nRouter provides a unified OpenAI-compatible endpoint that serves chat completions, streaming, embeddings, vision, and audio models across leading foundation providers. You configure your client once with an sk-nrouter-* key, and your entire stack (agents, RAG pipelines, support bots, and coding tools) can access any supported model without separate provider accounts or SDK swaps.
How does smart tier routing reduce operating costs across use cases?
With nrouter/auto, the gateway reads each request’s task type and picks a model tier that fits it, under the quality or cost strategy you choose. Routine extraction and classification can go to smaller models while complex reasoning goes to frontier models, and every request still settles at the served model’s exact list price. You can also name the model yourself on any request.
What happens if a foundation model provider experiences an outage during production traffic?
If the provider returns 429 or 503, never opens a connection, or sends no first byte in time, nRouter moves the request to the next model in your fallback list. A 500, 502, 504, or a failure after response bytes arrive is not replayed, because the provider may already have billed for that work.
How do inline guardrails protect production applications from data leaks and prompt injections?
Every request passes through an inline guardrail inspection layer before payload egress. It redacts personal data such as SSNs and card numbers, blocks known credential formats, and blocks jailbreaks and prompt-injection attempts before they reach foundation models. Redacted events and blocked attempts are captured in security audit logs.
Can I enforce distinct budgets and rate limits per workload, team, or agent?
Yes. Limits apply at the organization, team, and key level, and you can issue one virtual key per workload or agent. You can assign hard or soft budget envelopes (with automatic HTTP 402 rejection when depleted) and configure distinct requests-per-minute (RPM) and tokens-per-minute (TPM) limits to prevent any single workload from exhausting shared account capacity.
Does nRouter add noticeable latency to streaming or interactive voice applications?
Routing decisions run in memory before dispatch, so model inference dominates the time. With stream: true the gateway opens the connection immediately and sends the answer as server-sent events, which suits voice bots, IDE completions, and realtime conversational agents.