AI Agents
Agents that stay up and stay in budget.
Keep multi-step agent runs alive when a provider degrades, cap each agent with a per-key budget, and log every tool call.
Reliability, spend control, and a trail.
A run that survives a provider blip, a budget it cannot exceed, and a log of every step.
Auto-failover for long runs
If a provider cannot take a step, the next model in your fallback list answers.
- Triggers: 429, 503, a connection that never opened, no first byte in time
- A 500, 502 or 504 is not replayed: the provider may already have billed
- x-nr-routing and x-nr-attempts show every failover on the response
Hard 402 budget ceilings
Assign one virtual key per agent with an isolated budget. Runaway loops stop at HTTP 402.
- Credits are held before each call and settled at the real cost after
- Per-key RPM/TPM ceilings stop one agent from starving the rest
Tool-call audit ledger
Planning, tool selection, and reflection calls log model, latency, tokens, and exact list-price cost.
- Tag requests agent:<id> to fill the Agents spend report
- Filter the request log by the agent’s key to review a whole run
Step-level model tiering
Route tool-argument extraction to smaller models while reserving frontier reasoning models for planning.
Browse model catalogPoint your agent framework at one endpoint.
Configure LangGraph, CrewAI, AutoGen, or OpenAI SDK clients with an agent-scoped virtual key.
pip install openai| 1 | # Cache: enabled (org default). Pass nrouter_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 |
|
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NROUTER_API_KEY"], |
| 7 | base_url="https://api.nrouter.ai/v1", |
| 8 | ) |
| 9 |
|
| 10 | response = client.chat.completions.create( |
| 11 | model="gpt-5.4-mini", |
| 12 | temperature=1, |
| 13 | max_completion_tokens=1024, |
| 14 | messages=[ |
| 15 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 16 | ], |
| 17 | extra_body={ |
| 18 | # "nrouter_cache": False, # Uncomment to skip cache |
| 19 | }, |
| 20 | ) |
| 21 |
|
| 22 | print(response.choices[0].message.content) |
FAQs
Why do autonomous AI agents require an LLM gateway?
Autonomous agents fan out dozens or hundreds of sequential LLM calls per goal: planning, tool evaluation, schema reflection, and final synthesis. A single provider rate limit or timeout aborts the entire multi-step run. nRouter provides multi-provider failover, hard spending limits, and a request log to make agents reliable, cost-bounded, and auditable.
How does nRouter prevent runaway spend during infinite agent loops?
Each autonomous agent runs with an isolated virtual key assigned a strict dollar ceiling. The moment an agent’s cumulative spend meets its cap, the gateway rejects subsequent calls with HTTP 402 Payment Required. Credit holds before dispatch prevent balances from ever going negative.
How does step-level model tiering reduce autonomous agent costs?
Not every step in an agent trajectory requires an expensive frontier reasoning model. nRouter enables agents to route JSON extraction, tool argument parsing, and interim checks to smaller models, reserving frontier models for high-level goal decomposition.
How do inline guardrails protect agents against prompt injection and malicious tools?
As agents execute tools and retrieve untrusted external inputs (web scrapers, SQL queries, APIs), inline guardrails detect and neutralize indirect prompt injections before they can hijack the agent’s execution loop. System prompts, API keys, and sensitive payloads are shielded from prompt exfiltration.
Do popular agent frameworks (LangChain, AutoGen, CrewAI) integrate out of the box?
Yes. Because nRouter provides an OpenAI-compatible endpoint, any agent framework (LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, OpenAI Swarm) works by setting the baseURL to https://api.nrouter.ai/v1 and supplying your nRouter key. Function calling, streaming, and tool execution are supported natively.