Code Generation
Coding assistants with model choice and per-seat budgets.
Pick the model per task from one catalog, stream completions token by token, block leaked secrets, and cap spend per developer seat.
The right model, fast, and within budget.
Speed for inline completion, depth for reasoning, and spend controls for every developer seat.
Model choice per task
Use a fast model for inline autocomplete and a reasoning model for refactors from one catalog.
- Responses stream as server-sent events, token by token
- nrouter/auto can pick a right-sized model for each request
Secrets blocked before egress
The Secret Scanner blocks supported credential formats in a prompt before the provider call.
- Provider API keys, cloud keys, GitHub, Slack and Stripe tokens
- Private keys, JWTs, and database connection passwords
- Part of the platform floor: no request or scope can turn it off
Per-seat budgets and 402 caps
Issue one virtual key per developer with a hard HTTP 402 ceiling and per-key RPM and TPM rate limits.
- Cost per seat is the key’s own spend, at exact list price
- A per-key RPM/TPM cap stops one seat hogging shared capacity
Every generation logged
Filter the request log by a developer key to inspect model, latency, tokens, and exact cost per call.
Explore observabilitySet the model per request.
Point your editor extension or coding tool at the OpenAI-compatible gateway endpoint.
pip install openai| 1 | # Cache: enabled (org default). Pass nrouter_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 |
|
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NROUTER_API_KEY"], |
| 7 | base_url="https://api.nrouter.ai/v1", |
| 8 | ) |
| 9 |
|
| 10 | response = client.chat.completions.create( |
| 11 | model="gpt-5.4-mini", |
| 12 | temperature=1, |
| 13 | max_completion_tokens=1024, |
| 14 | messages=[ |
| 15 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 16 | ], |
| 17 | extra_body={ |
| 18 | # "nrouter_cache": False, # Uncomment to skip cache |
| 19 | }, |
| 20 | ) |
| 21 |
|
| 22 | print(response.choices[0].message.content) |
FAQs
Can I use different models for autocomplete versus deep code reasoning?
Yes. The model catalog exposes every model behind one key. Set the model per request (a fast model for inline completion, a stronger reasoning model for refactors or test generation) with no separate provider account or SDK swap. Smart routing (nrouter/auto) can also automate this selection, sending each request to a right-sized model instead of defaulting everything to your most expensive one.
How does nRouter keep code completion latency low?
Routing decisions happen in-memory in the gateway, before the request is dispatched — LLM inference time dominates the total. Your configured routing order decides the primary model, failover moves to the next provider when a trigger fires, and streaming responses pass through token by token as they are generated.
Can I budget LLM spend per developer seat?
Yes. Issue one virtual key per developer or per engineering team and set a hard budget ceiling. The moment spend would exceed the cap, the gateway returns HTTP 402. Spend, rate limits, and the request log all scope to that key, so engineering leaders have total visibility into cost per seat.
How do inline guardrails protect proprietary code and system credentials?
Before prompts reach commercial foundation models, inline guardrail filters inspect code payloads for embedded API keys, private certificates, connection strings, and sensitive credentials. The platform Secret Scanner blocks known credential formats in prompts and responses, so a pasted key never reaches an external provider log.
Does streaming work for token-by-token completion in IDE extensions?
Yes. nRouter is OpenAI-compatible and streams responses as server-sent events, so editor extensions (VS Code, JetBrains, Cursor, Neovim) get tokens as they are generated. If the primary provider throttles or sends no first byte in time, the next model in your fallback list answers.