Budgets: Decide where spending stops.
You set per-team LLM spend budgets by creating limits at organization, team and API-key level, each with its own reset period. When a limit is reached, the gateway blocks the request before the model runs — or warns or throttles instead, if you choose that. Calls already in progress count against the budget.
Watch a budget decide.
How budgets work. Checked before the model runs.
Organization, team, and API key limits.
Set organization and team budgets in one place, and assign separate spending limits to individual API keys.
- Owners and Organization Admins manage organization and team budgets
- Per-key spending limits are configured separately on the Keys page
- A tighter parent Block or Throttle budget stops requests first
Count calls in progress.
Checks count what is spent, reserved for calls in progress, and needed for this request under one lock.
- Block mode refuses over-limit calls with HTTP 402 and code budget_exceeded
- Throttle mode refuses with HTTP 429; Warn mode returns HTTP 200 with a header
- Refused requests stop before reservation and cost $0.00
Choose a reset period.
Set daily, weekly, monthly, or fiscal-year reset periods, or choose a non-resetting lifetime limit.
- Window boundaries use one server clock, not the client’s
- Allowed calls settle at the served model list price, 0% platform fee
- Corrupt or unreadable budget records fail closed
Three budget behaviors: Block, Throttle, and Warn.
| Capability | Refuse the request (Block or Throttle)Stops the call before the model runs with $0.00 reserved or charged. | Allow spending to continue (Warn)Runs the model, settles usage at list price, and signals the caller in headers. |
|---|---|---|
| Block mode | stops the request with HTTP 402 Payment Required, code budget_exceeded, and x-nr-limit-source: budget | allows requests to continue past the limit and returns HTTP 200 OK with x-nr-budget-warning |
| Throttle mode | refuses the request with HTTP 429 Too Many Requests and x-nr-limit-source: budget without a computed retry time or queueing | stamps x-nr-budget-warning: <scope> soft_budget <spend>/<max_budget> when an allowed call crosses a soft threshold |
| Financial outcome | $0.00 reserved or charged because the request is refused before reservation and model execution | reserves credits as usual and settles the served model at its published list price (0% platform fee) |
| Fail-closed check | if a budget record is corrupt or cannot be evaluated, the gateway refuses the request | applications can inspect x-nr-budget-warning and x-nr-request-cost on every allowed response |
FAQs
What happens when a budget is reached?
It depends on the budget behavior you choose. Block mode refuses the request before credit reservation or model execution and returns HTTP 402 Payment Required with code budget_exceeded and x-nr-limit-source: budget. Throttle mode also refuses the request before model execution and returns HTTP 429 Too Many Requests with x-nr-limit-source: budget; it does not queue traffic or compute a Retry-After header. Warn mode allows the request to run and stamps x-nr-budget-warning on the HTTP 200 response. When Block or Throttle refuses a request, no credit hold was taken, so there is nothing to release and $0.00 is reserved or charged.
Who can create and manage budgets, and where are API key limits set?
Owners and Organization Admins create and edit budgets on the Budgets page, which supports organization, team, and admin-managed user-scoped ceilings. Organization Members and Viewers cannot create self-managed personal budgets; members receive read-only visibility into their own and their team’s spend. Spending limits (max_budget) and optional RPM or TPM rate limits for individual virtual keys are configured separately on the Keys page by Owners, Organization Admins, or Team Admins for keys in their assigned team.
Why can a request be refused when my displayed spend is still below the limit?
A snapshot of past settled spend is not enough to protect a shared limit under concurrent traffic. During atomic reservation, the gateway sums what is already spent in the current window, what is currently reserved for calls in progress, and the upfront reservation needed for the incoming request under one row lock. Because the upfront reservation for a call (or the highest-rate candidate in a routing chain) can exceed its final settled cost, a request is refused if that combined total would exceed a Block or Throttle ceiling.
How do overlapping organization, team, and API key limits interact?
On the Budgets page, a team-scoped (or admin-managed user-scoped) budget cap cannot be saved above the organization’s budget cap for a matching or longer reset period. On the Keys page, a virtual key’s max_budget is saved as entered even if it exceeds an organization or team budget; the dashboard displays an advisory notice that a tighter parent Block or Throttle budget will stop requests first, whereas a parent Warn budget allows requests to continue up to the key’s own limit. Every applicable limit is checked on each request, and refusal messages identify the specific key or named scope that stopped the call.
Which reset periods are available, and when do budgets reset?
Budgets support four reset periods—daily, weekly (Monday-anchored), monthly, and fiscal-year (anchored to your configured fiscal start month)—plus a non-resetting lifetime limit. Window boundaries are calculated on the database clock so dashboard spend totals and gateway checks share the exact same boundary.
How does settlement work on allowed requests, and how do I resume traffic after a refusal?
When a request passes all budget checks, the gateway places an atomic credit reservation, calls the model, settles the served model’s published list price (with a 0% platform fee) while releasing any unused reservation in the ledger, and then reports the settled charge in the x-nr-request-cost response header. Corrupt or unreadable budget records fail closed. After a Block or Throttle refusal, traffic resumes as soon as an Owner or Organization Admin raises the ceiling, switches the budget behavior to Warn, or the current reset period rolls over (or a Team Admin raises a team key’s limit when the key limit was reached).
- $0.00 reserved or charged applies to requests refused by a budget check (Block HTTP 402 or Throttle HTTP 429) before credit reservation and model execution. Requests allowed under Warn mode or below a limit reserve credits and settle at the served model published list price with a 0% platform fee.
- Organization and team budgets are created and edited by Owners and Organization Admins on the Budgets page, while per-key spending limits live on virtual keys on the Keys page. Setting a key limit above an organization or team Block or Throttle budget saves with an advisory notice that the tighter parent limit stops requests first.
- Budget windows support daily, weekly, monthly, and fiscal-year reset periods plus non-resetting lifetime limits, evaluated on the database clock. Each check sums settled spend in the current window, active reservations for calls in progress, and the upfront reservation needed for the incoming request; corrupt or unreadable budget records fail closed.