Stream Responses
Stream a model's answer token by token: set stream to true and read server-sent events, with end-of-stream billing on the tokens actually emitted.
Streaming delivers tokens incrementally via server-sent events (SSE), minimizing time to first byte (TTFB) while settling usage at exact list prices on completion.
Implementation
Enable streaming by setting "stream": true on Chat Completions or Responses:
from nroutersdk import nRouter
client = nRouter()
stream = client.chat.completions.create(
model="gpt-5.4-mini",
messages=[{"role": "user", "content": "Explain quantum computing briefly."}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)TypeScript uses for await (Python SDK). Each chunk arrives as a data: line, ending with data: [DONE].
Telemetry & Guides
- TTFB Latency:
x-nr-latency-msreports edge ingress to initial SSE header flush. - Settlement & Model: Tokens settle atomically at list price at end-of-stream; the served model appears in
x-nr-modeland per chunk undernrouter/auto. - Related Guides: Tool Calling, Structured Output, and Reasoning Models.
Frequently asked questions
Does streaming change what a request costs?
No; requests settle at the served model's exact list price based on tokens actually emitted at stream termination.
Does streaming work with nrouter/auto and fallbacks?
Yes; routing operates normally, and each stream chunk identifies the serving model under nrouter/auto.
How do I know the stream is finished?
The completion emits a chunk with a terminal finish_reason followed by a data: [DONE] event.