> ## Documentation Index
> Fetch the complete documentation index at: https://docs.otamd.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming

> Read text incrementally and handle usage, completion, and interruption.

Set `stream: true` to receive server-sent events. The OpenAI SDK parses events for you.

## Chat completion stream

```javascript theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LIMITLESS_API_KEY,
  baseURL: process.env.LIMITLESS_BASE_URL,
  maxRetries: 0,
});

const stream = await client.chat.completions.create({
  model: process.env.LIMITLESS_MODEL,
  messages: [{ role: "user", content: "Write a short welcome message." }],
  max_tokens: 512,
  stream: true,
  stream_options: { include_usage: true },
});

try {
  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
    if (chunk.usage) console.log("\nUsage:", chunk.usage);
  }
} catch (error) {
  console.error("The stream did not finish:", error.message);
}
```

The gateway requests `include_usage: true` for all chat streams, even when the caller omits it or sets it to `false`. A usage chunk can have an empty `choices` array. Always check that a choice exists before reading its delta.

At the wire level, events are `data: {JSON}` records separated by a blank line. Chat streams normally terminate with `data: [DONE]`. Network chunks do not necessarily align with event boundaries; use the SDK or a proper SSE parser.

## DeepSeek reasoning deltas

With `reasoning_effort: "high"` (or another supported thinking level), DeepSeek sends reasoning in `choices[0].delta.reasoning_content` and the answer in `choices[0].delta.content`. Append each to a separate display. A chunk may contain only one of these fields, and reasoning can arrive before answer text. Both count toward the output-token budget.

The Playground displays these separately and retains received partial content after Stop. See [reasoning](https://docs.otamd.com/guides/reasoning.md) for JSON responses and conversation replay.

## Responses stream

The Responses endpoint emits typed events. See [stateless Responses](https://docs.otamd.com/guides/responses.md) for a complete example. Do not parse Responses events as chat `choices` chunks.

## Handle interruption

When the gateway observes your client disconnect, it cancels generation. The backend may already have processed part of it. Test cancellation through your deployed client and proxy path: closing the local stream alone does not prove that the disconnect reached the gateway.

* A final usage event may be absent after interruption. Missing token counts remain **unknown** in Activity.
* HTTP `200` indicates that a stream started; it does not prove the full generation completed.
* A failed stream can retain partial text. Decide whether your application should display, discard, or label it.
* Do not replay a partial request automatically without considering duplicate generation and charges.

For an observed customer Stop/disconnect, confirmed final usage is charged normally. If final usage is unavailable, the gateway makes no customer charge and leaves the token counts unknown. Unknown usage from timeouts, service failures or platform shutdown can still require reconciliation. See [billing](https://docs.otamd.com/billing.md). DeepSeek does not estimate a charge from the number of streamed events: final reported usage is charged; an observed cancellation without final usage has no customer charge. Closing a client does not guarantee that the engine stops immediately or that a request with reported usage is free.

The streaming examples use the official [OpenAI streaming interface](https://developers.openai.com/api/docs/guides/streaming-responses). Limitless endpoint coverage is described in [compatibility](https://docs.otamd.com/guides/compatibility.md).

## Messages and the console Playground

The [Messages endpoint](https://docs.otamd.com/guides/messages.md) streams native Anthropic events. Its event names and content-block structure differ from Chat and Responses; use the matching SDK.

The console Playground displays answer text and supported model reasoning as they arrive. **Stop** cancels the browser request and asks the backend to stop generation. Text already received stays visible and is marked as stopped or incomplete. Once the gateway observes the cancellation, it charges final reported usage or makes no customer charge if final usage is unavailable. Generation may take time to stop. Review Activity for the recorded outcome and Billing for the settled charge.

Response [policies](https://docs.otamd.com/guides/policies.md) can delay visible output while evaluation runs. This may make a policy-protected request look less incremental even though the client requested streaming.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.