Streaming

Overview

• Set stream: true in the request
• Response is Server-Sent Events (SSE)
• Compatible with OpenAI SDK streaming, Vercel AI SDK, etc.
• Usage data included in the final chunk automatically (including usage.cost)
• For sansa-auto requests, the sansa routing metadata appears on the first chunk and again on the final usage-bearing chunk (see Models)

Quick Example

See the code panel for streaming examples in Python, TypeScript, curl, and raw fetch with SSE parsing.

SSE Format

Each event is a JSON object followed by a blank line.

Note: Line breaks added for readability. Actual events are single-line.

data: {
  "id": "...",
  "object": "chat.completion.chunk",
  "created": 1700000000,
  "model": "openai/gpt-5.4-mini",
  "choices": [{
    "index": 0,
    "delta": { "content": "Hello" },
    "finish_reason": null
  }]
}

Stream ends with:

data: [DONE]

The first chunk of a sansa-auto stream additionally carries the sansa routing object:

{
  "id": "...",
  "object": "chat.completion.chunk",
  "created": 1700000000,
  "model": "openai/gpt-5.4-mini",
  "choices": [{
    "index": 0,
    "delta": { "role": "assistant" },
    "finish_reason": null
  }],
  "sansa": {
    "routed": true,
    "routed_model": "openai/gpt-5.4-mini",
    "routing_latency_ms": 287
  }
}

The final chunk before [DONE] includes usage (with billed cost) and may also repeat the sansa routing fields:

{
  "id": "...",
  "model": "openai/gpt-5.4-mini",
  "choices": [{
    "index": 0,
    "delta": {},
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 84,
    "total_tokens": 96,
    "cost": 0.00081
  },
  "sansa": {
    "routed": true,
    "routed_model": "openai/gpt-5.4-mini",
    "routing_latency_ms": 287
  }
}

Streaming Chunk Shape

interface ChatCompletionChunk {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  
  // The underlying model that served the request.
  model: string;
  
  choices: {
    index: number;
    delta: {
      // Present in first chunk only.
      role?: "assistant";
      
      // Incremental text content.
      content?: string;
      
      // Incremental tool call data.
      tool_calls?: ToolCallChunk[];
      
      // Reasoning blocks (when enabled).
      reasoning_details?: ReasoningDetail[];
    };
    // Reason for stopping: "stop", "length", "tool_calls", or "error".
    finish_reason: string | null;
  }[];
  
  // Present in final chunk only.
  usage?: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
    // Total billed cost for this completion in USD.
    cost: number;
  };

  // Present on the first chunk of a sansa-auto stream, and again on the
  // final usage-bearing chunk. Omitted on direct model requests and on
  // content chunks in between.
  sansa?: {
    routed: boolean;
    routed_model: string | null;
    routing_latency_ms: number | null;
  };
}

Sansa always includes usage in the final streaming chunk, including usage.cost. The stream_options parameter is accepted for compatibility but ignored. See Models for the full shape of the sansa object.

Error Handling

Pre-Stream Errors

If validation fails before streaming starts (bad API key, invalid params, insufficient credits), a standard JSON error response is returned:

{
  "error": {
    "code": "insufficient_credits",
    "message": "Insufficient credits. Please add credits to continue."
  }
}

Mid-Stream Errors

If an error occurs after streaming has begun (HTTP 200 already sent), the error is sent as an SSE event:

data: {
  "error": {
    "code": "provider_error",
    "message": "Provider disconnected"
  },
  "choices": [{
    "finish_reason": "error"
  }]
}

data: [DONE]

Mid-stream errors always have finish_reason: "error" and are followed by [DONE].

Streaming with Tool Calls

When the model makes a tool call during streaming:

• delta.tool_calls appears incrementally
• id and function.name arrive in the first tool call chunk
• function.arguments streams as partial JSON fragments (must be accumulated)
• finish_reason is "tool_calls" in the final choice chunk

Streaming with Reasoning

When reasoning tokens are enabled:

• delta.reasoning_details appears before delta.content
• Reasoning blocks stream incrementally
• Content starts streaming after reasoning completes

Note: The OpenAI SDK does not currently auto-accumulate reasoning_details from streaming deltas, so reasoning continuity across turns is not available when streaming. If reasoning continuity matters (e.g., multi-turn tool calling), use non-streaming requests. See the Reasoning docs for details.

Streaming with Structured Outputs

Structured outputs (response_format: { type: "json_schema" }) work with streaming. The model streams valid partial JSON that forms a complete valid response when done.