Streaming
Stream Hull Gateway responses token by token with server-sent events.
Hull Gateway streams responses from every model using server-sent events (SSE), so your application can show tokens as they're generated.
Enabling streaming
Set stream: true in your request body to enable streaming:
{
"model": "anthropic/claude-sonnet-5",
"messages": [{ "role": "user", "content": "Write a short story" }],
"stream": true
}Hull Gateway sets stream_options.include_usage to true on every streaming Chat Completions request, so the final chunk always includes token usage.
Streaming with the Vercel AI SDK
The Vercel AI SDK parses the stream for you:
import { streamText } from "ai"
import { createOpenAI } from "@ai-sdk/openai"
const hull = createOpenAI({
baseURL: "https://api.withhull.com/api/gateway",
apiKey: process.env.HULL_API_KEY,
})
const result = streamText({
model: hull.chat("anthropic/claude-sonnet-5"),
prompt: "Write a short story about a robot.",
})
for await (const textPart of result.textStream) {
process.stdout.write(textPart)
}
// Access usage data after streaming completes
const usage = await result.usage
console.log("Tokens used:", usage)Streaming with the OpenAI SDK
import OpenAI from "openai"
const client = new OpenAI({
apiKey: process.env.HULL_API_KEY,
baseURL: "https://api.withhull.com/api/gateway",
})
const stream = await client.chat.completions.create({
model: "anthropic/claude-sonnet-5",
messages: [{ role: "user", content: "Write a short story" }],
stream: true,
})
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content
if (content) {
process.stdout.write(content)
}
}Raw SSE format
Each event is a JSON chunk prefixed with data: , and the stream ends with data: [DONE]:
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"role":"assistant","content":"Once"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"content":" upon"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"content":" a"},"finish_reason":null}]}
data: [DONE]Usage in the final chunk
Token usage data is included in the final chunk before [DONE], with an empty choices array:
{
"id": "chatcmpl-abc123",
"object": "chat.completion.chunk",
"usage": {
"prompt_tokens": 12,
"completion_tokens": 150,
"total_tokens": 162
},
"choices": []
}Stream cancellation
To cancel a stream, abort the connection:
const controller = new AbortController()
const response = await fetch("https://api.withhull.com/api/gateway/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.HULL_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "anthropic/claude-sonnet-5",
messages: [{ role: "user", content: "Write a long essay" }],
stream: true,
}),
signal: controller.signal,
})
// Cancel after 5 seconds
setTimeout(() => controller.abort(), 5000)What happens after you disconnect depends on the model provider. Some stop generating immediately; others may keep processing for a while. Usage is recorded for what was generated.
Error handling during streaming
Errors before streaming starts
If an error happens before any tokens are sent, you get a normal JSON error response with an HTTP status code, for example 402 when you have no usage available. See API reference.
Errors during streaming
Once tokens have been sent, the HTTP status (200) can't change. A provider error mid-stream arrives as an SSE event with an error object, or the stream ends without a normal finish_reason. Check for both in your client.
SSE clients
These libraries parse SSE streams:
- eventsource-parser: a lightweight SSE parser
- OpenAI SDK: built-in streaming support
- Vercel AI SDK: high-level streaming helpers