Skip to content
Copyright © 2022-2026 basebox GmbH, all rights reserved.
Licensed to be used in conjunction with basebox, only.

// integration

Streaming

Applies to

Product: Cloud · Server · Audience: Developer / Integrator

How basebox streams responses over the OpenAI-compatible API (Server-Sent Events), what a client has to handle and how reasoning content arrives. Streaming is the norm for interactive applications: the first tokens appear within a fraction of a second instead of only after the complete answer.

Requesting a stream

Set "stream": true in the request:

curl -N -X POST "$BASEBOX_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASEBOX_API_TOKEN" \
  -H "X-Realm: $BASEBOX_REALM" \
  -d '{
    "model": "<model-id>",
    "messages": [{"role": "user", "content": "Count to 10 slowly"}],
    "stream": true
  }'

-N disables curl's output buffering so you see the chunks immediately.

The format

The response is a text/event-stream. Each event is a line data: followed by a JSON object of type chat.completion.chunk; events are separated by blank lines. The stream ends with data: [DONE].

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"1"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" 2"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

What a client has to do:

  1. Read line by line; ignore lines without the data: prefix and blank lines.
  2. Detect [DONE] and close the stream.
  3. Parse every other payload as JSON and evaluate choices[0].delta.
  4. Append delta.content in order of arrival – that builds the answer text.
  5. Watch finish_reason: stop (finished normally), length (output limit reached), tool_calls (the model calls tools).

Reasoning content

With models that support reasoning and an active thinking mode, the stream delivers the train of thought separately from the answer text: it arrives in a dedicated delta field before the actual content tokens begin. Treat it as the basebox interface does – as a collapsible intermediate step, not as part of the answer. Store or show it only if your use case requires it.

More reasoning means more tokens and a longer time to the first answer token; plan your timeouts accordingly. Background: Reasoning effort.

Tool calling in the stream

When the model calls functions, the calls arrive as delta.tool_calls in fragments: first name and ID, then the arguments piece by piece as a JSON string. Collect the fragments per index until finish_reason: "tool_calls" appears, execute the function and send the result as a message with role: "tool" in the next request.

With the OpenAI SDK

stream = client.chat.completions.create(
    model=model_id,
    messages=[{"role": "user", "content": "Count to 10 slowly"}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)
print()

The SDK takes care of parsing the events and detecting [DONE].

Connections and timeouts

  • Long connections are expected. The basebox ingress is set to a 24-hour read/send timeout and does not buffer streaming responses.
  • Check your own proxies. A reverse proxy or corporate proxy between your client and basebox that buffers responses turns the stream into a block response. Disable buffering for this path.
  • Set client timeouts generously – reasoning models and long answers take time. Prefer an idle timeout (time without a new chunk) over a total timeout.
  • No resumption. If a stream breaks, there is no continuation from the last position; send the request again.
  • Closing the connection = cancel. End the stream on the client side when users stop the answer; that saves tokens.

Errors in the stream

Errors before the first chunk arrive as a normal HTTP error response (401, 403, 400, 404) – see Error handling. If the stream breaks after that, you usually see an abrupt end of connection without [DONE]; treat a stream without [DONE] as incomplete.

Common problems

Symptom Cause Solution
Answer arrives all at once at the end Buffering (curl without -N, proxy, framework) Disable buffering
Stream breaks after a fixed number of seconds Client or proxy timeout Increase timeouts; idle timeout instead of total timeout
Text contains the train of thought Reasoning delta treated as content Evaluate fields separately
finish_reason: "length" Model output limit reached Request a shorter answer or split the request
Empty deltas Role/metadata chunks Ignore, append only content

Next: Error handling · User guide