You switch on streaming, send a request to GLM, and watch an empty chat window. Several seconds later, the entire answer appears in one jump. The API returned successfully, yet the feature your customer actually needed—visible progress—never happened.
For a developer building a support assistant or an internal coding tool, this is more useful to investigate than a general model speed ranking. You need to know where incremental output stopped being incremental: at the model, at the API gateway, inside your backend, or in the browser.
A GLM API relay is part of that delivery path. Choosing one should involve observing the stream your application receives, not merely checking that a model name appears in a catalog. This guide follows a hypothetical investigation and gives you a repeatable way to locate the delay. The example observations are illustrations, not a benchmark of GLM or any gateway.
The incident: a working API with a frozen interface
Imagine a team building a customer-support drafting assistant. A support agent selects a ticket, clicks “Draft reply,” and expects the opening sentence to appear while the rest is generated. The backend sends a GLM request through a gateway. Short replies look acceptable; longer replies leave the screen blank.
The first proposed fix is to change models. That may alter generation speed, but it cannot establish whether the existing output was held in a buffer. Start by keeping the model, prompt, and request settings fixed. Change one delivery boundary at a time.
If you are still deciding which model family suits the application, the existing Chinese LLM API selection guide covers that earlier decision. Here, assume GLM is already the intended model and investigate the stream itself.
Four clocks tell four different stories
“Time to first token” is often used too loosely in application dashboards. Your program may measure the first HTTP header, the first network chunk, the first parsed event, or the first answer character. Those are different milestones.
Record four timestamps using a monotonic clock in the receiving process: request started, first complete SSE event parsed, first nonempty answer delta observed, and completion recognized. If possible, separately record response headers and first body bytes. A heartbeat or metadata event can arrive before any answer text.
| Observation | What it establishes | What it does not establish |
|---|---|---|
| Headers arrive early | A response has begun | The model has produced an answer |
| Events arrive regularly | This receiving layer sees incremental events | The browser renders them immediately |
| Reasoning fields appear first | The response contains an earlier reasoning phase | Answer text was buffered |
| Text arrives in one burst | This receiving layer receives text in a burst | Which upstream component caused it |
Keep measurements from different machines separate unless their clocks are synchronized. For comparisons across services, durations within each process are often easier to interpret than subtracting two unrelated wall-clock timestamps.
First observation: inspect the gateway response directly
Use a synthetic prompt that requests several paragraphs. Avoid customer records while collecting diagnostic output. Select the exact GLM model identifier enabled for your account and copy the complete Chat Completions URL from your gateway documentation.
The Treerouter endpoint configuration reference helps distinguish a base URL from an operation URL. The following command expects a complete operation URL. It deliberately leaves the model identifier configurable so the probe does not depend on a particular release remaining available.
# Required: GLM_CHAT_URL, GLM_MODEL, TREEROUTER_API_KEY; jq and curl.
jq -n --arg model "$GLM_MODEL" '{
model: $model,
stream: true,
messages: [{role: "user", content:
"Explain five practical uses of an API. Give each use a short paragraph."
}]
}' > glm-stream-probe.json
curl --no-buffer --fail-with-body --max-time 120 \
"$GLM_CHAT_URL" \
-H "Authorization: Bearer $TREEROUTER_API_KEY" \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
--data-binary @glm-stream-probe.jsonRun this from a trusted development machine or backend. Keep the key out of browser code, screenshots, and shared terminal recordings. The command performs a billable request if your account charges for inference; it is not a free speed test.
If your first request does not reach a streaming response at all, use the existing GLM API integration walkthrough for the initial setup context. Resolve authentication and model availability before interpreting delivery timings.
The --no-buffer option affects curl's own output buffering. It does not disable buffering inside a gateway, reverse proxy, or application server. If this probe shows gradual output while your application does not, you have narrowed the investigation to the additional layers your application uses.
Z.AI's streaming documentation describes the streaming request setting and incremental response fields. Its Chat Completion reference is the source for the direct-provider contract. A relay may expose an adapted interface, so compare the response you actually receive with the documented contract for that route.
Second observation: separate reasoning from answer text
A stream can be active while the final-answer panel remains empty. In the documented GLM streaming shape, reasoning content and answer content can occupy different delta fields. A client that displays only answer text can therefore appear idle even when it is receiving valid events.
Count the arrival of each category without storing its full content. For example, record that reasoning activity was observed, then record the time of the first answer delta. Your interface can show an honest status such as “Processing your request” without revealing raw reasoning or pretending a final answer exists.
Check model-specific controls in the official thinking guide. Do not copy a “disable thinking” parameter from an older integration and assume every GLM model accepts it. Parameter compatibility and delivery buffering are separate questions; changing both at once makes the evidence harder to interpret.
Third observation: stop treating network chunks as events
A network read is not an SSE message. One read may contain half of an event, several complete events, or a character split across byte boundaries. If your parser attempts to decode JSON on every arbitrary read, it can produce intermittent errors or wait until the connection closes.
Use a maintained SSE parser for production. For a controlled diagnostic, the small function below accepts already decoded lines from a streaming HTTP client. It accumulates data: fields until the blank line ending the event. It does not implement reconnection, event IDs, or every EventSource behavior.
def sse_data_events(decoded_lines):
"""Diagnostic data-event parser; input is decoded lines without newlines."""
data_lines = []
for line in decoded_lines:
if line == "":
if data_lines:
yield "\n".join(data_lines)
data_lines = []
continue
if line.startswith(":"):
continue # SSE comment / heartbeat
field, separator, value = line.partition(":")
if field == "data":
if value.startswith(" "):
value = value[1:]
data_lines.append(value if separator else "")
# Do not invent a completed event from an unterminated trailing fragment.
sample = [": heartbeat", "", 'data: {"part":1}', "", "data: [DONE]", ""]
assert list(sse_data_events(sample)) == ['{"part":1}', "[DONE]"]The framing rules come from the WHATWG Server-Sent Events specification. Let the HTTP library perform incremental UTF-8 decoding, then parse complete events. After framing, handle the route's JSON schema and termination signal separately.
This distinction is especially valuable when a stream works for English text but behaves inconsistently with Chinese characters or emoji. Those symptoms can reveal a byte-decoding mistake rather than a model limitation.
Fourth observation: examine the backend's response path
Search your backend for convenience calls that read the entire upstream body before returning it. A function that awaits a complete JSON object or concatenates every chunk into one string will defeat incremental delivery even if the upstream gateway streamed perfectly.
Place one timestamp where your backend receives a complete upstream event and another where it writes the corresponding downstream event. Use a synthetic request identifier to connect the two. You do not need to log prompt contents to determine whether your own code held the response.
If there is a reverse proxy you control, inspect its buffering settings for the specific streaming route. The NGINX proxy buffering documentation explains the relevant directive. Apply changes in a test environment and at the narrowest appropriate location; disabling buffering globally is not a sensible substitute for locating the affected route.
Also check middleware that compresses, transforms, or caches responses. A seemingly harmless response formatter can turn a streaming generator into a fully materialized body. Keep content transformation out of the delivery path until the minimal stream works end to end.
Fifth observation: inspect rendering rather than requesting again
If the backend forwards events promptly but the screen still updates in one jump, examine the browser consumer and rendering schedule. Reading response.text() waits for the full response. A streaming reader needs to process incremental data and update the intended view as complete application events become available.
Do not render every byte separately. Small, bounded rendering batches can reduce interface overhead while retaining visible progress. The interval is a product decision, not a claim about GLM speed. Measure perceived responsiveness with the interface you actually ship.
When an existing coding client is involved, its adapter can introduce another boundary. The custom-model troubleshooting article covers client configuration, while the agent harness model-connection guide explains the broader integration context. Use those when the issue is confined to that client rather than the raw API route.
What a useful comparison looks like
Compare three receiving points: a minimal gateway probe, your backend, and your final interface. Keep the same model, prompt class, output allowance, and reasoning configuration. Run a small bounded set of tests; a single fast request or one slow request cannot establish a stable service characteristic.
For each point, record first event, first answer, completion, and whether the answer passed the application check. Include interrupted requests in the report. Excluding every failure can make an unreliable route appear impressively fast.
If both the direct gateway probe and application receive bursts, you have evidence to take to the service operator. You still do not know whether the delay originated in model processing, an upstream route, or gateway buffering. A precise support request says what you observed and includes timestamps and request identifiers, rather than declaring a cause prematurely.
If delays mainly appear under higher load, move to the separate GLM API concurrency planning guide. Capacity planning answers a different question from locating a buffering boundary.
Keep partial output from becoming a false success
Visible text does not prove a complete response. A connection may close after several useful paragraphs. Your application should distinguish an answer that reached its documented completion signal from a partial answer interrupted by transport failure.
Show partial content honestly. For a drafting assistant, keeping the text with an “interrupted” label may be useful. For an application expecting a complete structured record, hold the result until validation succeeds. Do not silently commit partial data simply because users saw tokens appear.
This behavior matters when comparing API relays. The valuable feature is not only early output; it is enough preserved protocol information to explain whether that output completed. Treat completion handling as part of the same end-to-end test as first-answer latency.
Turn the investigation into a small gateway pilot
For a GLM integration through Treerouter, begin with one enabled model, one synthetic streaming request, and the receiving-point comparison above. Confirm the current endpoint, account access, and pricing before running the pilot. Register when you are ready to test that concrete workflow, rather than registering solely because a headline promises a faster model.
The pilot should produce a short evidence sheet: where the first event appeared, where answer text first appeared, which layer delayed delivery, and how interrupted output was handled. That sheet helps an individual developer fix a frozen interface and gives an enterprise team something more useful than a generic “streaming supported” claim.
Once this path works, adding more model routes becomes a controlled integration task. You have a known delivery contract, a repeatable probe, and an application that can tell the difference between waiting, receiving, rendering, and finishing.
