Streaming End to End
How token streaming works from the provider to the browser — server-sent events, backpressure, cancellation, and errors that arrive mid-stream.
Streaming looks like a single feature: words appear as the model writes them. In reality it's a relay across several systems, each with its own buffers and failure modes. The provider streams to your server, your server streams to a proxy or CDN, the proxy streams to the browser, and the browser streams into React. A stream that works perfectly on your laptop can arrive in one lump behind a corporate proxy, die silently after 60 seconds on a serverless platform, or keep billing you for output after the user has closed the tab.
This lesson follows a token from the model to the screen. You'll see what the provider actually sends, design a small typed protocol for your own server-to-browser stream, apply backpressure so a slow reader doesn't pile up memory, wire cancellation through every hop, and handle errors that arrive after you've already sent a 200 status.
The four hops of a stream
Every streamed response crosses the same hops. Knowing them turns "streaming is broken" into a specific, checkable question.
| Hop | What happens | What goes wrong |
|---|---|---|
| Provider → your server | The model API sends server-sent events as tokens are generated | Overload errors mid-stream; long silences while the model thinks |
| Your server → network | Your route handler writes chunks to the HTTP response | Compression or framework buffering; platform time limits |
| Network → browser | Proxies, load balancers, and CDNs relay bytes | Proxies that buffer whole responses; idle-connection timeouts |
| Browser → screen | Your code parses chunks and updates the UI | Chunks split mid-event; re-rendering on every token |
Server-sent events (SSE) is the format used on the first hop and, in this lesson, on the second and third too. It's a simple text protocol over a normal HTTP response: each event is a few field: value lines followed by a blank line. It flows one way, server to client, which is exactly what a model response needs.
The previous lesson, Building a Streaming Chat Feature, streamed plain text. That's fine until you need to send anything besides text: a "thinking" status, a message ID, a clean "done" signal, or an error after the stream has started. That's what we'll fix.