Streaming AI Chat Replies (SSE) — fetch ReadableStream + Edge Worker

ChatGPT and most web AI chats do not wait for the whole reply to finish. Instead, they paint each generated character as it arrives — the familiar smooth streaming interface.

This article explains how to implement that streaming (Server-Sent Events / ReadableStream) in JavaScript / TypeScript, including the Japanese-specific mojibake trap.

Streaming vs one-shot receive

How the two approaches compare in UX and engineering.

Mode	First character (TTFB)	Transport	Pros	Gotchas
One-shot (JSON)	Seconds to 10+ seconds (slow)	Normal HTTP POST	Simple to implement	Users abandon long replies
Streaming (SSE)	Hundreds of milliseconds (very fast)	Chunked stream	Feels dramatically faster	Must handle mojibake at chunk boundaries

To make users feel “this is fast,” shortening the time to first token (TTFT) matters far more than the total generation time.

Four implementation steps

How to make streaming work across the Edge server and the browser client.

1. Relay the stream on the server Call the Gemini API or Workers AI with stream: true and return the ReadableStream as-is to the client.

2. Set the right response headers Send Content-Type: text/event-stream and Cache-Control: no-cache.

3. Get a reader on the client After const response = await fetch('/api/chat', ...), grab the stream reader with response.body.getReader().

4. Loop-read and decode chunks In a while (true) loop call reader.read(), then decode the received binary with TextDecoder({ stream: true }) and paint it into the UI.

Minimal code example (client-side stream receive)

async function sendChatMessage(message: string, onChunk: (text: string) => void) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages: [{ role: 'user', content: message }] }),
  });

  if (!response.body) return;

  const reader = response.body.getReader();

  // Required: { stream: true } to survive multi-byte splits across chunks
  const decoder = new TextDecoder('utf-8');
  let fullText = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    // Decode the byte array (partial trailing bytes are buffered internally)
    const chunkText = decoder.decode(value, { stream: true });
    fullText += chunkText;

    // Update the UI
    onChunk(fullText);
  }
}

Failure boundary — real incidents and fixes

Streaming problems you are likely to run into, and their fixes.

1. Japanese multi-byte characters split across chunks

• Symptom: Hiragana and kanji render as ”�” garbage or get split unnaturally.
• Cause & fix: A UTF-8 Japanese character is 3 bytes. If a network chunk boundary lands mid-character, that byte alone cannot be decoded. Passing the .decode(value, { stream: true }) option to new TextDecoder() buffers incomplete bytes automatically until the next chunk arrives.

2. CDN / proxy buffering the response

• Symptom: The server streams, but the browser receives the whole message at once.
• Cause & fix: An intermediary (Cloudflare, Nginx) may be buffering or caching the response. Add explicit headers like X-Accel-Buffering: no and Cache-Control: no-cache.

The Kaga chat on this site uses exactly this TextDecoder({ stream: true }) + Cloudflare Edge stream-relay approach for real-time display.