Build a Streaming AI Chat in Next.js from Scratch

An in-app AI chat is the feature users expect first and judge fastest. They've used polished chat apps, so they notice immediately when yours makes them stare at a spinner for eight seconds, loses the conversation on refresh, double-sends when they hit Enter twice, or leaves a half-written answer on screen with no explanation when the network drops.

In this lesson you'll build Deskmate's "Ask Deskmate" assistant from scratch in a Next.js app: a streaming route handler that calls the model, a React client that renders replies as they arrive, conversation storage in Postgres, and the details that make it feel finished: a Stop button, retry, sensible scrolling, and keyboard behavior that doesn't fight the user. You'll write it with the Anthropic SDK directly so you can see every moving part, then see what the Vercel AI SDK would take off your plate.

What we're building, and who owns what

The assistant lives in a side panel. A support agent asks things like "How do I merge two customer accounts?" and gets an answer streamed into the panel. Conversations persist, so reopening the panel shows the history.

Four pieces, each with one job:

Piece	Job
`chat-store.ts`	Load and save conversations and messages in Postgres
`app/api/chat/route.ts`	Authenticate, save the user's message, call the model, stream text back, save the reply
`components/chat.tsx`	Render messages, send new ones, read the stream, Stop and Retry
`app/assistant/[conversationId]/page.tsx`	Load an existing conversation on the server for the first render

One decision shapes everything else: the server owns the conversation. The browser sends only the conversation ID and the new message text, never the full history. If the client sent the history, anyone with dev tools could rewrite it, including inventing assistant replies that "agreed" to things, and your server would forward that to the model as truth. Managing Conversation History and State covers this and the rest of the storage rules in depth; this lesson applies them.

We'll stream plain text from the server to the browser. It's the simplest protocol that works and it's enough to build a good chat. Streaming End to End, the next lesson, upgrades it to typed events so you can carry status, errors, and metadata alongside the text.

Conversation storage

Two tables: one row per conversation and one row per message. Message content is stored as the API's content blocks, verbatim, so it can be replayed exactly.

db/migrations/005_chat.sql

create table chat_conversations (
  id uuid primary key default gen_random_uuid(),
  user_id text not null,
  title text,
  created_at timestamptz not null default now()
);
 
create table chat_messages (
  id uuid primary key default gen_random_uuid(),
  conversation_id uuid not null references chat_conversations(id) on delete cascade,
  seq int not null,
  role text not null check (role in ('user', 'assistant')),
  content jsonb not null,
  status text not null default 'complete' check (status in ('complete', 'partial', 'error')),
  client_message_id text,
  created_at timestamptz not null default now(),
  unique (conversation_id, seq),
  unique (conversation_id, client_message_id)
);

The store module is a handful of small functions. Note what loadTranscript does: it skips failed replies and merges any consecutive same-role messages, so a user message whose reply failed doesn't leave the transcript in a shape the API won't accept.

src/lib/chat-store.ts

import type Anthropic from "@anthropic-ai/sdk";
import postgres from "postgres";
 
const sql = postgres(process.env.DATABASE_URL!);
 
type Row = { role: "user" | "assistant"; content: Anthropic.ContentBlockParam[]; status: string };
 
export async function getConversationForUser(id: string, userId: string) {
  const [row] = await sql<{ id: string }[]>`
    select id from chat_conversations where id = ${id} and user_id = ${userId}`;
  return row ?? null;
}
 
async function rows(conversationId: string) {
  return sql<Row[]>`
    select role, content, status from chat_messages
    where conversation_id = ${conversationId} and status <> 'error'
    order by seq`;
}
 
/** The transcript as the model should see it. */
export async function loadTranscript(conversationId: string): Promise<Anthropic.MessageParam[]> {
  const merged: { role: "user" | "assistant"; content: Anthropic.ContentBlockParam[] }[] = [];
  for (const r of await rows(conversationId)) {
    const prev = merged.at(-1);
    if (prev && prev.role === r.role) prev.content = [...prev.content, ...r.content];
    else merged.push({ role: r.role, content: r.content });
  }
  return merged;
}
 
/** The transcript as the UI should show it: just the text. */
export async function loadForDisplay(conversationId: string) {
  return (await rows(conversationId)).map((r, i) => ({
    id: String(i),
    role: r.role,
    text: r.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join(""),
    status: r.status === "partial" ? ("stopped" as const) : ("done" as const),
  }));
}
 
export async function appendMessage(
  conversationId: string,
  role: "user" | "assistant",
  content: Anthropic.ContentBlockParam[],
  opts: { status?: "complete" | "partial" | "error"; clientMessageId?: string } = {},
) {
  await sql`
    insert into chat_messages (conversation_id, seq, role, content, status, client_message_id)
    values (
      ${conversationId},
      (select coalesce(max(seq), 0) + 1 from chat_messages where conversation_id = ${conversationId}),
      ${role}, ${JSON.stringify(content)}::jsonb, ${opts.status ?? "complete"}, ${opts.clientMessageId ?? null}
    )
    on conflict (conversation_id, client_message_id) do nothing`;
}

The client_message_id unique constraint is what makes sending idempotent: the browser generates an ID for each message, and if the same send arrives twice (a double click, a retry), the second insert does nothing.

The streaming route handler

The route handler is where the work happens. It authenticates, validates the input, saves the user's message before calling the model (so it's never lost), streams the reply, and saves the reply when it finishes. requireUser stands in for your app's existing auth helper; it returns the signed-in user or throws.

src/lib/ai.ts

import Anthropic from "@anthropic-ai/sdk";
 
export const anthropic = new Anthropic();
export const MODELS = { fast: "claude-haiku-5-5", default: "claude-sonnet-5-5", best: "claude-opus-5-5" } as const;

src/app/api/chat/route.ts

import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";
import { anthropic, MODELS } from "@/lib/ai";
import { requireUser } from "@/lib/auth";
import { appendMessage, getConversationForUser, loadTranscript } from "@/lib/chat-store";
 
export const maxDuration = 120; // seconds; long answers need more than many platforms' defaults
 
const Body = z.object({
  conversationId: z.uuid(),
  clientMessageId: z.string().min(1).max(100),
  text: z.string().trim().min(1).max(8000),
});
 
const SYSTEM = `You are Ask Deskmate, the in-app assistant for support agents using Deskmate, a help-desk product.
Answer questions about using Deskmate concisely. Use short paragraphs and numbered steps for procedures.
If you don't know how a Deskmate feature works, say so and suggest checking the help center. Don't guess at menu names.`;
 
export async function POST(request: Request) {
  const user = await requireUser();
  const parsed = Body.safeParse(await request.json());
  if (!parsed.success) return Response.json({ error: "Invalid request" }, { status: 400 });
  const { conversationId, clientMessageId, text } = parsed.data;
 
  if (!(await getConversationForUser(conversationId, user.id))) {
    return Response.json({ error: "Not found" }, { status: 404 });
  }
 
  await appendMessage(conversationId, "user", [{ type: "text", text }], { clientMessageId });
  const history = await loadTranscript(conversationId);
 
  // A replayed send whose reply was already saved: don't generate twice.
  if (history.at(-1)?.role === "assistant") {
    return Response.json({ error: "Already answered; reload the conversation" }, { status: 409 });
  }
 
  const stream = anthropic.messages.stream(
    { model: MODELS.default, max_tokens: 8000, output_config: { effort: "low" }, system: SYSTEM, messages: history },
    { signal: request.signal },
  );
 
  const encoder = new TextEncoder();
  let sent = "";
 
  const body = new ReadableStream<Uint8Array>({
    async start(controller) {
      try {
        for await (const event of stream) {
          if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
            sent += event.delta.text;
            controller.enqueue(encoder.encode(event.delta.text));
          }
        }
        const final = await stream.finalMessage();
        await appendMessage(conversationId, "assistant", final.content);
        controller.close();
      } catch (err) {
        const stopped = err instanceof Anthropic.APIUserAbortError;
        await appendMessage(conversationId, "assistant", [{ type: "text", text: sent || "(no output)" }], {
          status: stopped ? "partial" : "error",
        });
        if (!stopped) console.error("chat stream failed", err);
        controller.error(err);
      }
    },
    cancel() {
      stream.abort(); // the browser went away or clicked Stop
    },
  });
 
  return new Response(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8", "Cache-Control": "no-store" },
  });
}

The details that matter:

effort: "low" keeps the time to first token short. A help assistant answering "how do I…" questions doesn't need deep reasoning, and in chat, waiting is the most visible cost. If evals show weak answers on hard questions, raise it (see Reasoning Models).
Cancellation flows upstream. When the user clicks Stop, the browser aborts its request. Depending on your platform, the server sees that as request.signal aborting (which the SDK is listening to) or as the stream's cancel callback firing. Handling both means you stop reading from the model as soon as nobody is listening, instead of paying for a full answer nobody will see.
Stopped replies are saved as partial, so the conversation still makes sense after a reload and the user can say "go on." Failed replies are saved as error for your records, and loadTranscript excludes them, so a retry starts clean.
maxDuration is a Next.js route segment option that tells your deployment platform how long this route may run. Streaming responses that outlive a platform's default limit get cut off mid-sentence. Deploying AI Features covers hosting limits in detail.

The React client

The client sends the message, then reads the response body chunk by chunk and appends each piece to the last message. It keeps an AbortController per request so Stop can cancel it, and it gives each message a status so the UI can show "stopped" or "failed, retry."

src/components/chat.tsx

"use client";
 
import { useEffect, useRef, useState, type KeyboardEvent } from "react";
 
export type ChatMessage = {
  id: string;
  role: "user" | "assistant";
  text: string;
  status: "done" | "streaming" | "stopped" | "error";
};
 
export function Chat({ conversationId, initial }: { conversationId: string; initial: ChatMessage[] }) {
  const [messages, setMessages] = useState(initial);
  const [input, setInput] = useState("");
  const [busy, setBusy] = useState(false);
  const abortRef = useRef<AbortController | null>(null);
  const listRef = useRef<HTMLDivElement>(null);
  const stickToBottom = useRef(true);
 
  useEffect(() => {
    const el = listRef.current;
    if (el && stickToBottom.current) el.scrollTop = el.scrollHeight;
  }, [messages]);
 
  async function send(text: string, clientMessageId: string, isRetry = false) {
    const replyId = crypto.randomUUID();
    const reply: ChatMessage = { id: replyId, role: "assistant", text: "", status: "streaming" };
    setMessages((prev) => [
      ...(isRetry ? prev.slice(0, -1) : prev), // a retry replaces the failed reply
      ...(isRetry ? [] : [{ id: clientMessageId, role: "user" as const, text, status: "done" as const }]),
      reply,
    ]);
    const update = (patch: Partial<ChatMessage>) =>
      setMessages((prev) => prev.map((m) => (m.id === replyId ? { ...m, ...patch } : m)));
 
    const controller = new AbortController();
    abortRef.current = controller;
    setBusy(true);
    stickToBottom.current = true;
 
    try {
      const res = await fetch("/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ conversationId, clientMessageId, text }),
        signal: controller.signal,
      });
      if (!res.ok || !res.body) throw new Error(`Request failed (${res.status})`);
 
      const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
      let full = "";
      for (;;) {
        const { done, value } = await reader.read();
        if (done) break;
        full += value;
        update({ text: full });
      }
      update({ status: "done" });
    } catch {
      update({ status: controller.signal.aborted ? "stopped" : "error" });
    } finally {
      abortRef.current = null;
      setBusy(false);
    }
  }
 
  function submit() {
    const text = input.trim();
    if (!text || busy) return;
    setInput("");
    void send(text, crypto.randomUUID());
  }
 
  function retry() {
    const lastUser = [...messages].reverse().find((m) => m.role === "user");
    if (lastUser && !busy) void send(lastUser.text, lastUser.id, true);
  }
 
  function onKeyDown(e: KeyboardEvent<HTMLTextAreaElement>) {
    // Enter sends, Shift+Enter adds a newline; never send mid-composition (IME input).
    if (e.key === "Enter" && !e.shiftKey && !e.nativeEvent.isComposing) {
      e.preventDefault();
      submit();
    }
  }
 
  return (
    <div className="chat">
      <div
        ref={listRef}
        className="chat-log"
        aria-live="polite"
        onScroll={(e) => {
          const el = e.currentTarget;
          stickToBottom.current = el.scrollHeight - el.scrollTop - el.clientHeight < 40;
        }}
      >
        {messages.map((m, i) => (
          <div key={m.id} className={`msg msg-${m.role}`}>
            <p style={{ whiteSpace: "pre-wrap" }}>{m.text || (m.status === "streaming" ? "Thinking…" : "")}</p>
            {m.status === "stopped" && <small>Stopped</small>}
            {m.status === "error" && i === messages.length - 1 && (
              <small>
                Something went wrong. <button onClick={retry}>Retry</button>
              </small>
            )}
          </div>
        ))}
      </div>
      <textarea value={input} onChange={(e) => setInput(e.target.value)} onKeyDown={onKeyDown} rows={2} />
      {busy ? (
        <button onClick={() => abortRef.current?.abort()}>Stop</button>
      ) : (
        <button onClick={submit} disabled={!input.trim()}>Send</button>
      )}
    </div>
  );
}

Retry reuses the original clientMessageId. The server's insert is a no-op for a message it already has, and because the failed reply was excluded from the transcript, the model sees the user's message as the latest turn and answers it again. No duplicate bubbles, no duplicate rows.

The page that hosts the component loads the conversation on the server, so a refresh shows the full history immediately. In Next.js 16, params is a promise:

src/app/assistant/[conversationId]/page.tsx

import { notFound } from "next/navigation";
import { Chat } from "@/components/chat";
import { requireUser } from "@/lib/auth";
import { getConversationForUser, loadForDisplay } from "@/lib/chat-store";
 
export default async function AssistantPage({ params }: { params: Promise<{ conversationId: string }> }) {
  const { conversationId } = await params;
  const user = await requireUser();
  if (!(await getConversationForUser(conversationId, user.id))) notFound();
 
  const initial = await loadForDisplay(conversationId);
  return <Chat conversationId={conversationId} initial={initial} />;
}

ExerciseBuild it, then try to break it

Build the four pieces above in a Next.js app (any small app works; Deskmate is optional). Create a conversation row by hand and open its page.
Ask a question that produces a long answer and confirm text appears within a second or two and grows smoothly.
Click Stop halfway. Reload the page. The partial answer should still be there, marked as stopped. Then type "continue" and check the model picks up where it left off.
Simulate a failure: temporarily set an invalid API key, send a message, then restore the key and click Retry. Check the database: one user row, one error row, one complete row.
Double-click Send rapidly, and press Enter while composing text with an IME (or a dead key), and confirm neither creates duplicate messages.

Success looks like: every interruption you can think of leaves the conversation in a state that makes sense to both the user and the model after a reload.

Polish that makes it feel finished

The code above works. These details are what make people trust it.

Show life before the first token. With adaptive thinking, there can be a pause before any text arrives. The "Thinking…" placeholder covers it; Latency UX goes further with staged progress for longer waits.

Scroll politely. The stickToBottom ref follows new text only while the user is already at the bottom. If they scroll up to reread something, streaming text no longer yanks them back down. This is one of the most common complaints about homemade chat UIs.

Render formatting safely. Models often answer in Markdown. If you render it, use a Markdown renderer with raw HTML disabled, and treat links and images in model output as untrusted. Securing Model Output explains why that matters. Plain text with preserved line breaks, as above, is a safe starting point.

Make it accessible. aria-live="polite" lets screen readers announce new content, but announcing every streamed token is noisy; many teams announce only completed messages. Accessible AI Interfaces covers the trade-offs.

Give the empty state a job. A blank panel with a text box leaves users guessing what the assistant can do. Two or three suggested questions drawn from real usage teach capabilities better than a paragraph of instructions. See Onboarding, Empty States, and Suggested Prompts.

Title conversations in the background. After the first exchange, a small call to claude-haiku-5-5 can generate a four-word title for the conversation list. Run it after the response finishes so it never delays the chat.

The same chat with the AI SDK

Everything above is code you now understand line by line. The Vercel AI SDK (introduced in SDKs and Abstraction Layers) packages much of it: a useChat hook that manages message state, streaming, status, and stop, and server helpers that produce a typed stream format the hook understands. The server side of an equivalent route looks like this:

src/app/api/chat-sdk/route.ts

import { anthropic } from "@ai-sdk/anthropic";
import { convertToModelMessages, streamText, type UIMessage } from "ai";
 
export async function POST(request: Request) {
  const { messages }: { messages: UIMessage[] } = await request.json();
 
  const result = streamText({
    model: anthropic("claude-sonnet-5-5"),
    instructions: "You are Ask Deskmate, the in-app assistant for Deskmate support agents.",
    messages: await convertToModelMessages(messages),
    providerOptions: { anthropic: { effort: "low" } },
    abortSignal: request.signal,
  });
 
  return result.toUIMessageStreamResponse();
}

On the client, useChat from @ai-sdk/react gives you messages, sendMessage, status, and stop. Notice that this minimal version accepts the full history from the browser, which is the trust problem from the start of this lesson. To keep the server in charge, configure the hook's transport to send only the newest message (the prepareSendMessagesRequest option on DefaultChatTransport), load the stored history on the server, and save the result in the response's onEnd callback.

Which should you choose? Use the AI SDK when you want tool calls rendered as UI, multiple providers, or rich message types without designing your own protocol. Write it yourself when the chat is simple, when you need full control over storage and the wire format, or when you want fewer dependencies to keep current. Either way, the decisions in this lesson (server-owned history, idempotent sends, saved partials, cancellation that reaches the model) are yours to make; no library makes them for you.

Key takeaways

The server owns the conversation. The browser sends a conversation ID, a client message ID, and the new text, never the history.
Save the user's message before calling the model, stream the reply as it's generated, and save the reply when the stream ends.
Wire cancellation end to end: Stop aborts the fetch, and the server aborts the model stream so you stop paying for unread output.
Give messages a status (streaming, stopped, error) and make Retry idempotent by reusing the client message ID.
Polish is part of the feature: early feedback, polite scrolling, safe rendering, and an empty state that teaches.

Next: Streaming End to End — what's actually happening between the model and the browser, and how to handle backpressure, cancellation, and errors that arrive mid-stream.

What we're building, and who owns what#

Conversation storage#

The streaming route handler#

The React client#

Polish that makes it feel finished#

The same chat with the AI SDK#