Architecture for AI-Powered Apps
Reference architectures for AI features in web apps: where model calls live, an AI service layer, data flows, and the decisions that are hard to undo.
Your first AI feature probably started as one messages.create call inside a route handler. That's the right way to start. The trouble begins with the fifth feature, when there are model IDs in eleven files, three slightly different retry wrappers, no record of which prompt produced which output, and a background job that calls the model with nobody's budget attached. Switching models becomes a scavenger hunt, and an incident review becomes guesswork.
This lesson gives you a reference architecture for AI features in a web app, the kind you can grow from one feature to fifty without rewriting. You'll see the three shapes AI features take, where model calls should live, how to build a thin AI service layer every feature goes through, what to record about every generation, and which architectural decisions are expensive to reverse, so you make those deliberately.
Three shapes of AI features
Almost every AI feature fits one of three execution shapes. Identifying the shape first tells you most of the architecture.
| Shape | Examples in Deskmate | Latency budget | Runs in | When it fails |
|---|---|---|---|---|
| Interactive | Help-center chat, inline reply drafting, natural-language search | Seconds; first token fast | Request path, usually streamed | The user sees it immediately and retries |
| Background | Ticket summaries on arrival, auto-tagging, embeddings for new articles | Seconds to minutes; nobody is waiting | Job queue and workers | Nobody notices unless you monitor it |
| Workflow | Nightly account-health reports, importing and analyzing 50,000 historical tickets, generating help articles with human review | Minutes to days | A durable workflow engine | Partway through, with some steps done |
The shapes have different failure modes, and that drives the design. Interactive features need fast fallbacks and good error UX (Designing for Uncertainty and Errors). Background features need retries, idempotency, and monitoring, covered in Background and Bulk AI Jobs. Workflows need checkpoints so they can resume, the subject of Long-Running AI Workflows.
A common mistake is building a background-shaped feature in the interactive shape. Summarizing a ticket when the agent opens it puts a model call on the critical path of every page load. Summarizing it when the message arrives makes the page instant and moves the failure somewhere you can retry it.