Long-Running AI Workflows
Durable workflow engines and job queues for multi-step AI work that runs for minutes — retries, checkpoints, and resuming after failures.
Some AI work doesn't fit in a request, or even in a single background job. A customer imports 40,000 historical tickets and wants them tagged and analyzed. A weekly job clusters resolved tickets, drafts help articles, and waits for a support lead to approve each one. Each of these is a chain of steps that runs for minutes, hours, or days, calls models many times, and must not start over from zero when a deploy restarts the worker at step 7 of 9.
That's what durable workflows are for. In this lesson you'll learn when a job queue stops being enough, how durable workflow engines checkpoint progress, and how to build two real Deskmate workflows: a bulk analysis that runs through the Message Batches API at half price, and an article draft that waits a week for human review. Along the way you'll handle model-specific failures correctly, choose step boundaries that don't waste money on retries, and operate workflows once they're running.
When a job queue stops being enough
Background and Bulk AI Jobs covered the job queue: move slow work out of the request, retry it, make it idempotent, and report progress. A queue is the right tool when each job is one unit of work. It starts to strain when a job becomes a process:
| Sign | Why a plain job struggles |
|---|---|
| Several dependent steps, each with model calls | A failure at step 5 retries steps 1 to 4, paying for their model calls again |
| Waiting on something slow | A batch that takes an hour, or a human approval that takes days, holds a worker or needs hand-built polling |
| Fan-out and fan-in | Splitting into 20 sub-tasks and combining the results needs coordination state somewhere |
| Runs longer than your deploy cycle | Every deploy kills in-flight work |
| Partial progress matters | Losing six hours of analysis to a crash is a support ticket |
You can build checkpointing yourself with a steps table and careful code. For one or two workflows, that's reasonable. Beyond that, a durable workflow engine gives you the same guarantees as infrastructure.