Background and Bulk AI Jobs

Some AI work doesn't belong in a request at all. Summarizing every new ticket the moment it arrives, reclassifying 40,000 old tickets after a taxonomy change, extracting fields from a 2,000-row customer import: none of these have a user staring at a spinner, and all of them will eventually hit a timeout, a rate limit, or a deploy that kills the process halfway through.

The fix is old and unglamorous: a queue and background workers. But AI jobs have quirks that ordinary background jobs don't. Every attempt costs money, the same input can produce a different output, and rate limits are tight enough to shape your whole design. In this lesson you'll build a small Postgres-backed job system for Deskmate, with leases, retries that distinguish temporary errors from permanent ones, idempotency so a job that runs twice does no harm, and progress reporting for bulk runs.

What belongs in the background#

A rough test: move AI work out of the request path when any of these is true.

• Nobody is waiting for it. The work is triggered by an event (a ticket arrives, an email lands) rather than a click.
• It takes longer than a user will wait. More than a few seconds without visible progress feels broken.
• It touches many records. Anything that loops over a table belongs in jobs, one per record or small group.
• It needs retries to be reliable. A request handler gets one shot; a job can try again in five minutes.

Here's how that sorts Deskmate's AI features:

The second row is a pattern worth adopting widely: precompute in the background what users will read later. The summary is waiting when the agent opens the ticket, so it costs no latency at all.