Classification, Tagging, and Routing
Build label-and-route features with fixed taxonomies, multi-label tags, an "other" escape hatch, and the metrics that tell you it's working.
Most help desks have someone whose morning starts with triage: read each new ticket, decide what kind of problem it is, tag it, and drop it in the right queue. It's repetitive, it's judgment-heavy, and it's the bottleneck every other step waits on. It's also one of the most dependable things you can hand to a language model, if you treat it as a classification system and not as a clever prompt.
In this lesson you'll build triage for Deskmate: a taxonomy designed so that people (and models) can apply it consistently, a classifier that returns one category plus any number of tags, a router where your code makes the decisions, and the metrics that tell you whether it's working or slowly drifting.
Design the taxonomy before the prompt
The quality ceiling of a classifier is set by its labels, not its model. If two experienced support agents would disagree about whether a ticket is "billing" or "account," no prompt will make the model agree with both of them. So start with the labels.
A good taxonomy for routing has four properties:
- Each label maps to an action. A label exists because something different happens next: a different queue, a different SLA, a different team. If two labels lead to the same place, merge them.
- Labels are mutually exclusive for anything that picks a single destination. A ticket goes to one queue.
- The set is exhaustive, which in practice means an explicit
otherlabel for things that don't fit. - Boundaries are written down. The definitions that matter most aren't "billing means billing." They're the tie-breakers: "A failed payment is
billing, even if the customer calls it a bug."
Here's Deskmate's starting taxonomy, kept in code as the single source of truth for the prompt, the schema, and the router:
export const CATEGORIES = {
billing: {
definition: "Charges, invoices, refunds, failed payments, plan prices.",
boundary: "A failed payment or wrong charge is billing even if the customer calls it a bug.",
queue: "billing",
},
bug: {
definition: "Deskmate behaves differently from how it should: errors, crashes, broken features.",
boundary: "If the customer is asking how to do something, it is how_to, not bug, unless they report an error.",
queue: "engineering-triage",
},
account_access: {
definition: "Login problems, password resets, 2FA, SSO, locked accounts, user invitations.",
boundary: "An SSO error is account_access, not bug.",
queue: "support-tier1",
},
how_to: {
definition: "Questions about how to use a feature that is working as designed.",
boundary: "Feature requests ('can you add...') are feature_request, not how_to.",
queue: "support-tier1",
},
feature_request: {
definition: "Requests for new features or changes to existing behavior.",
boundary: "Complaints that a feature is missing count as feature_request.",
queue: "product-feedback",
},
other: {
definition: "Anything that does not clearly fit one of the categories above, including spam and messages with no request.",
boundary: "Use other instead of forcing a poor fit.",
queue: "support-tier1",
},
} as const;
export type Category = keyof typeof CATEGORIES;
export const CATEGORY_NAMES = Object.keys(CATEGORIES) as [Category, ...Category[]];
export const TAGS = {
data_loss: "The customer reports missing or deleted data.",
security: "Possible security issue: unauthorized access, exposed data, suspicious activity.",
churn_risk: "The customer mentions cancelling, switching providers, or ending their contract.",
outage: "Several users or the whole account cannot use Deskmate at all.",
} as const;
export type Tag = keyof typeof TAGS;
export const TAG_NAMES = Object.keys(TAGS) as [Tag, ...Tag[]];Before you write a single prompt, run a quick agreement test: you and a colleague each label the same 30 real tickets using only these definitions. Where you disagree, the definitions are ambiguous. Fix them now, because the model will disagree in exactly the same places.