Model Routing and Cascades

Picking one model per feature is the right first step, but it forces a compromise. Most requests a feature handles are easy; a few are genuinely hard. Choose the model the hard ones need and you overpay on every easy request. Choose the model the easy ones need and the hard ones fail. In Deskmate's drafted replies, "How do I reset my password?" and "Your export deleted three months of our data and legal is asking questions" go through the same button.

Routing solves this by choosing a model for each request instead of each feature. You'll build the three patterns that cover nearly every case: a rule-based router using facts you already have, a classifier router that asks a small model how hard a request is, and a cascade that tries the cheap model first and escalates when its answer fails a check. You'll also learn when routing isn't worth the complexity, which is often.

When per-request routing pays off#

Routing adds code, a new way for things to go wrong, and a second dimension to every eval. Before building one, check that three conditions hold:

1. Difficulty varies a lot within the feature. If nearly every request is similar, one well-chosen model is simpler and just as good. Routing pays when there's a mix of trivial and hard.
2. The price or latency gap between models is large. At current prices, Claude Haiku 5.5 costs a twentieth of Sonnet 5.5 per token. A gap that size makes routing worthwhile at volume. Between two models with similar prices, it rarely is.
3. You can tell easy from hard, cheaply. Either from facts you already have about the request, or with a check you can run on the answer. If you can't, the router just adds noise.

It also helps to be clear about what this lesson is not. Classification, Tagging, and Routing in Module 3 is about routing tickets to the right team or queue, which is a product feature. Here, you're routing model calls to the right model, which is an infrastructure decision invisible to users.