Model routing

Most features start by sending every request to one model. That forces a compromise: pay for a strong model on every trivial request, or use a cheap one and fail on the hard cases. Model routing moves the choice from the feature to the individual request.

Routers come in a few shapes. A rule-based router uses facts you already have, such as input length, the customer's plan, language, or which screen the request came from. A classifier router asks a small, fast model to label the request's type or difficulty before the real call. A cascade, a related pattern, tries the cheap model first and escalates only when its answer fails a check. Routing can also pick between providers for latency, region, or data-handling reasons.

Routing adds a component that can be wrong, a call that costs time, and behavior that's harder to reason about, so it pays off only when traffic is high and genuinely mixed in difficulty. Measure it like a feature: log which route each request took, sample both routes for quality, and confirm the savings survive the cost of the router itself. Often one well-chosen model per feature is simpler and good enough.