Shadow mode

In shadow mode, each request, or a sample of them, is handled by the live version as usual, and the same input is also sent to a candidate: a new model, prompt, or pipeline. The candidate's output is stored but never shown to anyone. You see how it behaves on real traffic with no user exposure.

It's the right tool for changes too risky or too subtle to try on real users, even at a small percentage. Migrating a feature customers depend on to a new model is the classic case. You can compare outputs pairwise, run your graders on both, check the candidate's latency and cost, and pull the inputs where the two versions disagree, which are usually the most informative ones to read. It's also a safe way to start a brand-new feature: run it silently for a while before anyone sees it.

Shadowing adds model spend for every sampled request, so sample instead of mirroring everything, and run the candidate after the user's response is sent so it adds no latency. Make sure the shadow path has no side effects: no tool calls that write data, no emails, no billing events. And it can't tell you how users react; once the candidate looks good, a staged rollout or A/B test answers that.