Eval-driven development

Eval-driven development brings the test-first habit to AI features. Before tuning a prompt, you define what good looks like as a set of cases and checks. Every change is then judged by whether the scores improve, not by whether a few hand-picked examples look better.

In practice it's a loop. Error analysis turns up a failure category; you add cases and a check for it; you change the prompt, context, or model; you rerun the suite and confirm the target improved without breaking anything else. The same suite runs in CI on every pull request that touches a prompt, model ID, schema, or retrieval setting, with thresholds that block the merge when quality drops.

Two pitfalls are common. Overfitting: iterate against the same cases for weeks and your prompt learns your test set, so keep a held-out split and keep adding fresh production samples. Cost and noise: model-graded checks cost money and vary between runs, so run cheap code checks on every change and the full suite before release, with thresholds that tolerate some noise. The discipline pays off most at model upgrades, when you can test a new model on your own traffic the day it ships.