Constrained decoding

Normally a model can produce any token at any step. Constrained decoding narrows the choice: at each step, tokens that would break the target format are ruled out, so the finished output must match the schema or grammar. If the schema says the next thing must be a number or one of three enum values, nothing else can come out.

It is the mechanism behind many structured-output features. Hosted providers apply it for you when you supply a schema, and open-weight serving stacks such as vLLM support schema-constrained output too, which matters when you run your own models and can't rely on a provider's implementation.

Constrained decoding guarantees shape, not substance. A model forced into a schema it doesn't understand can still produce valid nonsense, and over-tight constraints can hurt quality by forcing a value where the honest answer is unknown. Give it escape hatches in the schema, keep instructions clear, and validate the content after parsing.

It's also why prompting a model to output only JSON is a weaker approach: without constraints, an otherwise good response can still include a stray sentence or a missing bracket that breaks your parser.