Hallucination

A hallucination is a model producing something false with the same fluency as something true. It happens because models generate likely-sounding text, not verified facts. Ask about a policy the model has never seen, and it may write a reasonable-sounding one; ask for a source, and it may produce a title that doesn't exist.

In products, hallucinations show up in predictable places: questions outside the provided context, specific numbers and dates, names and IDs, citations, and extraction from documents where a field is missing and the model fills the gap. They are also more likely when the prompt implies an answer must exist.

You can't prompt hallucinations away entirely, so design for them. Ground answers in your own data and instruct the model to say when the context doesn't cover the question. Have it cite or quote sources and verify those quotes in code. Use schemas that allow null for missing fields. Fetch real records with tools instead of letting the model recall them. And measure the rate with evals, because a feature that is wrong one time in fifty can still ship if the interface makes checking easy.