principles.fyi · the brain · concept

hallucination

When a model states something false but says it with total confidence.

A hallucination is fluent, confident output that is factually wrong or made up — a fake citation, an invented statistic, a plausible-sounding but untrue claim. It arises because the model is trained to produce probable-sounding next tokens (Book 01), not to check facts; when it lacks the real answer it will still generate the most likely-looking continuation. This is a core reliability problem for generative AI, which is why outputs need verification and why techniques like grounding the model in retrieved sources are used to reduce it.