Hallucination

Hallucination is the phenomenon where a language model produces text that is fluent and confident but factually wrong: inventing statistics, fabricating citations, misattributing quotes to real people, generating non-existent function signatures, or asserting events that did not occur. It is not a bug — it is a structural consequence of how LLMs work. The model is trained to predict statistically likely next tokens, not to retrieve verified facts. When the training distribution suggests a confident answer that no specific source actually supports, the model generates plausible text rather than declining.

Hallucination types map to different mitigation strategies: (1) factual hallucination — wrong dates, numbers, attributions; mitigated by retrieval-augmented generation and explicit grounding instructions; (2) intrinsic hallucination — the output contradicts the source the model was given; mitigated by tighter instruction tuning and prompt constraints; (3) extrinsic hallucination — the output introduces information not in the source; mitigated by output constraints and post-hoc verification; (4) confabulation — fabricating coherent but fictional content (fake URLs, made-up book titles, invented people); the hardest type to detect because the output reads as authoritative.

Mitigation in modern systems is a stack, not a single fix. Frontier models reduce raw hallucination rates through RLHF and constitutional training that rewards calibrated uncertainty ("I don't know" responses). RAG architectures cut hallucination dramatically by grounding answers in retrieved documents. Chain-of-thought prompting surfaces reasoning steps that can be audited. Output constraints ("cite only the sources I provided; if the answer isn't there, say so") provide an explicit floor. Post-hoc verification — fact-checking against authoritative APIs or databases — catches what slipped through.

Measurement frameworks have matured: TruthfulQA tests adversarial truthfulness on common misconceptions; HaluEval benchmarks hallucination across summarization, QA, and dialogue; SimpleQA tests short-form factual accuracy; FEVER tests fact verification against Wikipedia. Production deployments typically build domain-specific eval sets — a customer-support bot needs to be evaluated on customer-support hallucinations, not generic ones.

For end users, the practical pattern is to treat AI output as a draft that needs verification on any claim with stakes: numbers, names, dates, citations, code that touches data or money. The "confidently wrong" failure mode is the most damaging because it bypasses the user's skepticism. Confident-sounding text from any LLM should be checked against a primary source before being used in any high-stakes context.