Despite years of mitigation efforts, hallucinations remain a persistent problem even in today's most advanced large language models (LLMs). A new paper from researchers including Liu, Emmy, Gangal, Varun, Zou, Chelsea, Yu, Michael, Huang, Xiaoqi, Chang, Alex, Tao, Zhuofu, Singh, Karan, Kumar, Sachin, and Y. Feng, published on arXiv, argues that the root cause is a lack of a unified definition. The paper, titled "A Unified Definition of Hallucination: It's The World Model, Stupid!" and accessible at arXiv:2512.21577, reviews existing definitions and folds them into a single, coherent framework.
The Persistence of Hallucination
According to the paper, hallucinations have troubled language models since their inception. The authors note that despite numerous attempts at mitigation, the problem endures in frontier LLMs. They ask why this is the case, and their answer points to the need for a common understanding of what hallucination actually is.
A Unified Definition: Inaccurate World Modeling
The researchers propose defining hallucination as simply inaccurate (internal) world modeling, in a form where it is observable to the user. For example, stating a fact which contradicts a knowledge base, or producing a summary which contradicts the source. The key insight is that by varying the reference world model and conflict policy, this framework unifies prior definitions. This means previous definitions—such as factuality errors, faithfulness errors, or input-conflicting outputs—are all subsumed under the broader concept of world model inaccuracy.
Why This Distinction Matters
The unified view is useful, the authors argue, because it forces evaluations to clarify their assumed reference "world." It also distinguishes true hallucinations from planning or reward errors—a crucial difference for model developers. Additionally, the common language provided enables better comparison across benchmarks and more coherent discussion of mitigation strategies.
The HalluWorld Benchmark
Building on this definition, the paper connects the framework to HalluWorld, a complementary benchmark that instantiates fully specified reference world models for stress-testing model hallucinations. This allows researchers to systematically test how well models track the world model they are supposed to follow.
Implications for AI Evaluation
The proposed definition has significant implications for how AI systems are evaluated and improved. By clarifying what constitutes a hallucination versus a different type of error, developers can more precisely target their mitigation efforts. The paper emphasizes that without a unified definition, different benchmarks measure different things, making progress hard to track.
The Authors and Context
This paper appears in the Computation and Language section of arXiv, a preprint server. The authors come from a mix of academic and industrial backgrounds, though specific affiliations are not detailed in the abstract provided. The paper is licensed under CC BY 4.0.
For enterprise technology leaders, the inability to trust LLM outputs—especially when they hallucinate—directly impacts use cases like automated reporting, knowledge management, and decision support. While the paper does not directly address supply chain or trade applications, the underlying problem of world model accuracy is universal. As LLMs are increasingly deployed in mission-critical enterprise workflows, a precise definition of hallucination becomes essential for setting expectations, designing tests, and selecting models. The HalluWorld benchmark could provide a standard way to evaluate enterprise-grade LLMs before deployment, reducing the risk of costly errors.
| Aspect | Previous Definitions | Unified Definition (this paper) |
|---|---|---|
| Scope | Multiple, often conflicting | Single framework subsuming all |
| Core concept | Factuality, faithfulness, etc. | Inaccurate world modeling |
| Observable | Varies | Always observable to user |
| Error types | Mixed | Separates hallucination from planning/reward errors |
| Benchmark | Fragmented | HalluWorld with fully specified world models |
