As large language models (LLMs) are increasingly applied in sensitive domains such as medicine and law, hallucination detection has become a critical task. Although numerous benchmarks have been proposed to advance research in this area, most of them are artificially constructed--either through deliberate hallucination induction or simulated interactions--rather than derived from genuine LLM-human dialogues. Consequently, these benchmarks fail to fully capture the characteristics of hallucinations that occur in real-world usage. To address this limitation, we introduce AuthenHallu, the first hallucination detection benchmark built entirely from authentic LLM-human interactions. For AuthenHallu, we select and annotate samples from genuine LLM-human dialogues, thereby providing a faithful reflection of how LLMs hallucinate in everyday user interactions. Statistical analysis shows that hallucinations occur in 31.4% of the query-response pairs in our benchmark, and this proportion increases dramatically to 60.0% in challenging domains such as Math & Number Problems. Furthermore, we explore the potential of using vanilla LLMs themselves as hallucination detectors and find that, despite some promise, their current performance remains insufficient in real-world scenarios. The data and code are publicly available at https://github.com/TAI-HAMBURG/AuthenHallu.
Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Gaoling School of Artifical Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation
Institute for Advanced Algorithms Research(高级算法研究院);State Key Laboratory of Media Convergence Production Technology and Systems(媒体融合生产技术与系统国家重点实验室);School of Information, Renmin University of China(中国人民大学信息学院)
FFE-Hallu:Hallucinations in Fixed Figurative Expressions:Benchmark of Idioms and Proverbs in the Persian Language
Tehran Institute for Advanced Studies, Khatam University, Iran(德黑兰高级研究 institute,Khatam 大学,伊朗);School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran(电气与计算机工程学院,工程学院,德黑兰大学,德黑兰,伊朗)
Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention
Department of Informatics, King's College London, UK(伦敦国王学院信息学院);Department of Statistics, University of Warwick, UK(沃里克大学统计系);School of Computer Science, The University of Sheffield, UK(谢菲尔德大学计算机科学学院);The Alan Turing Institute, UK(艾伦·图灵研究所)
Detecting LLM Fact-conflicting Hallucinations Enhanced by Temporal-logic-based Reasoning
Huazhong University of Science and Technology(华中科技大学);National University of Singapore(新加坡国立大学);University of New South Wales(新南威尔士大学);Nanyang Technological University(南洋理工大学)
DSC2025 -- ViHallu Challenge: Detecting Hallucination in Vietnamese LLMs
Faculty of Information Science and Engineering, University of Information Technology(信息科学与工程学院,信息技术大学);University of Information Technology, Ho Chi Minh City, Vietnam(信息技术大学,胡志明市,越南);Vietnam National University, Ho Chi Minh City, Vietnam(越南国家大学,胡志明市,越南)
NCL-UoR at SemEval-2025 Task 3: Detecting Multilingual Hallucination and Related Observable Overgeneration Text Spans with Modified RefChecker and Modified SeflCheckGPT
School of Computing, Newcastle University(新萨勒姆大学计算机学院);Department of Computer Science, University of Reading(阅读大学计算机科学系);Previously at School of Computing, Newcastle University(新萨勒姆大学计算机学院)
HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University(北京未来区块链与隐私计算先进研究院,北京航空航天大学人工智能学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室);Beijing Academy of Blockchain and Edge Computing(北京区块链与边缘计算研究院);Renmin University of China(中国人民大学)
Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92%
Astrophysics Science Division, NASA, Goddard Space Flight Center(国家航空航天局(NASA)天体物理学科学部门,戈达德太空飞行中心);Department of Physics, The Catholic University of America(美国天主教大学物理系)
EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models
Lappeenranta-Lahti University of Technology LUT(拉佩兰塔-拉赫蒂技术大学);University of Oulu(奥卢大学);Southeast University(东南大学);Shanghai Jiao Tong University(上海交通大学);Brno University of Technology(布拉格技术大学)
MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations
Université Côte d’Azur, CNRS, LJAD, France(法国埃克塞特大学、法国国家科学研究中心、LJAD研究所);Center for Artificial Intelligence and Data Science (CAIDAS)(人工智能与数据科学中心);Julius-Maximilans-Universität Würzburg, Germany(德国维尔茨堡约利乌斯-马克斯米利安大学)
Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)(模式识别新实验室、多模态人工智能系统国家重点实验室、自动化研究所、中国科学院)
Enhancing Text Authenticity: A Novel Hybrid Approach for AI-Generated Text Detection
University of Pittsburgh(匹兹堡大学);University of Maryland Eastern Shore(马里兰大学东海岸分校);Miami University(迈阿密大学);San Francisco Bay University(旧金山湾大学);Boston University(波士顿大学);Northern Arizona University(北亚利桑那大学)