Despite the many advances of Large Language Models (LLMs) and their unprecedented rapid evolution, their impact and integration into every facet of our daily lives is limited due to various reasons. One critical factor hindering their widespread adoption is the occurrence of hallucinations, where LLMs invent answers that sound realistic, yet drift away from factual truth. In this paper, we present a novel method for detecting hallucinations in large language models, which tackles a critical issue in the adoption of these models in various real-world scenarios. Through extensive evaluations across multiple datasets and LLMs, including Llama-2, we study the hallucination levels of various recent LLMs and demonstrate the effectiveness of our method to automatically detect them. Notably, we observe up to 87% hallucinations for Llama-2 in a specific experiment, where our method achieves a Balanced Accuracy of 81%, all without relying on external knowledge.
Halluscoring 2026: The first shared task on llms hallucination detection and answer verification
King Fahd University of Petroleum and Minerals(法赫德国王石油矿产大学);Hamad Bin Khalifa University(哈马德·本·哈利法大学);University of the Basque Country(巴斯克大学);Imam Abdulrahman bin Faisal University(伊玛目阿卜杜勒拉赫曼·本·费萨尔大学);University of Biskra(比斯克拉大学);University of British Columbia(不列颠哥伦比亚大学);Universiti Malaysia Kelantan(马来西亚吉兰丹大学);King Saud University(沙特国王大学)
A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Center for Data Science, Peking University(北京大学数据科学中心);The MOE Key Laboratory of Computational Linguistics, Peking University(北京大学计算语言学教育部重点实验室)
Confabulations from ACL Publications (CAP): A Dataset for Scientific Hallucination Detection
Charles University, Prague(查尔斯大学,布拉格);Independent Researcher(独立研究者);University of Helsinki, Finland(赫尔辛基大学);University of British Columbia, Vancouver, Canada(不列颠哥伦比亚大学,温哥华,加拿大);Chosun University, South Korea(成均馆大学,韩国)
Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG
Lero, the Research Ireland Centre for Software, University of Limerick(Lero,爱尔兰科学基金会软件研究中心,利默里克大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Ottawa(渥太华大学)
HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answering
Hamad Bin Khalifa University(哈马德·本·哈利法大学);Qatar University(卡塔尔大学);University of Biskra(比斯克拉大学);University of the Basque Country(巴斯克大学);Nazarbayev University(纳扎尔巴耶夫大学);Universiti Malaysia Kelantan(马来西亚吉兰丹大学)
Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer
Graduate School of Data Science(数据科学研究生院);Department of Rural Systems Engineering(农村系统工程系);Electrical Engineering and Computer Science(电子工程与计算机科学);Department of Aerospace Engineering(航空航天工程系)
SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Models
Department of Mathematics and Computer Science, University of Catania(卡塔尼亚大学数学与计算机科学系);L3S Research Center, Leibniz University Hannover(汉诺威莱布尼茨大学L3S研究中心)
DelucionQA: Detecting Hallucinations in Domain-specific Question Answering
University of Illinois Chicago(伊利诺伊大学芝加哥分校);Bosch Research North America(博世北美研究院);Bosch Center for Artificial Intelligence (BCAI)(博世人工智能中心(BCAI));UNC Chapel-Hill(北卡罗来纳大学教堂山分校)
HalluScore: Large Language Model Hallucination Question Answering Benchmark
Department of Information and Computer Science, King Fahd University of Petroleum and Minerals(国王法赫德石油与矿物大学信息与计算机科学系);SDAIA-KFUPM Joint Research Center for Artificial Intelligence(SDAIA-KFUPM人工智能联合研究中心)
HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);China Industrial Control Systems Cyber Emergency Response Team(中国工业控制系统网络应急响应团队)
Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer
Technion – Israel Institute of Technology(技术ion-以色列理工学院);University of Oxford and WhiteBox(牛津大学和WhiteBox);School of Computer Science and Engineering, The Hebrew University of Jerusalem(耶路撒冷希伯来大学计算机科学与工程学院)
Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs
University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学);Electronic Science Research Institute of China Electronics(中国电子科技集团电子科学研究院)
Do LLM hallucination detectors suffer from low-resource effect?
Indian Institute of Technology Kharagpur(印度理工学院卡拉格普尔分校);Ruhr University Bochum(波鸿鲁尔大学);UAR Research Center for Trustworthy Data Science and Security(UAR可信数据科学与安全研究中心)
Investigating Hallucination in Conversations for Low Resource Languages
University of North Alabama(北阿拉巴马大学);Wichita State University(威奇托州立大学);Murray State University(默里州立大学);California State Polytechnic University Pomona(加州州立理工大学波莫纳分校);Auburn University at Montgomery(奥本大学蒙哥马利分校);Auburn University(奥本大学);Gustavus Adolphus College(古斯塔夫阿道夫学院);Stanford University(斯坦福大学);Meta;Amazon GenAI(亚马逊生成式人工智能)
A Concise Review of Hallucinations in LLMs and their Mitigation
Computer Engineering K J Somaiya Institute of Technology Mumbai, India;Data Science K J Somaiya Institute of Technology Mumbai, India;Information Technology K J Somaiya Institute of Technology Mumbai, India