Large language models (LLMs) have generated significant attention since their inception, finding applications across various academic and industrial domains. However, these models often suffer from the "hallucination problem", where outputs, though grammatically and logically coherent, lack factual accuracy or are entirely fabricated. A particularly troubling issue discovered and widely discussed recently is the numerical comparison error where multiple LLMs incorrectly infer that "9.11$>$9.9". We discovered that the order in which LLMs generate answers and reasoning impacts their consistency. Specifically, results vary significantly when an LLM generates an answer first and then provides the reasoning versus generating the reasoning process first and then the conclusion. Inspired by this, we propose a new benchmark method for assessing LLM consistency: comparing responses generated through these two different approaches. This benchmark effectively identifies instances where LLMs fabricate answers and subsequently generate justifications. Furthermore, we introduce a novel and straightforward prompt strategy designed to mitigate this issue. Experimental results demonstrate that this strategy improves performance across various LLMs compared to direct questioning. This work not only sheds light on a critical flaw in LLMs but also offers a practical solution to enhance their reliability.
"Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing
AI Institute, University of South Carolina(南卡罗来纳大学AI研究院);Indian Institute of Technology, Kharagpur(印度理工学院克勒格布尔分校);Islamic University of Technology(伊斯兰科技大学);Stanford University(斯坦福大学);Amazon AI(亚马逊AI研究院)
BenHalluEval: A Multi-Task Hallucination Evaluation Framework for Large Language Models on Bengali
Department of Computer Science and Engineering, Islamic University of Technology(伊斯兰科技大学计算机科学与工程系);Department of Computer Science and Engineering, University of California(加州大学计算机科学与工程系)
AI Hallucination from Students' Perspective: A Thematic Analysis
Department of Computer and Information Engineering, Center for Cyber Physical Systems, Khalifa University(计算机与信息工程系,跨物理系统中心,哈利法大学);Department of Psychology, University Malaysia Sabah(心理学系,马来西亚Sabah大学)
Predicting LLM Correctness in Prosthodontics Using Metadata and Hallucination Signals
1 Department of Data Science, Monash University Indonesia, Tangerang, Indonesia;2 Independent Researcher;3 Department of Prosthodontics, University of Pittsburgh, Pittsburgh, Pennsylvania;4 Department of Restorative Sciences, University of North Carolina Adams School of Dentistry, Chapel Hill, North Carolina;5 Department of Computer Science, Boston University, Boston, Massachusetts
DSC2025 -- ViHallu Challenge: Detecting Hallucination in Vietnamese LLMs
Faculty of Information Science and Engineering, University of Information Technology(信息科学与工程学院,信息技术大学);University of Information Technology, Ho Chi Minh City, Vietnam(信息技术大学,胡志明市,越南);Vietnam National University, Ho Chi Minh City, Vietnam(越南国家大学,胡志明市,越南)
Copy-Paste to Mitigate Large Language Model Hallucinations
Department of Computer Science, Tianjin University of Technology(天津理工大学计算机学院);National Institute of Health Data Science, Peking University(北京大学国家健康数据科学研究院);Tencent Jarvis Lab(腾讯Jarvis实验室)
SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine
International Digital Economy Academy(国际数字经济学院);School of Software & Microelectronics, Peking University(北京大学软件与微电子学院);National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室);School of Computer Science, Peking University(北京大学计算机学院);Data Science and Artificial Intelligence, Monash University(莫纳什大学数据科学与人工智能学院);ModelTC
Hallucination as a Feature, not a Defect: Evaluating a multi-agent architecture to transform speculative language-model outputs into testable scientific hypotheses
Topology-Aware Reasoning over Incomplete Knowledge Graph with Graph-Based Soft Prompting
Chalmers University of Technology and University of Gothenburg, Sweden(楚姆勒大学技术学院和哥德堡大学,瑞典);Technical University of Denmark, Kgs. Lyngby, Denmark(丹麦技术大学,基斯勒吕辛,丹麦)
Evaluating Hallucinations in Audio-Visual Multimodal LLMs with Spoken Queries under Diverse Acoustic Conditions
Department of Intelligent Software, Sungkyunkwan University(智能软件系,成均馆大学);Department of Computer Science and Engineering, Sungkyunkwan University(计算机科学与工程系,成均馆大学)