Hallucinations are one of the major issues affecting LLMs, hindering their wide adoption in production systems. While current research solutions for detecting hallucinations are mainly based on heuristics, in this paper we introduce a mathematically sound methodology to reason about hallucination, and leverage it to build a tool to detect hallucinations. To the best of our knowledge, we are the first to show that hallucinated content has structural differences with respect to correct content. To prove this result, we resort to the Minkowski distances in the embedding space. Our findings demonstrate statistically significant differences in the embedding distance distributions, that are also scale free -- they qualitatively hold regardless of the distance norm used and the number of keywords, questions, or responses. We leverage these structural differences to develop a tool to detect hallucinated responses, achieving an accuracy of 66\% for a specific configuration of system parameters -- comparable with the best results in the field. In conclusion, the suggested methodology is promising and novel, possibly paving the way for further research in the domain, also along the directions highlighted in our future work.
Probabilistic distances-based hallucination detection in LLMs with RAG
Markov Lab, Department of Mathematics(马尔可夫实验室,数学系);Computer Science, Saint-Petersburg University(计算机科学,圣彼得堡大学);AI Center, Skoltech(人工智能中心,斯克里普丘克技术学院);SB AI Lab(SB人工智能实验室);AI Center, Skoltech, Risk department, Sber(人工智能中心,斯克里普丘克技术学院,风险部门)
A Geometric Analysis of Small-sized Language Model Hallucinations
Engineering (CEMSE) division, King Abdullah University of Science and Technology (KAUST)(卡塔尔科技大学工程学院(CEMSE));Istituto di Informatica e Telematica (IIT), National Research Council of Italy (CNR)(意大利国家研究理事会信息与电信研究所(IIT));Department of Information Engineering, University of Pisa(比萨大学信息工程系)
From Flat Facts to Sharp Hallucinations: Detecting Stubborn Errors via Gradient Sensitivity
School of Intelligent Manufacturing Ecosystem, Xi’an Jiaotong-Liverpool University, People’s Republic of China(智能制造生态系统学院,西安交通大学-利物浦大学,中华人民共和国);Department of Computer Science, University of Liverpool, United Kingdom(计算机科学系,利物浦大学,英国);Faculty of Engineering and Technology, Sunway University, Malaysia(工程与技术学院,Sunway大学,马来西亚);School of Robotics, Xi’an Jiaotong-Liverpool University, People’s Republic of China(机器人学院,西安交通大学-利物浦大学,中华人民共和国)
HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models
Simula Metropolitan Center for Digital Engineering (SimulaMet)(Simula数字工程研究中心);Oslo Metropolitan University (OsloMet)(奥斯陆 Metropolitan 大学);Simula Research Laboratory(Simula研究实验室)
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
Provable Responsible AI and Data Analytics (PRADA) Lab(可证明负责任的人工智能与数据分析实验室);King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹大学);University of Electronic Science and Technology of China(中国电子科学技术大学);University of Copenhagen(哥本哈根大学)
Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models
School of Computation, Information and Technology, Technical University of Munich(计算、信息与技术学院,慕尼黑技术大学);School of Social Sciences and Technology, Technical University of Munich(社会科学与技术学院,慕尼黑技术大学)
Innovation: An Almost Characterization of Hallucination
School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai, Maharashtra - 400 005, India(技术与计算机科学学院,塔塔基础研究机构,孟买,马哈拉施特拉邦 - 400 005, 印度)
Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability
U2IS, ENSTA, Institut Polytechnique de Paris(巴黎综合理工学院ENSTA学院U2IS实验室);ISIR, Université Sorbonne, Pierre et Marie Curie(索邦大学皮埃尔和玛丽·居里分校ISIR实验室);AMIAD, Pôle Recherche(AMIAD研究部)
Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor
College of Computer Science, Nankai University(南开大学计算机科学学院);Haihe Lab of ITAI College of intelligent Science and Technology, Inner Mongolia University of Technology(内蒙古工业大学智能科学与技术学院海河实验室);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Department of Electrical and Computer Engineering, Stony Brook University(石溪大学电气与计算机工程系);Department of Computer Science, Michigan Technological University(密歇根技术大学计算机科学系)
Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
Institute for Clarity in Documentation(文献清晰研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康校区);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)