我们提出了语义熵探针(Semantic Entropy Probes, SEPs),这是一种用于大语言模型(Large Language Models, LLMs)不确定性量化的低成本可靠方法。幻觉指的是听起来合理但事实错误且随意的模型生成内容,是LLM实际应用面临的主要挑战。Farquhar等人(2024)的近期研究提出了语义熵(semantic entropy, SE),可通过估计一组模型生成内容在语义空间中的不确定性来检测幻觉,但SE计算带来的5到10倍计算成本增长阻碍了其实际应用。为解决这一问题,我们提出SEPs,可从单次生成的隐藏状态中直接近似SE。SEPs训练简单,测试时无需采样多个模型生成结果,将语义不确定性量化的开销降至几乎为零。我们证明,SEPs在幻觉检测上保持了高性能,且相较于直接预测模型准确率的既往探针方法,对分布外数据的泛化性更好。我们在不同模型和任务上的结果表明,模型隐藏状态能够捕获SE,消融实验进一步揭示了这一特性对应的token位置和模型层。
英文摘要:
We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs). Hallucinations, which are plausible-sounding but factually incorrect and arbitrary model generations, present a major challenge to the practical adoption of LLMs. Recent work by Farquhar et al. (2024) proposes semantic entropy (SE), which can detect hallucinations by estimating uncertainty in the space semantic meaning for a set of model generations. However, the 5-to-10-fold increase in computation cost associated with SE computation hinders practical adoption. To address this, we propose SEPs, which directly approximate SE from the hidden states of a single generation. SEPs are simple to train and do not require sampling multiple model generations at test time, reducing the overhead of semantic uncertainty quantification to almost zero. We show that SEPs retain high performance for hallucination detection and generalize better to out-of-distribution data than previous probing methods that directly predict model accuracy. Our results across models and tasks suggest that model hidden states capture SE, and our ablation studies give further insights into the token positions and model layers for which this is the case.
Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning
College of AI, Tsinghua University(清华大学人工智能学院);Department of Electronic Engineering, BNRist, Tsinghua University(清华大学电子工程系);School of Electronic and Information Engineering, Beijing Jiaotong University(北京交通大学电子信息学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs
Mohamed bin Zayed University of Artificial Intelligence(莫扎德·本·泽亚德人工智能大学);Fraunhofer Institute for Open Communication Systems(弗劳恩霍夫开放通信系统研究所);Technical University of Munich(慕尼黑技术大学);New York University Abu Dhabi(纽约大学阿布扎克分校);Alibaba International Digital Commerce(阿里巴巴国际数字商业)
FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs
Vectara, Inc.(Vectara公司);Iowa State University(爱荷华州立大学);University of Southern California(南加利福尼亚大学);Entropy Technologies(熵技术公司);University of Waterloo(滑铁卢大学);University of Wisconsin, Madison(威斯康星大学麦迪逊分校)
Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?
RISE Research Institutes of Sweden, Department of Computer Science(瑞典RISE研究机构,计算机科学系);Uppsala University, Department of Linguistics and Philology(乌普萨拉大学,语言学与哲学系);KTH Royal Institute of Technology, Division of Software and Computer Systems(皇家理工学院,软件与计算机系统系);University of Edinburgh, School of Informatics(爱丁堡大学,信息学院)
DSC2025 -- ViHallu Challenge: Detecting Hallucination in Vietnamese LLMs
Faculty of Information Science and Engineering, University of Information Technology(信息科学与工程学院,信息技术大学);University of Information Technology, Ho Chi Minh City, Vietnam(信息技术大学,胡志明市,越南);Vietnam National University, Ho Chi Minh City, Vietnam(越南国家大学,胡志明市,越南)
HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
CS Department Virginia Tech(弗吉尼亚理工学院计算机科学系);EECS Department MIT(麻省理工学院电子工程与计算机科学系);CS Department Dartmouth College(达特茅斯学院计算机科学系);Statistics Department Virginia Tech(弗吉尼亚理工学院统计学系);CS Department Michigan State University(密歇根州立大学计算机科学系)
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Tencent AI lab(腾讯AI实验室);Soochow University(苏州大学);Zhejiang University(浙江大学);Renmin University of China(中国人民大学);Nanyang Technological University(南洋理工大学);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Institute for AI, Tsinghua University(清华大学人工智能研究院);Huzhou University(湖州大学);Institute of Dataspace, Hefei Comprehensive National Science Center(合肥综合性国家科学中心数据空间研究院);University of Science and Technology of China(中国科学技术大学)
Probabilistic distances-based hallucination detection in LLMs with RAG
Markov Lab, Department of Mathematics(马尔可夫实验室,数学系);Computer Science, Saint-Petersburg University(计算机科学,圣彼得堡大学);AI Center, Skoltech(人工智能中心,斯克里普丘克技术学院);SB AI Lab(SB人工智能实验室);AI Center, Skoltech, Risk department, Sber(人工智能中心,斯克里普丘克技术学院,风险部门)
Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)(模式识别新实验室、多模态人工智能系统国家重点实验室、自动化研究所、中国科学院)
How to Detect and Defeat Molecular Mirage: A Metric-Driven Benchmark for Hallucination in LLM-based Molecular Comprehension
Shenzhen Graduate School, Peking University(北京大学深圳研究生院);International Digital Economy Academy (IDEA)(国际数字经济学院(IDEA));Pengcheng Laboratory(鹏城实验室);Hong Kong University of Science and Technology(香港科技大学)