arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.28264cs.AI
收藏

熵分布作为生成模型中幻觉的指纹

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models

Global Technology Applied Research(全球技术应用研究)

Mattia J. Villani, Pranav Deshpande, Akshay Seshadri, Romina Yalovetzky, Niraj Kumar

AI总结:

本文提出基于token级熵分布(而非仅均值)的校准熵分数(CES),通过单次前向传递和黑盒logits访问实现幻觉检测,并提供理论保证和实证验证。

摘要:

大型语言模型(LLMs)经常生成事实上不正确的输出,通常称为幻觉,这削弱了信任并限制了在高风险环境中的部署。现有的幻觉检测方法通常需要多次前向传递或访问模型内部。在这项工作中,我们提供了理论背景和实证证据,表明token级熵的分布(超越困惑度或长度归一化熵所捕获的均值)作为幻觉的指纹,其分布形状和尾部行为携带独立信号。我们将幻觉检测形式化为统计假设检验,并提出校准熵分数(CES),一种轻量级算法,仅需单次前向传递和黑盒访问token logits。CES通过校准的参考CDF将均值信号与生成熵的最大信号相结合,产生可直接跨模型和任务比较的分数。我们通过新颖的随机长度Dvoretzky-Kiefer-Wolfowitz不等式建立了有限样本校准保证,并证明了CES检测幻觉的概率随生成长度指数级收敛到1。在八个QA基准和十个生成模型(涵盖开源和API访问模型)上,CES在所有单次黑盒方法中实现了最高的检测性能,同时提供了现有启发式方法所缺乏的正式误差保证。值得注意的是,CES在统计上与需要更高计算成本的多样本方法无法区分,缩小了轻量级与昂贵检测之间的差距,使其适用于实时、大规模部署。

英文摘要:

Large Language Models (LLMs) often generate factually incorrect outputs, commonly termed hallucinations, that undermine trust and limit deployment in high-stakes settings. Existing hallucination detection methods typically require multiple forward passes, or access to model internals. In this work, we provide theoretical background and empirical evidence that the distribution of token-level entropies, beyond the mean captured by perplexity or length-normalised entropy, serves as a fingerprint of hallucination, with distributional shape and tail behaviour carrying independent signal. We formalize hallucination detection as a statistical hypothesis test and propose the Calibrated Entropy Score (CES), a lightweight algorithm requiring only a single forward pass and black-box access to token logits. CES combines the mean signal with the maximum signal of the generated entropy through a calibrated reference CDF, producing scores that are directly comparable across models and tasks. We establish finite-sample calibration guarantees via a novel random-length Dvoretzky--Kiefer--Wolfowitz inequality, and also prove that CES detects hallucinations with probability converging to one exponentially fast in the generation length. Across eight QA benchmarks and ten generator models spanning open-source and API access models, CES achieves the highest detection performance among all single-pass black-box methods while providing formal error guarantees that existing heuristics lack. Remarkably, CES is statistically indistinguishable from multi-sample methods that require far greater computational cost, closing the gap between lightweight and expensive detection and making it suitable for real-time, large-scale deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑