arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.08663cs.CLcs.AIcs.CY
收藏

幻觉检测:一种使用嵌入距离分析的概率框架

Hallucination Detection: A Probabilistic Framework Using Embeddings Distance Analysis

University of Pisa(比萨大学)

Emanuele Ricco, Lorenzo Cima, Roberto Di Pietro

AI总结:

针对大语言模型幻觉问题,提出基于嵌入空间闵可夫斯基距离分析的数学严谨方法,首次证明幻觉内容与正确内容的结构差异,开发检测工具在特定配置下准确率达66%,为该领域研究提供新方向。

摘要:

幻觉是影响大语言模型(LLMs)的主要问题之一,阻碍其在生产系统中的广泛应用。当前幻觉检测的研究方案主要基于启发式方法,本文提出一种数学严谨的方法来推理幻觉,并利用该方法构建幻觉检测工具。据我们所知,我们首次证明幻觉内容与正确内容存在结构差异。为验证这一结果,我们采用嵌入空间中的闵可夫斯基距离(Minkowski distances)。研究发现,嵌入距离分布存在统计显著差异,且具有尺度不变性——无论使用何种距离范数、关键词数量、问题或响应数量,这些差异在性质上均成立。我们利用这些结构差异开发了一个幻觉响应检测工具,在特定系统参数配置下准确率达66%,与该领域最佳结果相当。总之,所提出的方法具有创新性和应用前景,可能为该领域的进一步研究(包括我们未来工作中强调的方向)铺平道路。

英文摘要:

Hallucinations are one of the major issues affecting LLMs, hindering their wide adoption in production systems. While current research solutions for detecting hallucinations are mainly based on heuristics, in this paper we introduce a mathematically sound methodology to reason about hallucination, and leverage it to build a tool to detect hallucinations. To the best of our knowledge, we are the first to show that hallucinated content has structural differences with respect to correct content. To prove this result, we resort to the Minkowski distances in the embedding space. Our findings demonstrate statistically significant differences in the embedding distance distributions, that are also scale free -- they qualitatively hold regardless of the distance norm used and the number of keywords, questions, or responses. We leverage these structural differences to develop a tool to detect hallucinated responses, achieving an accuracy of 66\% for a specific configuration of system parameters -- comparable with the best results in the field. In conclusion, the suggested methodology is promising and novel, possibly paving the way for further research in the domain, also along the directions highlighted in our future work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑