arXiv 2506.098862026-02-24cs.CLcs.AI

基于概率距离的LLM中幻觉检测方法（RAG）

Probabilistic distances-based hallucination detection in LLMs with RAG

Markov Lab, Department of Mathematics（马尔可夫实验室，数学系）；Computer Science, Saint-Petersburg University（计算机科学，圣彼得堡大学）；AI Center, Skoltech（人工智能中心，斯克里普丘克技术学院）；SB AI Lab（SB人工智能实验室）；AI Center, Skoltech, Risk department, Sber（人工智能中心，斯克里普丘克技术学院，风险部门）

Rodion Oblovatny, Alexandra Kuleshova, Konstantin Polev, Alexey Zaytsev

备注：: Updated approach to constructing a hallucination detection score. Added results from experiments with the NLI task. The approach with trainable deep kernels has been removed, with a focus on the unsupervised approach

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种基于概率距离的LLM幻觉检测方法，利用提示标记与响应标记嵌入分布的距离来检测幻觉，具有高效性和可迁移性。

摘要：

在大型语言模型（LLMs）中检测幻觉对于其在许多应用中的安全性至关重要。没有适当的检测，这些系统往往提供有害且不可靠的答案。近年来，LLMs已被积极应用于检索增强生成（RAG）设置中。然而，即使在这一设置中，幻觉仍然存在，尽管已经提出了许多幻觉检测方法，但大多数方法并非专门针对RAG系统。为克服这一限制，我们介绍了一种基于估计提示标记嵌入分布与语言模型响应标记嵌入分布之间距离的幻觉检测方法。该方法通过检查标记隐藏状态的几何结构，可靠地提取文本中的事实性信号，同时对长序列友好。广泛的实验表明，我们的方法实现了最先进的或具有竞争力的性能。它还具有从解决NLI任务转移到幻觉检测任务的可迁移性，使其成为一种完全无监督且高效的具有竞争力最终任务性能的方法。

英文摘要：

Detecting hallucinations in large language models (LLMs) is critical for their safety in many applications. Without proper detection, these systems often provide harmful, unreliable answers. In recent years, LLMs have been actively used in retrieval-augmented generation (RAG) settings. However, hallucinations remain even in this setting, and while numerous hallucination detection methods have been proposed, most approaches are not specifically designed for RAG systems. To overcome this limitation, we introduce a hallucination detection method based on estimating the distances between the distributions of prompt token embeddings and language model response token embeddings. The method examines the geometric structure of token hidden states to reliably extract a signal of factuality in text, while remaining friendly to long sequences. Extensive experiments demonstrate that our method achieves state-of-the-art or competitive performance. It also has transferability from solving the NLI task to the hallucination detection task, making it a fully unsupervised and efficient method with a competitive performance on the final task.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。