arXiv 2410.114142025-01-21cs.CL

ReDeEP：通过机制可解释性检测检索增强生成中的幻觉

ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability

Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Kuaishou Technology Co., Ltd.（快手科技有限公司）；School of Information Technology and Management, University of International Business and Economics（对外经济贸易大学信息学院）

Zhongxiang Sun, Xiaoxue Zang, Kai Zheng, Yang Song, Jun Xu, Xiao Zhang, Weijie Yu, Yang Song, Han Li

备注：: 23pages

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对RAG模型即使有相关检索仍产生幻觉的问题，本文通过机制可解释性发现知识FFN过度强调参数知识与复制头保留外部知识不足是诱因，提出解耦检测方法ReDeEP及缓解方法AARF，显著提升检测准确率。

摘要：

检索增强生成（RAG）模型旨在整合外部知识，以减少因参数化（内部）知识不足而产生的幻觉。然而，即使检索到的内容准确且相关，RAG模型仍可能因生成与检索信息相冲突的输出而产生幻觉。检测此类幻觉需要厘清大型语言模型（LLM）如何利用外部知识与参数化知识。当前的检测方法往往只关注其中一种机制，或未对其交织效应进行解耦，这使得准确检测变得困难。在本文中，我们研究了RAG场景中幻觉背后的内部机制。我们发现，当LLM中的知识前馈神经网络（Knowledge FFNs）在残差流中过度强调参数化知识，而复制头（Copying Heads）未能有效保留或整合来自检索内容的外部知识时，幻觉就会发生。基于这些发现，我们提出了ReDeEP，一种通过解耦LLM对外部上下文和参数化知识的利用来检测幻觉的新方法。我们的实验表明，ReDeEP显著提高了RAG幻觉检测的准确性。此外，我们引入了AARF，通过调节知识前馈神经网络和复制头的贡献来缓解幻觉。

英文摘要：

Retrieval-Augmented Generation (RAG) models are designed to incorporate external knowledge, reducing hallucinations caused by insufficient parametric (internal) knowledge. However, even with accurate and relevant retrieved content, RAG models can still produce hallucinations by generating outputs that conflict with the retrieved information. Detecting such hallucinations requires disentangling how Large Language Models (LLMs) utilize external and parametric knowledge. Current detection methods often focus on one of these mechanisms or without decoupling their intertwined effects, making accurate detection difficult. In this paper, we investigate the internal mechanisms behind hallucinations in RAG scenarios. We discover hallucinations occur when the Knowledge FFNs in LLMs overemphasize parametric knowledge in the residual stream, while Copying Heads fail to effectively retain or integrate external knowledge from retrieved content. Based on these findings, we propose ReDeEP, a novel method that detects hallucinations by decoupling LLM's utilization of external context and parametric knowledge. Our experiments show that ReDeEP significantly improves RAG hallucination detection accuracy. Additionally, we introduce AARF, which mitigates hallucinations by modulating the contributions of Knowledge FFNs and Copying Heads.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。