弥合外部与参数化知识：通过共享-私有语义协同缓解LLM的幻觉
Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
AI总结：
本文提出DSSP-RAG框架，通过共享-私有语义协同机制，缓解LLM在整合外部知识时的幻觉问题，并提升生成性能。
摘要：
检索增强生成（RAG）旨在通过检索并整合相关外部知识来缓解大型语言模型（LLMs）的幻觉问题。然而，外部知识可能包含噪声并与LLMs的参数化知识冲突，导致性能下降。当前LLMs缺乏解决此类冲突的内在机制。为此，我们提出了一种双流知识增强框架，用于共享-私有语义协同（DSSP-RAG）。其核心在于将传统的自注意力机制细化为混合注意力机制，以区分共享和私有语义，从而实现受控的知识整合。一种无监督的幻觉检测方法能够捕捉LLMs的内在认知不确定性，确保仅在必要时引入外部知识。为减少外部知识中的噪声，提出了一种能量商（EQ），由任务对齐层与任务不匹配层之间的注意力差异矩阵定义。大量实验表明，DSSP-RAG在强基线模型上实现了更优的性能。
英文摘要：
Retrieval-augmented generation (RAG) aims to mitigate the hallucination of Large Language Models (LLMs) by retrieving and incorporating relevant external knowledge into the generation process. However, the external knowledge may contain noise and conflict with the parametric knowledge of LLMs, leading to degraded performance. Current LLMs lack inherent mechanisms for resolving such conflicts. To fill this gap, we propose a Dual-Stream Knowledge-Augmented Framework for Shared-Private Semantic Synergy (DSSP-RAG). Central to it is the refinement of the traditional self-attention into a mixed-attention that distinguishes shared and private semantics for a controlled knowledge integration. An unsupervised hallucination detection method that captures the LLMs' intrinsic cognitive uncertainty ensures that external knowledge is introduced only when necessary. To reduce noise in external knowledge, an Energy Quotient (EQ), defined by attention difference matrices between task-aligned and task-misaligned layers, is proposed. Extensive experiments show that DSSP-RAG achieves a superior performance over strong baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
