知识间的拉锯战：探索并解决检索增强语言模型中的知识冲突
Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models
AI总结：
本文系统研究检索增强语言模型中的知识冲突，揭示其邓宁-克鲁格效应、可得性偏差、多数决原则和确认偏差等行为，并提出冲突解缠对比解码方法CD2以有效解决冲突。
摘要：
检索增强语言模型（RALMs）通过从外部来源检索证据来精炼和扩展其内部记忆，已展现出显著潜力。然而，当RALMs将内部记忆与外部来源整合时，不可避免地会遇到知识冲突。知识冲突可能使RALMs陷入知识间的拉锯战，从而限制其实际适用性。本文重点探索并解决RALMs中的知识冲突。首先，我们提出了一个用于从多个维度评估知识冲突的评估框架。然后，我们从以下两个角度研究RALMs的行为和偏好：（1）内部记忆与外部来源之间的冲突：我们发现更强的RALMs会出现邓宁-克鲁格效应，即使提供了正确证据，也持续偏好其错误的内部记忆。此外，RALMs对常识表现出可得性偏差；（2）真实证据、不相关证据和误导性证据之间的冲突：我们揭示RALMs遵循多数决原则，倾向于信任出现更频繁的证据。此外，我们发现RALMs表现出确认偏差，更愿意选择与其内部记忆一致的证据。为解决知识冲突这一挑战，我们提出了一种名为冲突解缠对比解码（CD2）的方法，以更好地校准模型的置信度。实验结果表明，我们的CD2能够有效解决RALMs中的知识冲突。
英文摘要：
Retrieval-augmented language models (RALMs) have demonstrated significant potential in refining and expanding their internal memory by retrieving evidence from external sources. However, RALMs will inevitably encounter knowledge conflicts when integrating their internal memory with external sources. Knowledge conflicts can ensnare RALMs in a tug-of-war between knowledge, limiting their practical applicability. In this paper, we focus on exploring and resolving knowledge conflicts in RALMs. First, we present an evaluation framework for assessing knowledge conflicts across various dimensions. Then, we investigate the behavior and preference of RALMs from the following two perspectives: (1) Conflicts between internal memory and external sources: We find that stronger RALMs emerge with the Dunning-Kruger effect, persistently favoring their faulty internal memory even when correct evidence is provided. Besides, RALMs exhibit an availability bias towards common knowledge; (2) Conflicts between truthful, irrelevant and misleading evidence: We reveal that RALMs follow the principle of majority rule, leaning towards placing trust in evidence that appears more frequently. Moreover, we find that RALMs exhibit confirmation bias, and are more willing to choose evidence that is consistent with their internal memory. To solve the challenge of knowledge conflicts, we propose a method called Conflict-Disentangle Contrastive Decoding (CD2) to better calibrate the model's confidence. Experimental results demonstrate that our CD2 can effectively resolve knowledge conflicts in RALMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
