砍掉头部即可终结冲突：一种解释并缓解语言模型中知识冲突的机制
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models
AI总结：
本文从信息流视角揭示语言模型中记忆头与上下文头整合矛盾信息导致知识冲突的机制，并提出无需更新参数的 PH3 注意力头剪枝方法，以可控地增强内部记忆或外部上下文利用并提升开放域问答性能。
摘要：
近来，检索增强和工具增强通过提供外部上下文，展现出扩展语言模型（LM）内部记忆边界的显著能力。然而，内部记忆与外部上下文不可避免地发生冲突，导致 LM 内部出现知识冲突。本文旨在从信息流的视角解释知识冲突的机制，随后通过在关键节点进行精确干预来缓解冲突。我们发现，在后几层中存在一些具有相反作用的注意力头：记忆头能够从内部记忆中唤起知识，而上下文头能够从外部上下文中检索知识。此外，我们揭示 LM 中知识冲突出现的关键节点，是记忆头与上下文头对不一致信息流的整合。受这些洞见启发，我们提出一种名为 Pruning Head via PatH PatcHing（PH3）的新方法，该方法能够通过剪枝相互冲突的注意力头来高效缓解知识冲突，且无需更新模型参数。PH3 可以灵活控制八个 LM 使用内部记忆（提升 44.0%）或外部上下文（提升 38.5%）。此外，PH3 还能够提升 LM 在开放域问答任务上的性能。我们还开展了大量实验，证明该方法具有跨模型、跨关系和跨格式的泛化能力。
英文摘要：
Recently, retrieval augmentation and tool augmentation have demonstrated a remarkable capability to expand the internal memory boundaries of language models (LMs) by providing external context. However, internal memory and external context inevitably clash, leading to knowledge conflicts within LMs. In this paper, we aim to interpret the mechanism of knowledge conflicts through the lens of information flow, and then mitigate conflicts by precise interventions at the pivotal point. We find there are some attention heads with opposite effects in the later layers, where memory heads can recall knowledge from internal memory, and context heads can retrieve knowledge from external context. Moreover, we reveal that the pivotal point at which knowledge conflicts emerge in LMs is the integration of inconsistent information flows by memory heads and context heads. Inspired by the insights, we propose a novel method called Pruning Head via PatH PatcHing (PH3), which can efficiently mitigate knowledge conflicts by pruning conflicting attention heads without updating model parameters. PH3 can flexibly control eight LMs to use internal memory ($\uparrow$ 44.0%) or external context ($\uparrow$ 38.5%). Moreover, PH3 can also improve the performance of LMs on open-domain QA tasks. We also conduct extensive experiments to demonstrate the cross-model, cross-relation, and cross-format generalization of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
