IRCAN：通过识别和重加权上下文感知神经元来缓解LLM生成中的知识冲突
IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons
AI总结：
为缓解LLM生成中参数化知识与上下文新知识的冲突，提出IRCAN框架，通过积分梯度识别并重加权上下文感知神经元，引导模型生成上下文敏感输出，实验证明其效果显著且可即插即用。
摘要：
人们普遍认为，大型语言模型（LLMs）在海量数据上训练后会编码大量的知识。最近的研究揭示了LLM生成中的知识冲突现象，即过时或错误的参数化知识（即编码知识）与上下文中提供的新知识相矛盾。为了缓解此类知识冲突，我们提出了一个新颖的框架，IRCAN（Identifying and Reweighting Context-Aware Neurons，识别和重加权上下文感知神经元），以利用在处理上下文线索中至关重要的神经元。具体而言，IRCAN首先利用源自积分梯度的上下文感知归因分数，识别对上下文处理有显著贡献的神经元。随后，通过重加权来增强所识别的上下文感知神经元。这样做可以引导LLMs针对上下文中提供的新知识生成上下文敏感的输出。在多种模型和任务上进行的广泛实验表明，IRCAN不仅在处理知识冲突方面取得了显著改善，而且还提供了一个可扩展的、即插即用的解决方案，能够与现有模型无缝集成。我们的代码已在 https://github.com/danshi777/IRCAN 发布。
英文摘要：
It is widely acknowledged that large language models (LLMs) encode a vast reservoir of knowledge after being trained on mass data. Recent studies disclose knowledge conflicts in LLM generation, wherein outdated or incorrect parametric knowledge (i.e., encoded knowledge) contradicts new knowledge provided in the context. To mitigate such knowledge conflicts, we propose a novel framework, IRCAN (Identifying and Reweighting Context-Aware Neurons) to capitalize on neurons that are crucial in processing contextual cues. Specifically, IRCAN first identifies neurons that significantly contribute to context processing, utilizing a context-aware attribution score derived from integrated gradients. Subsequently, the identified context-aware neurons are strengthened via reweighting. In doing so, we steer LLMs to generate context-sensitive outputs with respect to the new knowledge provided in the context. Extensive experiments conducted across a variety of models and tasks demonstrate that IRCAN not only achieves remarkable improvements in handling knowledge conflicts but also offers a scalable, plug-and-play solution that can be integrated seamlessly with existing models. Our codes are released at https://github.com/danshi777/IRCAN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
