上下文复制调制：熵神经元在管理参数化与上下文知识冲突中的作用
Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts
AI总结：
本文研究了熵神经元在大型语言模型面临上下文与参数化知识冲突时抑制上下文复制行为的作用，发现消融这些神经元会显著改变生成过程，从而加深了对模型内部动态的理解。
摘要：
大型语言模型（LLMs）在面临与其内部参数化知识相冲突的上下文信息时，其行为是不一致的，目前对于其预期结果的分布尚无普遍接受的解释。近期的研究在自回归 transformer 模型中发现了一类神经元——称为熵神经元——它们对模型输出熵产生显著影响，同时对预测 token 的排名产生总体适度的作用。在本文中，我们通过考察这些神经元在解决上下文信息与参数化信息之间冲突中的作用，调查了关于这些神经元参与抑制 transformer 中上下文复制行为的初步主张。我们表明，熵神经元负责在一系列 LLMs 中抑制上下文复制，并且消融它们会导致生成过程发生显著变化。这些结果增强了我们对 LLMs 在处理冲突信息时内部动态的理解。
英文摘要：
The behavior of Large Language Models (LLMs) when facing contextual information that conflicts with their internal parametric knowledge is inconsistent, with no generally accepted explanation for the expected outcome distribution. Recent work has identified in autoregressive transformer models a class of neurons -- called entropy neurons -- that produce a significant effect on the model output entropy while having an overall moderate impact on the ranking of the predicted tokens. In this paper, we investigate the preliminary claim that these neurons are involved in inhibiting context copying behavior in transformers by looking at their role in resolving conflicts between contextual and parametric information. We show that entropy neurons are responsible for suppressing context copying across a range of LLMs, and that ablating them leads to a significant change in the generation process. These results enhance our understanding of the internal dynamics of LLMs when handling conflicting information.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
