arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.02688cs.CL
收藏

Flash Interpretability: 使用 LM-Head 解码大型语言模型中的特化特征神经元

Flash Interpretability: Decoding Specialised Feature Neurons in Large Language Models with the LM-Head

Imperial College London(帝国理工学院)

Harry J Davies

AI总结:

本研究提出一种通过 LM-head 直接解码神经元权重为词元概率的方法,在 Llama 3.1 8B 上识别特化特征神经元,并验证其有效性,实现了快速且计算高效的神经元特征映射。

摘要:

大型语言模型(LLMs)通常具有数十亿个参数,因此其运作过程往往难以解释。在本工作中,我们证明可以通过模型的最终投影层(LM-head)直接将神经元权重解码为词元概率。这一点在 Llama 3.1 8B 中得到展示,我们使用 LM-head 寻找特化特征神经元的示例,例如“狗”神经元和“加利福尼亚”神经元,并通过钳制这些神经元来影响输出中该概念的概率,从而验证了这一点。我们在预训练模型和 Instruct 模型上评估了该方法,发现在 instruct 模型的上投影层中,超过 75% 的神经元与预训练模型具有相同的最高关联词元。最后,我们证明钳制“狗”神经元会使 instruct 模型在被问及最喜欢的动物时总是讨论狗。通过我们的方法,可以在不到 10 秒内以最小的计算量映射 Llama 3.1 8B 全部上投影神经元的顶层特征。

英文摘要:

Large Language Models (LLMs) typically have billions of parameters and are thus often difficult to interpret in their operation. In this work, we demonstrate that it is possible to decode neuron weights directly into token probabilities through the final projection layer of the model (the LM-head). This is illustrated in Llama 3.1 8B where we use the LM-head to find examples of specialised feature neurons such as a "dog" neuron and a "California" neuron, and we validate this by clamping these neurons to affect the probability of the concept in the output. We evaluate this method on both the pre-trained and Instruct models, finding that over 75% of neurons in the up-projection layers in the instruct model have the same top associated token compared to the pretrained model. Finally, we demonstrate that clamping the "dog" neuron leads the instruct model to always discuss dogs when asked about its favourite animal. Through our method, it is possible to map the top features of the entirety of Llama 3.1 8B's up-projection neurons in less than 10 seconds, with minimal compute.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑