arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.06005cs.CL
收藏

解构词汇空间中的MLP神经元权重

Disentangling MLP Neuron Weights in Vocabulary Space

Blavatnik School of Computer Science and AI, Tel Aviv University(特拉维夫大学布拉瓦特尼克计算机科学与人工智能学院)

Asaf Avrahamy, Yoav Gur-Arieh, Mor Geva

AI总结:

本文提出ROTATE方法,通过优化神经元权重旋转以最大化词汇空间的峰度,解构MLP神经元权重,揭示词汇通道,提升语言模型可解释性。

摘要:

解释模型权重中的信息仍是机制可解释性中的基本挑战。本文引入ROTATE(旋转优化的词嵌入对齐),一种无需前向传播的数据自由方法,直接在权重空间中解构MLP神经元。我们的方法基于关键统计观察:编码一致、单义概念的神经元在投影到模型词汇空间时表现出高峰度。通过优化神经元权重的旋转以最大化词汇空间的峰度,我们的方法恢复了稀疏、可解释的方向,称为词汇通道。在Llama-3.1-8B-Instruct和Gemma-2-2B-it上的实验表明,ROTATE一致恢复了忠实于神经元行为的词汇通道。消去单个通道会选择性地禁用相应的输入激活或特定概念的促进。此外,聚合通道级描述可获得全面的神经元描述,在与优化激活基线的头对头比较中表现优异。通过提供神经元权重的数据自由分解,ROTATE为解释语言模型提供了可扩展、细粒度的构建块。

英文摘要:

Interpreting the information encoded in language model weights remains a fundamental challenge in mechanistic interpretability. In this work, we introduce ROTATE (Rotation-Optimized Token Alignment in weighT spacE), a data-free method requiring no forward passes that disentangles MLP neurons directly in weight space. Our approach relies on a key statistical observation: neurons that encode coherent, monosemantic concepts exhibit high kurtosis when projected onto the model's vocabulary. By optimizing rotations of neuron weights to maximize their vocabulary-space kurtosis, our method recovers sparse, interpretable directions which we name vocabulary channels. Experiments on Llama-3.1-8B-Instruct and Gemma-2-2B-it demonstrate that ROTATE consistently recovers vocabulary channels that are faithful to the neuron's behavior; ablating individual channels selectively disables corresponding input activations or the promotion of specific concepts. Moreover, aggregating channel-level descriptions yields comprehensive neuron descriptions that outperform optimized activation-based baselines by 2-3x in head-to-head comparisons. By providing a data-free decomposition of neuron weights, ROTATE offers a scalable, fine-grained building block for interpreting language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑