神经元以范围说话：摆脱离散神经元归因
Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution
AI总结：
研究发现大语言模型中神经元具有多义性，提出NeuronLens框架通过范围归因提升模型可解释性和可控性。
摘要：
大语言模型中的广泛多义性削弱了离散神经元-概念归因，对模型解释和控制构成重大挑战。我们系统分析了基于编码器和解码器的LLMs，在不同数据集上观察到，即使对于特定语义概念具有高显著性的神经元也表现出多义行为。重要的是，我们发现一种一致模式：概念条件下的神经元激活幅度形成 distinct、往往呈高斯分布的分布，重叠最小。基于这一观察，我们假设通过解释和干预特定概念的激活范围，可以在LLMs中实现更精确的可解释性和定向操控。为此，我们引入了NeuronLens，一种基于范围的解释和操控框架，将概念归因局部化到神经元内的激活范围。广泛的实证评估显示，基于范围的干预能够有效操控目标概念，同时对辅助概念和整体模型性能的副作用比神经元级别遮蔽要小得多。
英文摘要：
Pervasive polysemanticity in large language models (LLMs) undermines discrete neuron-concept attribution, posing a significant challenge for model interpretation and control. We systematically analyze both encoder and decoder based LLMs across diverse datasets, and observe that even highly salient neurons for specific semantic concepts consistently exhibit polysemantic behavior. Importantly, we uncover a consistent pattern: concept-conditioned activation magnitudes of neurons form distinct, often Gaussian-like distributions with minimal overlap. Building on this observation, we hypothesize that interpreting and intervening on concept-specific activation ranges can enable more precise interpretability and targeted manipulation in LLMs. To this end, we introduce NeuronLens, a novel range-based interpretation and manipulation framework, that localizes concept attribution to activation ranges within a neuron. Extensive empirical evaluations show that range-based interventions enable effective manipulation of target concepts while causing substantially less collateral degradation to auxiliary concepts and overall model performance compared to neuron-level masking.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
