发表机构
Nantong University; Chongqing University of Post and Telecommunications; China Southern Power Grid Company Limited; Meituan（南通大学; 重庆邮电大学; 中国南方电网有限责任公司; 美团）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对LLM神经元功能一致性的可扩展统计估计难题，提出RACE框架，其领域特异性更优、计算开销低两个数量级，可有效评估Transformer神经元的全领域功能一致性。
AI 中文摘要
在机制可解释性研究中，发现跨整个领域的稳定神经元行为仍是一项挑战。现有方法通常依赖实例级点估计或计算成本高昂的流程，要么掩盖了总体变异性，要么限制了可扩展的全领域分析。我们提出RACE（Residual Alignment for Consistency Estimation，用于一致性估计的残差对齐），这是一种评估Transformer神经元全领域功能一致性的前向传播统计框架。扰动实验表明，与基于梯度的点估计相比，RACE实现了更优异的领域特异性；同时，令牌分布级别的结果验证了所选神经元与目标领域的关联；此外，其计算开销比基于梯度的方法低两个数量级。
英文摘要
Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Compared with gradient-based point estimates, RACE produces neuron rankings that yield more domain-specific effects under perturbation. Token-distribution shifts support the connection between the selected neurons and the target domain, while scoring requires roughly one-hundredth of the computational overhead of the gradient-based methods. Code is available at https://github.com/Nexround/RACE.
CommentsEMNLP-26 Main Conference
