通过交互感知反向传播修正基于梯度的电路定位
Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation
AI总结：
针对现有电路定位方法忽略组件交互导致重要性误估的问题，提出GIM技术，通过在反向传播中显式建模特征交互，在机械可解释性基准的电路定位任务上达到最优性能。
摘要：
电路定位方法旨在识别大型语言模型中负责特定行为的模型组件子集，从而实现详细的机制分析。现有方法大多假设组件独立运作，并通过孤立地扰动每个组件来估计重要性。然而，神经网络中的组件之间存在交互，忽略这些交互会导致对组件重要性的系统性误估。我们发现一个特别有问题的交互是注意力自修复，其中softmax重新分配导致有影响力的注意力分数的梯度随着其他具有相似值的位置的补偿而消失。我们引入了梯度交互修改（GIM），这是一种在反向传播过程中显式考虑特征交互的技术。GIM在机械可解释性基准的电路定位任务上达到了最先进的性能，并在多种任务的特征归因上优于现有的基于梯度的方法。通过考虑交互效应并解释为何先前方法低估了组件重要性，GIM使得对大型语言模型进行更忠实的机制分析成为可能。GIM作为Python包可在https://github.com/corticph/gim获取。
英文摘要：
Circuit localization methods aim to identify the subset of model components responsible for specific behaviors in large language models, enabling detailed mechanistic analysis. Most existing methods assume components act independently and estimate importance by perturbing each component in isolation. However, components in neural networks interact, and ignoring these interactions leads to systematic misestimation of component importance. We find that one particularly problematic interaction is attention self-repair, in which softmax redistribution causes gradients for influential attention scores to vanish as other positions with similar values compensate. We introduce Gradient Interaction Modifications (GIM), a technique that explicitly accounts for feature interactions during backpropagation. GIM achieves state-of-the-art performance on the circuit localization track of the Mechanistic Interpretability Benchmark and outperforms existing gradient-based methods on feature attribution across diverse tasks. By accounting for interaction effects and explaining why prior methods underestimate component importance, GIM enables more faithful mechanistic analysis of large language models. GIM is available as a Python package at https://github.com/corticph/gim.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
