权重修补：向LLMs中源级机理定位迈进
Weight Patching: Toward Source-Level Mechanistic Localization in LLMs
AI总结：
本文提出权重修补方法，通过参数空间干预分析配对同架构模型，揭示指令跟随任务中从浅层候选源侧载体到聚合路由模块再到下游执行电路的层次结构，提升模型融合效果。
摘要：
机理可解释性旨在将模型行为定位到因果实现其的内部组件。先前工作在激活空间定位和因果追踪方面有所进展，但激活空间中看似重要的模块可能仅聚合或放大上游信号，而非在自身参数中编码目标能力。为解决这一差距，我们提出权重修补，一种用于源导向分析的参数空间干预方法，在配对的同架构模型中，这些模型在感兴趣的输入下对目标能力的表达强度不同。给定一个基础模型和一个行为专门化的对应模型，权重修补将选定模块的权重从专业化模型替换到基础模型，在固定输入下。我们将其实例化在指令跟随上，并引入以向量锚点行为接口为中心的框架，提供一个共享的内部标准，判断开放生成中任务相关控制状态是否形成或恢复。在此框架下，分析揭示了从浅层候选源侧载体到聚合和路由模块，再到下游执行电路的层次结构。恢复的组件评分也可以指导机制感知的模型融合，提升在评估的专家组合中的选择性融合，并提供额外的外部验证。
英文摘要：
Mechanistic interpretability seeks to localize model behavior to the internal components that causally realize it. Prior work has advanced activation-space localization and causal tracing, but modules that appear important in activation space may merely aggregate or amplify upstream signals rather than encode the target capability in their own parameters. To address this gap, we propose Weight Patching, a parameter-space intervention method for source-oriented analysis in paired same-architecture models that differ in how strongly they express a target capability under the inputs of interest. Given a base model and a behavior-specialized counterpart, Weight Patching replaces selected module weights from the specialized model into the base model under a fixed input. We instantiate the method on instruction following and introduce a framework centered on a vector-anchor behavioral interface that provides a shared internal criterion for whether a task-relevant control state has been formed or recovered in open-ended generation. Under this framework, the analysis reveals a hierarchy from shallow candidate source-side carriers to aggregation and routing modules, and further to downstream execution circuits. The recovered component scores can also guide mechanism-aware model merging, improving selective fusion across the evaluated expert combinations and providing additional external validation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
