非线性补丁：大型语言模型中的指令向量
Patches of Nonlinearity: Instruction Vectors in Large Language Models
AI总结：
通过因果中介分析发现指令表示在模型中高度局部化，称为指令向量（IVs），并揭示其线性可分性与非线性因果交互并存，提出一种免于线性假设的新方法定位信息处理，发现IVs作为电路选择器。
摘要：
尽管指令调优语言模型最近取得了成功并被广泛使用，但关于模型如何在内部处理指令，我们知之甚少。在这项工作中，我们从机制角度通过研究指令特定表示在后期训练的不同阶段（监督微调（SFT）和直接偏好优化（DPO））是如何构建和利用的，来填补这一空白。通过因果中介分析，我们发现指令表示在模型中相当局部化。这些表示，我们称之为指令向量（IVs），展示了线性可分性与非线性因果交互的奇特并置，广泛质疑了机械可解释性中常见的线性表示假设的范围。为了解开非线性因果交互，我们提出了一种新颖的方法来定位语言模型中的信息处理，该方法不受基于补丁技术中隐含的线性假设的限制。我们发现，以早期层形成的任务表示为条件，后期层中选择不同的信息路径来解决该任务，即IVs充当电路选择器。
英文摘要：
Despite the recent success of instruction-tuned language models and their ubiquitous usage, very little is known of how models process instructions internally. In this work, we address this gap from a mechanistic point of view by investigating how instruction-specific representations are constructed and utilized in different stages of post-training: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Via causal mediation, we identify that instruction representation is fairly localized in models. These representations, which we call Instruction Vectors (IVs), demonstrate a curious juxtaposition of linear separability along with non-linear causal interaction, broadly questioning the scope of the linear representation hypothesis commonplace in mechanistic interpretability. To disentangle the non-linear causal interaction, we propose a novel method to localize information processing in language models that is free from the implicit linear assumptions of patching-based techniques. We find that, conditioned on the task representations formed in the early layers, different information pathways are selected in the later layers to solve that task, i.e., IVs act as circuit selectors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
