链式思维推理的局部因果归因
Local Causal Attribution of Chain-of-Thought Reasoning
AI总结：
提出AttriCoT算法，通过结构因果模型对链式思维推理中的单元进行局部因果归因，仅需O(U)次前向传播即可获得忠实于模型行为的归因结果。
摘要：
理解语言模型思维过程的因果结构对于透明度和安全性都至关重要。在这项工作中，我们采取局部方法，通过分析给定特定链式思维轨迹中各个组件（称为单元）之间的因果关系来实现这一目标。我们在这些单元上构建结构因果模型，并将每个单元与生成（后续）输出单元的对数概率联系起来。我们的算法称为AttriCoT，是一种黑盒方法，通过使用$O(U)$次前向传播估计结构因果模型中的重要性参数来执行归因，其中$U$是单元数量。在5个数据集和4个推理模型上的扰动曲线评估表明，AttriCoT产生的归因比替代方法更忠实于模型的行为。归因结果还揭示了模型和领域之间思维结构的显著差异。
英文摘要：
Understanding the causal structure of a language model's thought process is a problem of significant importance for both transparency and safety. In this work, we take a local approach toward this goal by analyzing the causal relationships among individual components, termed units, of a given, specific chain-of-thought trace. We construct a structural causal model on these units and relate each unit to the log probability of generating (subsequent) output units. Our algorithm, termed AttriCoT, is a black-box method that performs attribution by estimating importance parameters in the structural causal model using $O(U)$ forward passes through the model, where $U$ is the number of units. Evaluation of perturbation curves across 5 datasets and 4 reasoning models shows that AttriCoT produces attributions that are more faithful to the model's behavior than alternative methods. The attribution results also reveal notable differences in thought structure between models and domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
