聚焦你的问题！常识推理中 Toxic CoT 问题的解释与缓解
Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning
AI总结：
针对 CoT 导致原本正确答案变错的 Toxic CoT 问题，通过追踪发现浅层注意力层存在信息丢失，提出 RIDERS 方法补偿信息缺失，使该问题降低 23.6% 并提升整体推理性能 5.5%。
摘要：
大型语言模型展现出高水平的常识推理能力，尤其是在 Chain-of-Thought (CoT) 等增强方法的辅助下。然而，我们发现这些类似 CoT 的方法导致大量原本正确的答案变为错误，我们将此定义为 Toxic CoT 问题。为了解释并缓解这一问题，我们首先利用归因追踪和因果追踪方法探测 LLM 在 CoT 推理期间的内部工作机制。通过比较，我们证明模型在生成理由或答案时，浅层注意力层会出现来自问题的信息丢失。基于探测发现，我们设计了一种名为 RIDERS (Residual decodIng and sERial-position Swap) 的新方法，从解码和序列位置两个角度补偿模型中的信息缺失。在多个常识推理基准上的大量实验验证了该方法不仅显著消除了 Toxic CoT 问题（降低了 23.6%），还有效提升了模型整体的常识推理性能（提升了 5.5%）。
英文摘要：
Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT). However, we find these CoT-like methods lead to a considerable number of originally correct answers turning wrong, which we define as the Toxic CoT problem. To interpret and mitigate this problem, we first utilize attribution tracing and causal tracing methods to probe the internal working mechanism of the LLM during CoT reasoning. Through comparisons, we prove that the model exhibits information loss from the question over the shallow attention layers when generating rationales or answers. Based on the probing findings, we design a novel method called RIDERS (Residual decodIng and sERial-position Swap), which compensates for the information deficit in the model from both decoding and serial-position perspectives. Through extensive experiments on multiple commonsense reasoning benchmarks, we validate that this method not only significantly eliminates Toxic CoT problems (decreased by 23.6%), but also effectively improves the model's overall commonsense reasoning performance (increased by 5.5%).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
