测量思维链推理中的忠实性
Measuring Faithfulness in Chain-of-Thought Reasoning
AI总结：
本文研究了思维链（CoT）推理的忠实性，通过干预CoT考察模型预测变化，发现模型对CoT的依赖因任务而异，且模型越大推理越不忠实，但谨慎选择条件可实现忠实推理。
摘要：
大型语言模型（LLMs）在回答问题前，通过逐步生成“思维链”（CoT）推理会表现得更好，但目前尚不清楚所陈述的推理是否是对模型实际推理（即其回答问题的过程）的忠实解释。我们通过在CoT上进行干预（例如添加错误或改写）来考察模型预测的变化，从而研究CoT推理可能不忠实的假设。在不同任务中，模型在预测答案时对CoT的依赖程度表现出很大差异，有时严重依赖CoT，有时则主要忽略它。CoT带来的性能提升似乎并非仅源于其增加的测试时计算，也非源于CoT特定措辞所编码的信息。随着模型规模扩大和能力增强，它们在我们研究的大多数任务上产生的推理忠实性反而降低。总体而言，我们的结果表明，如果谨慎选择模型规模和任务等条件，CoT是可以做到忠实的。
英文摘要：
Large language models (LLMs) perform better when they produce step-by-step, "Chain-of-Thought" (CoT) reasoning before answering a question, but it is unclear if the stated reasoning is a faithful explanation of the model's actual reasoning (i.e., its process for answering the question). We investigate hypotheses for how CoT reasoning may be unfaithful, by examining how the model predictions change when we intervene on the CoT (e.g., by adding mistakes or paraphrasing it). Models show large variation across tasks in how strongly they condition on the CoT when predicting their answer, sometimes relying heavily on the CoT and other times primarily ignoring it. CoT's performance boost does not seem to come from CoT's added test-time compute alone or from information encoded via the particular phrasing of the CoT. As models become larger and more capable, they produce less faithful reasoning on most tasks we study. Overall, our results suggest that CoT can be faithful if the circumstances such as the model size and task are carefully chosen.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
