现实中的思维链推理并不总是忠实的
Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
AI总结：
研究发现，在自然语言提示下，模型有时会生成表面连贯但自相矛盾的思维链，揭示出隐含的事后合理化现象，且前沿模型也未能完全避免。
摘要：
最近的研究表明，当面对提示中的显式偏见时，模型通常会在其思维链（CoT）输出中省略提及这些偏见，揭示出口头推理可能给出模型如何得出错误结论的不正确图景（不忠实）。在这项工作中，我们展示了不忠实的CoT也发生在自然措辞、非对抗性的提示上，而无需添加人为偏见或编辑模型输出。我们发现，当分别呈现问题“X比Y大吗？”和“Y比X大吗？”时，模型有时会生成表面连贯的论证来证明系统性地对两者都回答“是”或都回答“否”是合理的，尽管存在矛盾。我们提供了初步证据表明这是由于模型对“是”或“否”的隐含偏见，并将其标记为隐含的事后合理化。我们的结果显示，生产模型的不忠实率高达13%，而前沿模型虽然更忠实，但没有一个完全忠实，包括像DeepSeek R1（0.37%）和Sonnet 3.7 with thinking（0.04%）这样的思考模型。我们还研究了不忠实的非逻辑捷径，即模型使用微妙的非逻辑推理来使对困难数学问题的推测性答案看起来经过严格证明。我们的发现表明，虽然CoT可用于评估输出，但它并不是产生模型答案的内部过程的完整描述，应在代理或安全关键环境中谨慎使用。
英文摘要：
Recent studies indicate that when faced with explicit biases in prompts, models often omit mentioning these biases in their Chain-of-Thought (CoT) output, revealing that verbalized reasoning can give an incorrect picture of how models arrive at conclusions (unfaithfulness). In this work, we show that unfaithful CoT also occurs on naturally worded, non-adversarial prompts without adding artificial biases or editing model outputs. We find that when separately presented with the questions "Is X bigger than Y?" and "Is Y bigger than X?", models sometimes produce superficially coherent arguments to justify systematically answering Yes to both or No to both, despite the contradiction. We present preliminary evidence that this is due to models' implicit biases towards Yes or No, labeling this Implicit Post-Hoc Rationalization. Our results reveal rates up to 13% for production models, and while frontier models are more faithful, none are entirely so, including thinking models like DeepSeek R1 (0.37%) and Sonnet 3.7 with thinking (0.04%). We also investigate Unfaithful Illogical Shortcuts, where models use subtly illogical reasoning to make speculative answers to hard math problems seem rigorously proven. Our findings indicate that while CoT can be useful for assessing outputs, it is not a complete account of the internal process that produced the model's answer and should be used with caution in agentic or safety-critical settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
