鲁棒答案，脆弱逻辑：在LLM推理中探究解耦假说
Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoning
AI总结：
研究揭示LLM在面对微小输入扰动时，仍能保持正确答案但推理过程不稳定，提出MATCHA框架以探测解耦假说，发现多步骤和常识任务更易受此影响。
摘要：
尽管链式思维（CoT）提示法已成为大型语言模型（LLM）复杂推理的基石，但生成推理的准确性仍是一个开放性问题。我们探讨了解耦假说：正确答案往往掩盖了脆弱的、事后理性的解释，这些解释与模型的预测没有因果联系。为了系统验证这一点，我们引入了MATCHA，一种新颖的答案条件探测框架。不同于传统评估只关注最终输出的准确性，MATCHA通过将生成过程条件化于模型预测的答案，从而隔离推理阶段，允许我们对推理本身进行压力测试。我们的实验揭示了一个关键的脆弱性：在不可察觉的输入扰动下，LLM经常保持正确答案，同时生成不一致或无意义的推理——即“为错误原因正确”。利用LLM法官量化这种鲁棒性差距，我们发现多步骤和常识任务比逻辑任务更容易受到这种解耦的影响。此外，我们证明了由MATCHA生成的对抗性示例能够非平凡地转移到黑盒模型中。我们的发现揭示了CoT鲁棒性的幻象，并强调了未来架构需要强制真正的答案-推理一致性，而不是仅仅表面的准确性。
英文摘要：
While Chain-of-Thought (CoT) prompting has become a cornerstone for complex reasoning in Large Language Models (LLMs), the faithfulness of the generated reasoning remains an open question. We investigate the Decoupling Hypothesis: that correct answers often mask fragile, post-hoc rationalizations that are not causally tied to the model's prediction. To systematically verify this, we introduce MATCHA, a novel Answer-Conditioned Probing framework. Unlike standard evaluations that focus on final output accuracy, MATCHA isolates the reasoning phase by conditioning generation on the model's predicted answer, allowing us to stress-test the stability of the rationale itself. Our experiments reveal a critical vulnerability: under imperceptible input perturbations, LLMs frequently maintain the correct answer while generating inconsistent or nonsensical reasoning - effectively being ``Right for the Wrong Reasons''. Using LLM judges to quantify this robustness gap, we find that multi-step and commonsense tasks are significantly more susceptible to this decoupling than logical tasks. Furthermore, we demonstrate that adversarial examples generated by MATCHA transfer non-trivially to black-box models. Our findings expose the illusion of CoT robustness and underscore the need for future architectures that enforce genuine answer-reasoning consistency rather than mere surface-level accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
