链式推理中的隐藏错误意识：信号是诊断性的，而非因果性的
Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal
AI总结：
研究揭示链式推理中模型内在检测自身推理错误但外在表现自信的现象，通过隐藏状态线性探针显示其正确性预测能力，但无法通过干预修复错误，表明信号为诊断而非因果。
摘要：
链式推理（CoT）提示假设生成的推理反映模型内部计算。本文显示这一假设在特定可测量方式下不成立：模型内部检测自身推理错误但外在表达自信。对隐藏状态的线性探针在生成推理步骤中预测正确性达到0.95 AUROC（从第一步0.79），而错误轨迹的言语自信度为4.55/5，几乎与正确轨迹（4.87/5）相同。文本表面分类器在相同数据上仅达到0.59，确认了0.20点的差距在生成文本中不可见。此隐藏错误意识在三种模型家族（Qwen、Llama、Phi）、1.5B-72B参数及RL训练的推理模型（DeepSeek-R1）中均成立，AUROC为0.852。自然问题是该信号能否修复其检测的错误。四类干预（激活引导、探针引导的最佳N、自我修正、激活修补）均失败；修补破坏输出连贯性。信号是诊断性的而非因果性的：计算质量的读数，而非引导其改变的杠杆。这界定了机械可解释性的边界：推理过程中的错误表示与先前工作成功编辑的事实知识表示本质上不同。
英文摘要：
Chain-of-thought (CoT) prompting assumes that generated reasoning reflects a model's internal computation. We show this assumption is wrong in a specific, measurable way: models internally detect their own reasoning errors but outwardly express confidence in them. A linear probe on hidden states predicts trace correctness with 0.95 AUROC -- from the very first reasoning step (0.79) -- while verbalized confidence for wrong traces is 4.55/5, nearly identical to correct ones (4.87/5). A text-surface classifier achieves only 0.59 on the same data, confirming a 0.20-point gap invisible in the generated text. This hidden error awareness holds across three model families (Qwen, Llama, Phi), 1.5B-72B parameters, and RL-trained reasoning models (DeepSeek-R1, 0.852 AUROC). The natural question is whether this signal can fix the errors it detects. It cannot. Four interventions -- activation steering, probe-guided best-of-N, self-correction, and activation patching -- all fail; patching destroys output coherence entirely. The signal is diagnostic, not causal: a readout of computation quality, not a lever to redirect it. This delineates a boundary for mechanistic interpretability: error representations during reasoning are fundamentally different from the factual knowledge representations that prior work has successfully edited.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
