大声思考：推理模型知道自己何时正确吗？
Thinking Out Loud: Do Reasoning Models Know When They're Right?
AI总结：
本文通过言语化置信度研究LRM的自我反思，发现推理蒸馏与强化学习可阶梯式改善校准，但会削弱小规模模型的知识边界意识并降低弃权能力。
摘要：
大型推理模型（LRM）近来通过利用增加的测试时计算，并表现出令人联想到人类式自我反思的行为，在复杂推理任务上展示了令人瞩目的能力。尽管LRM明显具备宝贵的自我反思能力，但这种能力如何与模型的其他行为相互作用，仍未得到充分研究。我们通过分析言语化置信度，即模型如何表达其确定性，来考察这种联系，并将其作为透视LRM中自我反思本质的窗口。我们发现，在推理密集的情境中，对推理轨迹进行监督微调（即蒸馏）以及强化学习，能够以渐进、阶梯式的方式改善言语化校准。然而，我们的结果也表明，推理模型对自身知识边界的意识可能有所减弱，事实性基准上“我不知道”回答率显著降低便证明了这一点。此外，我们考察了言语化置信度与推理链之间的关系，发现模型在给出较短或较不详尽的推理时，往往会表达更高的置信度。我们的发现凸显了面向推理的训练如何在提升以推理为中心的任务性能的同时，可能带来“推理税”：在小规模模型中，这种代价体现为模型准确识别自身知识局限的能力下降。更广泛地说，我们的工作展示了这种知识边界的侵蚀如何损害模型的忠实性，因为模型变得更加自信，却没有相应理解自己何时应当弃权（不执行）。
英文摘要：
Large reasoning models (LRMs) have recently demonstrated impressive capabilities in complex reasoning tasks by leveraging increased test-time computation and exhibiting behaviors reminiscent of human-like self-reflection. While LRMs show a clear capacity for valuable self-reflection, how this ability interacts with other model behaviors remains underexplored. We investigate this connection by analyzing verbalized confidence, how models articulate their certainty, as a lens into the nature of self-reflection in LRMs. We find that supervised fine-tuning on reasoning traces (i.e., distillation) and reinforcement learning can improve verbalized calibration in reasoning-intensive settings in a progressive, laddered fashion. However, our results also indicate that reasoning models may possess a diminished awareness of their own knowledge boundaries, as evidenced by significantly lower "I don't know" response rates on factuality benchmarks. Moreover, we examine the relationship between verbalized confidence and reasoning chains, finding that models tend to express higher confidence when providing shorter or less elaborate reasoning. Our findings highlight how reasoning-oriented training can enhance performance in reasoning-centric tasks while potentially incurring a "reasoning tax," a cost reflected in the model's reduced ability to accurately recognize the limits of its own knowledge in small-scale models. More broadly, our work showcases how this erosion of knowledge boundaries can compromise model faithfulness, as models grow more confident without a commensurate understanding of when they should abstain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
