何时继续思考:面向高效推理的自适应思考模式切换
When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning
AI总结:
本文提出自适应自我恢复推理(ASRR)框架,通过抑制冗余推理并利用内部自我恢复机制,根据问题难度动态分配推理预算,在多个基准上显著降低推理开销(最高32.5%)且几乎不损失准确率,同时提升安全性。
摘要:
大型推理模型(LRMs)通过长推理链取得了显著性能,但往往因冗余推理而产生过高的计算开销,尤其是在简单任务上。在本工作中,我们系统地量化了大型推理模型在“长思考”和“无思考”两种模式下的性能上限,并揭示了“内部自我恢复机制”现象,即模型在答案生成过程中会隐式地补充推理。基于这一发现,我们提出了自适应自我恢复推理(ASRR)框架,该框架抑制不必要的推理并实现隐式恢复。通过引入准确率感知的长度奖励调节,ASRR根据问题难度自适应地分配推理努力,在性能损失可忽略的情况下实现了高效率。在多个基准和模型上的实验表明,与GRPO相比,ASRR将推理预算降低了高达32.5%(1.5B模型)和25.7%(7B模型),同时准确率损失极小(pass@1分别下降1.2%和0.6%),并在安全基准上将无害率显著提升了高达21.7%。我们的结果凸显了ASRR在实现大型推理模型高效、自适应和更安全推理方面的潜力。
英文摘要:
Large reasoning models (LRMs) achieve remarkable performance via long reasoning chains, but often incur excessive computational overhead due to redundant reasoning, especially on simple tasks. In this work, we systematically quantify the upper bounds of LRMs under both Long-Thinking and No-Thinking modes, and uncover the phenomenon of "Internal Self-Recovery Mechanism" where models implicitly supplement reasoning during answer generation. Building on this insight, we propose Adaptive Self-Recovery Reasoning (ASRR), a framework that suppresses unnecessary reasoning and enables implicit recovery. By introducing accuracy-aware length reward regulation, ASRR adaptively allocates reasoning effort according to problem difficulty, achieving high efficiency with negligible performance sacrifice. Experiments across multiple benchmarks and models show that, compared with GRPO, ASRR reduces reasoning budget by up to 32.5% (1.5B) and 25.7% (7B) with minimal accuracy loss (1.2% and 0.6% pass@1), and significantly boosts harmless rates on safety benchmarks (up to +21.7%). Our results highlight the potential of ASRR for enabling efficient, adaptive, and safer reasoning in LRMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。