arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08354cs.AI
收藏

你的推理模型是否在隐式地知道何时停止思考?

Does Your Reasoning Model Implicitly Know When to Stop Thinking?

Beihang University(北京航空航天大学)

Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, Xuanda Wang, Zhixia Zhang, Hongyan Xie, Songshi Liang, Zehao Chen, Xuefeng Xiao, Fuzhen Zhuang, Jianxin Li, Deqing Wang, Yikun Ban

AI总结:

本文研究了大推理模型在复杂推理任务中停止思考的隐式能力,提出SAGE方法提升推理效率与准确性。

摘要:

近年来,大型推理模型(LRMs)通过长链思考(CoTs)在复杂推理任务中取得了显著进展。然而,这种方法常导致冗余,影响计算效率并造成实时应用中的延迟。近期研究表明,更长的推理链与正确性无关,甚至可能损害准确性。深入分析后发现,LRMs隐式地知道何时停止思考,但这一能力被当前采样范式所掩盖。为此,我们引入SAGE(Self-Aware Guided Efficient Reasoning)新采样范式,释放其高效推理潜力。进一步将SAGE作为混合采样整合到基于群体的强化学习(SAGE-RL)中,使SAGE-RL能有效将SAGE发现的高效推理模式融入标准pass@1推理,显著提升LRMs在多个挑战性数学基准上的推理准确性和效率。

英文摘要:

Recent advancements in large reasoning models (LRMs) have greatly improved their capabilities on complex reasoning tasks through Long Chains of Thought (CoTs). However, this approach often results in substantial redundancy, impairing computational efficiency and causing significant delays in real-time applications. Recent studies show that longer reasoning chains are frequently uncorrelated with correctness and can even be detrimental to accuracy. In a further in-depth analysis of this phenomenon, we surprisingly uncover and empirically verify that LRMs implicitly know the appropriate time to stop thinking, while this capability is obscured by current sampling paradigms. Motivated by this, we introduce SAGE (Self-Aware Guided Efficient Reasoning), a novel sampling paradigm that unleashes this efficient reasoning potential. Furthermore, integrating SAGE as mixed sampling into group-based reinforcement learning (SAGE-RL) enables SAGE-RL to effectively incorporate SAGE-discovered efficient reasoning patterns into standard pass@1 inference, markedly enhancing both the reasoning accuracy and efficiency of LRMs across multiple challenging mathematical benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑