arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.13626cs.CLcs.AI
收藏

思维操控:外部思维对大型推理模型可以是高效的

Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models

Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Ant Group(蚂蚁集团)

Yule Liu, Jingyi Zheng, Zhen Sun, Zifan Peng, Wenhan Dong, Zeyang Sha, Shiwen Cui, Weiqiang Wang, Xinlei He

AI总结:

本文发现将较小模型生成的外部思维链插入大型推理模型的思考标记间可减少其过度思考,提出Method流水线,在保持性能下显著降低计算成本。

摘要:

大型推理模型(LRMs)的最新进展已证明,在各类任务中扩展测试时计算能有效增强推理能力。然而,LRMs 常常遭受“过度思考”问题,即模型生成过多冗余的推理步骤,而性能提升有限。在本工作中,我们通过实验揭示了一个 LRM 行为的重要特征:将较小模型生成的外部思维链(CoTs)放置在思考标记()之间,可以有效操控模型生成更少的思考内容。基于这一发现,我们提出了一种简单而高效的流水线方法 \u201cMethod\u201d,使 LRMs 能够绕过不必要的中间步骤,从而显著降低计算成本。我们进行了大量实验来评估 \u201cMethod\u201d 的实用性和效率。例如,当应用于 QwQ-32B 模型在 LiveBench/Code 数据集上时,\u201cMethod\u201d 在保持原有性能的同时,将输出令牌数量减少了约 30%,且思维链生成器引入的开销极小。此外,我们识别出两种次优模式:盲目遵循有缺陷的外部思维和不必要的重新思考,并表明简单的缓解措施(如难度感知回退)可以进一步提升性能。总体而言,\u201cMethod\u201d 为优化 LRM 推理提供了一种实用、通用且高效的方式,使强大的推理模型在实际应用中更易获取和可扩展。

英文摘要:

Recent advancements in large reasoning models (LRMs) have demonstrated the effectiveness of scaling test-time computation to enhance reasoning capabilities on various tasks. However, LRMs often suffer from an ``overthinking'' problem, where the model generates excessively redundant reasoning steps with limited performance gains. In this work, we empirically reveal an important characteristic of LRM behaviors that placing external CoTs generated by smaller models between the thinking token (\texttt{<think>} and \texttt{</think>}) can effectively manipulate the model to generate fewer thoughts. Building on this finding, we propose a simple yet efficient pipeline, \Method, to enable LRMs to bypass unnecessary intermediate steps, thereby significantly reducing computational costs. We conduct extensive experiments to evaluate the utility and efficiency of \Method. For instance, when applied to QwQ-32B on the LiveBench/Code dataset, \Method keeps the original performance while reducing output token counts by approximately 30\%, with minimal overhead introduced by the CoT generator. Furthermore, we identify two suboptimal modes, blindly following flawed external thoughts and unnecessary rethinking, and show that simple mitigations, such as difficulty-aware fallbacks, can further improve performance. Overall, \Method offers a practical, general, and efficient way to optimize LRM inference, making powerful reasoning models more accessible and scalable for real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑