退一步以跃进：通过自回溯提升语言模型的推理能力
Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models
AI总结：
本文提出自回溯机制，使大语言模型在训练和推理中自主决定何时何地回溯，将慢思考转化为快思考，从而提升推理能力与效率，实验显示相比最优路径监督微调方法性能提升超过40%。
摘要：
将慢思考机制集成到大语言模型（LLMs）中，为实现第2级AGI推理器提供了一条有前景的途径，OpenAI的o1等系统就是例证。然而，仍存在若干重大挑战，包括低效的过度思考以及对辅助奖励模型的过度依赖。我们指出，这些局限源于LLMs无法内化搜索过程，而搜索过程是有效推理的关键组成部分。解决这一问题的关键一步是使LLMs能够自主决定何时何地进行回溯，这是传统搜索算法中的一项基本操作。为此，我们提出了一种自回溯机制，使LLMs在训练和推理过程中都具备回溯能力。该机制不仅通过自我改进将慢思考过程转化为快思考，从而提升推理效率，还增强了推理能力。实证评估表明，我们的方法显著增强了LLMs的推理能力，与最优路径监督微调方法相比，性能提升超过40%。我们相信，本研究为开发更先进、更稳健的推理器引入了一条新颖且有前景的路径。
英文摘要：
The integration of slow-thinking mechanisms into large language models (LLMs) offers a promising way toward achieving Level 2 AGI Reasoners, as exemplified by systems like OpenAI's o1. However, several significant challenges remain, including inefficient overthinking and an overreliance on auxiliary reward models. We point out that these limitations stem from LLMs' inability to internalize the search process, a key component of effective reasoning. A critical step toward addressing this issue is enabling LLMs to autonomously determine when and where to backtrack, a fundamental operation in traditional search algorithms. To this end, we propose a self-backtracking mechanism that equips LLMs with the ability to backtrack during both training and inference. This mechanism not only enhances reasoning ability but also efficiency by transforming slow-thinking processes into fast-thinking through self-improvement. Empirical evaluations demonstrate that our proposal significantly enhances the reasoning capabilities of LLMs, achieving a performance gain of over 40 percent compared to the optimal-path supervised fine-tuning method. We believe this study introduces a novel and promising pathway for developing more advanced and robust Reasoners.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
