深度自进化推理
Deep Self-Evolving Reasoning
AI总结：
本文提出DSER概率框架，将迭代推理建模为马尔可夫链并并行放大型改进趋势，使8B开放权重模型在AIME上解决多道难题并超过600B教师单轮准确率。
摘要：
长思维链推理已成为大型语言模型高级推理的基石。尽管近期的验证—精炼框架已使专有模型能够解决奥赛级问题，但其有效性依赖于强大且可靠的验证与纠错能力，而这些能力在开放权重、较小规模模型中仍然脆弱。本研究表明，即使在困难任务上只有较弱的验证和精炼能力，这类模型的推理极限也可以通过一种我们称为深度自进化推理（Deep Self-Evolving Reasoning，DSER）的概率范式得到显著拓展。我们将迭代推理概念化为一条马尔可夫链，其中每一步代表解空间中的一次随机转移。关键洞见在于，只要改进的概率略微超过退化的概率，就可保证收敛到正确解。通过并行运行多个长程自进化过程，DSER放大这些微小的正向趋势，使模型能够渐近逼近正确答案。在实验上，我们将DSER应用于DeepSeek-R1-0528-Qwen3-8B模型。在具有挑战性的AIME 2024-2025基准上，DSER解决了9道此前无法解决的问题中的5道，并提升了整体性能，使这一小规模模型通过多数投票超过其6000亿参数教师模型的单轮准确率。除了对测试时扩展的直接效用外，DSER框架还可用于诊断当前开放权重推理模型的根本局限。通过清晰刻画它们在自验证、精炼和稳定性方面的不足，我们的发现为开发具备强大、内在自进化能力的下一代模型确立了明确的研究议程。
英文摘要：
Long-form chain-of-thought reasoning has become a cornerstone of advanced reasoning in large language models. While recent verification-refinement frameworks have enabled proprietary models to solve Olympiad-level problems, their effectiveness hinges on strong, reliable verification and correction capabilities, which remain fragile in open-weight, smaller-scale models. This work demonstrates that even with weak verification and refinement capabilities on hard tasks, the reasoning limits of such models can be substantially extended through a probabilistic paradigm we call Deep Self-Evolving Reasoning (DSER). We conceptualize iterative reasoning as a Markov chain, where each step represents a stochastic transition in the solution space. The key insight is that convergence to a correct solution is guaranteed as long as the probability of improvement marginally exceeds that of degradation. By running multiple long-horizon, self-evolving processes in parallel, DSER amplifies these small positive tendencies, enabling the model to asymptotically approach correct answers. Empirically, we apply DSER to the DeepSeek-R1-0528-Qwen3-8B model. On the challenging AIME 2024-2025 benchmark, DSER solves 5 out of 9 previously unsolvable problems and boosts overall performance, enabling this compact model to surpass the single-turn accuracy of its 600B-parameter teacher through majority voting. Beyond its immediate utility for test-time scaling, the DSER framework serves to diagnose the fundamental limitations of current open-weight reasoners. By clearly delineating their shortcomings in self-verification, refinement, and stability, our findings establish a clear research agenda for developing next-generation models with powerful, intrinsic self-evolving capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
