arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03094cs.CL
收藏

测试时递归思考:无需外部反馈的自我提升

Test-time Recursive Thinking: Self-Improvement without External Feedback

Microsoft Research(微软研究院)

Yufan Zhuang, Chandan Singh, Liyuan Liu, Yelong Shen, Dinghuai Zhang, Jingbo Shang, Jianfeng Gao, Weizhu Chen

AI总结:

本文提出 TRT 框架,使 LLM 在无需外部反馈的情况下实现自我提升,开源模型在 AIME-25/24 上达到 100% 准确率,闭源模型在 LiveCodeBench 最难问题上提升 10.4-14.8 个百分点。

摘要:

现代大型语言模型(LLMs)在推理能力方面显示出快速的进步,主要得益于强化学习(RL)与可验证奖励。本文探讨LLMs是否能在不需额外训练的情况下实现自我提升。我们识别出两种核心挑战:(i)高效生成多样且高质量的候选解决方案;(ii)在没有地面真相监督的情况下可靠地选择正确答案。为解决这些挑战,我们提出了测试时递归思考(TRT),一种迭代自我提升框架,其生成过程会根据特定的 rollout 策略、积累的知识以及自动生成的验证信号进行条件生成。使用 TRT,开源模型在 AIME-25/24 上达到 100% 准确率,而在 LiveCodeBench 最困难的问题上,闭源模型在无外部反馈的情况下提升了 10.4-14.8 个百分点。

英文摘要:

Modern Large Language Models (LLMs) have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whether these LLMs can self-improve without the need for additional training. We identify two core challenges for such systems: (i) efficiently generating diverse, high-quality candidate solutions, and (ii) reliably selecting correct answers in the absence of ground-truth supervision. To address these challenges, we propose Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals. Using TRT, open-source models reach 100% accuracy on AIME-25/24, and on LiveCodeBench's most difficult problems, closed-source models improve by 10.4-14.8 percentage points without external feedback.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑