arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.02112cs.LGcs.CL
收藏

长程推理的递归模型

Recursive Models for Long-Horizon Reasoning

Toyota Technological Institute at Chicago(丰田技术研究所)

Chenxiao Yang, Nathan Srebro, Zhiyuan Li

AI总结:

提出递归模型,通过递归调用自身在隔离上下文中解决子任务,突破上下文长度限制,理论证明可指数级降低活跃上下文需求,实验验证在SAT求解和围棋任务中提升长程准确性。

摘要:

现代语言模型在有界上下文中推理,这一固有约束对长程推理构成了根本障碍。我们识别出递归是克服这一障碍的核心原则,并提出递归模型作为最小实现,其中模型可以递归地调用自身以在隔离上下文中解决子任务。我们证明任何可计算问题都允许推理的递归分解,其中每个子任务所需的活跃上下文比标准自回归模型指数级小;这严格超越了任何局限于单一序列的上下文管理方法,例如摘要。我们进一步将框架推广到具有任意上下文处理和控制流的现代智能体系统,并证明递归模型可以在这一更广泛的类别中实现最优能力。实验上,我们测试了两种设置:微调预训练基础模型以进行递归SAT求解,以及从零开始训练小模型处理由精确博弈树搜索生成的围棋轨迹。两者都表明在小的活跃上下文下长程准确性得到提升。

英文摘要:

Modern language models reason within bounded context, an inherent constraint that poses a fundamental barrier to long-horizon reasoning. We identify recursion as a core principle for overcoming this barrier, and propose recursive models as a minimal realization, where the model can recursively invoke itself to solve subtasks in isolated contexts. We prove that any computable problem admits a recursive decomposition of reasoning in which each subtask requires only exponentially smaller active context than standard autoregressive models; this strictly surpasses any context management approach confined to a single sequence, such as summarization. We further generalize our framework to modern agentic systems with arbitrary context processing and control flows, and prove that recursive models can achieve optimal power within this broader class. Experimentally, we test two settings: fine-tuning a pretrained base model for recursive SAT solving, and training a small model from scratch on Go traces generated by exact game-tree search. Both show improved long-horizon accuracy with small active contexts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑