arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.01348cs.CL
收藏

大规模程序知识提升推理能力

Procedural Knowledge at Scale Improves Reasoning

University of California, Los Angeles(加州大学洛杉矶分校);Meta FAIR

Di Wu, Devendra Singh Sachan, Wen-tau Yih, Mingda Chen

AI总结:

本研究提出Reasoning Memory框架,通过大规模检索增强生成方法,有效利用程序知识提升推理能力,在多个基准测试中表现优于现有方法。

摘要:

测试时扩展已成为提升语言模型在挑战性推理任务上的有效方法。然而,现有方法大多孤立处理每个问题,未能系统性地重用先前推理轨迹的知识。特别是,它们未能充分利用程序知识:如何重新表述问题、选择方法以及在需要时验证或回溯。我们引入了Reasoning Memory,一种检索增强生成(RAG)框架,用于推理模型,该框架在大规模上显式检索并重用程序知识。从现有的逐步推理轨迹语料库出发,我们将每个轨迹分解为自包含的子问题-子程序对,生成包含3200万紧凑程序知识条目的数据存储。在推理时,一个轻量级的“思考中提示”使模型能够口头阐述核心子问题,检索其推理轨迹中的相关子程序,并在多种检索到的子程序下进行推理,作为隐含的程序先验。在六个数学、科学和编码基准测试中,Reasoning Memory在多个方面均优于RAG方法,包括文档、轨迹和模板知识,以及计算匹配的测试时扩展基线。随着更高的推理预算,它在任务类型上比无检索提升高达19.2%,比最强的计算匹配基线提升7.9%。消融研究显示,这些收益来自两个关键因素:源轨迹的广泛程序覆盖和我们的分解与检索设计,两者共同使有效提取和重用程序知识成为可能。

英文摘要:

Test-time scaling has emerged as an effective way to improve language models on challenging reasoning tasks. However, most existing methods treat each problem in isolation and do not systematically reuse knowledge from prior reasoning trajectories. In particular, they underutilize procedural knowledge: how to reframe a problem, choose an approach, and verify or backtrack when needed. We introduce \textbf{Reasoning Memory}, a retrieval-augmented generation (RAG) framework for reasoning models that explicitly retrieves and reuses procedural knowledge at scale. Starting from existing corpora of step-by-step reasoning trajectories, we decompose each trajectory into self-contained subquestion-subroutine pairs, yielding a datastore of 32 million compact procedural knowledge entries. At inference time, a lightweight in-thought prompt lets the model verbalize the core subquestion, retrieve relevant subroutines within its reasoning trace, and reason under diverse retrieved subroutines as implicit procedural priors. Across six math, science, and coding benchmarks, Reasoning Memory consistently outperforms RAG with document, trajectory, and template knowledge, as well as a compute-matched test-time scaling baseline. With a higher inference budget, averaged across models and tasks, it improves over no retrieval by 19.5% and over the compute-matched baseline by 8.9%. Ablation studies show that these gains come from two key factors: the broad procedural coverage of the source trajectories and our decomposition and retrieval design, which together enable effective extraction and reuse of procedural knowledge. Our experiment code is available at https://github.com/facebookresearch/reasoning-memory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑