arXiv 2412.128812024-12-17cs.CLcs.AI

RAG-Star：利用检索增强验证与细化提升深思熟虑的推理能力

RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement

Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Wuhan University of Science and Technology（武汉科技大学）；National University of Singapore（新加坡国立大学）；Northeastern University at Qinhuangdao（东北大学秦皇岛分校）；BOSS Zhipin（BOSS直聘）

Jinhao Jiang, Jiayi Chen, Junyi Li, Ruiyang Ren, Shijie Wang, Wayne Xin Zhao, Yang Song, Tao Zhang

备注：: LLM;RAG;MCTS

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对复杂推理任务，提出RAG-Star方法，结合检索增强验证与蒙特卡洛树搜索，利用外部知识引导内部推理，实验证明其优于现有方法。

摘要：

现有的大语言模型（LLMs）展现出卓越的问题解决能力，但在处理复杂推理任务时可能面临困难。尽管思维链和基于树的搜索方法取得了成功，但它们主要依赖LLMs的内部知识来搜索中间推理步骤，仅限于处理涉及较少推理步骤的简单任务。在本文中，我们提出了RAG-Star，一种新颖的RAG方法，它整合检索到的信息来引导基于树的深思熟虑推理过程，该过程依赖于LLMs固有的知识。通过利用蒙特卡洛树搜索，RAG-Star基于LLM自身迭代地规划中间子查询和推理答案。为了整合内部和外部知识，我们提出了一种检索增强验证方法，利用查询感知和答案感知的奖励建模为LLMs的固有推理提供反馈。我们涉及Llama-3.1-8B-Instruct和GPT-4o的实验表明，RAG-Star显著优于之前的RAG和推理方法。

英文摘要：

Existing large language models (LLMs) show exceptional problem-solving capabilities but might struggle with complex reasoning tasks. Despite the successes of chain-of-thought and tree-based search methods, they mainly depend on the internal knowledge of LLMs to search over intermediate reasoning steps, limited to dealing with simple tasks involving fewer reasoning steps. In this paper, we propose \textbf{RAG-Star}, a novel RAG approach that integrates the retrieved information to guide the tree-based deliberative reasoning process that relies on the inherent knowledge of LLMs. By leveraging Monte Carlo Tree Search, RAG-Star iteratively plans intermediate sub-queries and answers for reasoning based on the LLM itself. To consolidate internal and external knowledge, we propose an retrieval-augmented verification that utilizes query- and answer-aware reward modeling to provide feedback for the inherent reasoning of LLMs. Our experiments involving Llama-3.1-8B-Instruct and GPT-4o demonstrate that RAG-Star significantly outperforms previous RAG and reasoning methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。