arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2504.14858cs.AIcs.CL

检索是不够的:通过测试时批评与优化增强RAG推理

Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization

Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);South China University of Technology(华南理工大学);University of British Columbia(不列颠哥伦比亚大学);Fudan University(复旦大学);University of Hong Kong(香港大学)

Jiaqi Wei, Hao Zhou, Xiang Zhang, Di Zhang, Zijie Qiu, Wei Wei, Jinzhe Li, Wanli Ouyang, Siqi Sun

AI总结:

针对RAG中LLM内部推理与检索证据不一致的推理错位问题,提出基于批评驱动对齐的迭代框架AlignRAG及其自主变体,通过对比批评合成机制训练批评语言模型,显著提升推理保真度与鲁棒性。

摘要:

检索增强生成(RAG)已成为实现知识奠基大语言模型(LLMs)的广泛采用范式。然而,标准RAG流水线通常无法确保模型推理与检索到的证据保持一致,导致事实不一致或无支持的结论。在本研究中,我们将RAG重新解释为检索增强推理,并识别出一个核心但未被充分探索的问题:推理错位——LLM内部推理轨迹与检索提供的证据约束之间的偏离。为解决这一问题,我们提出了AlignRAG,一个基于批评驱动对齐(CDA)的新型迭代框架。我们进一步引入了AlignRAG-auto,这是一个自主变体,可动态终止优化过程,无需预先指定批评迭代次数。AlignRAG的核心在于一个对比批评合成机制,该机制在减轻自我偏见的同时生成对检索敏感的批评。此机制使用标记的批评来训练专门的检索增强批评语言模型(CLM),这些批评能够区分与证据对齐和错位的推理。实证评估表明,我们的方法显著提高了推理保真度。我们的8B参数CLM在域外任务上比Self-Refine基线提高了12.1%的性能,并优于标准的72B参数CLM 2.2%。此外,AlignRAG-auto在动态确定最优优化步骤数的同时实现了这一最先进的性能,增强了效率和可用性。AlignRAG作为即插即用模块与现有RAG架构保持兼容,并在信息丰富和噪声检索场景下均展现出强大的鲁棒性。

英文摘要:

Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs). However, standard RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions. In this work, we reinterpret RAG as Retrieval-Augmented Reasoning and identify a central but underexplored problem: Reasoning Misalignment -- the divergence between an LLM's internal reasoning trajectory and the evidential constraints provided by retrieval. To address this issue, we propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA). We further introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations. At the heart of AlignRAG lies a contrastive critique synthesis mechanism that generates retrieval-sensitive critiques while mitigating self-bias. This mechanism trains a dedicated retrieval-augmented Critic Language Model (CLM) using labeled critiques that distinguish between evidence-aligned and misaligned reasoning. Empirical evaluations show that our approach significantly improves reasoning fidelity. Our 8B-parameter CLM improves performance over the Self-Refine baseline by 12.1% on out-of-domain tasks and outperforms a standard 72B-parameter CLM by 2.2%. Furthermore, AlignRAG-auto achieves this state-of-the-art performance while dynamically determining the optimal number of refinement steps, enhancing efficiency and usability. AlignRAG remains compatible with existing RAG architectures as a plug-and-play module and demonstrates strong robustness under both informative and noisy retrieval scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑