arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.00610cs.CL
收藏

RQ-RAG:学习为检索增强生成优化查询

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

Hong Kong University of Science and Technology(香港科技大学);Hong Kong Polytechnic University(香港理工大学);Massachusetts Institute of Technology(麻省理工学院)

Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, Jie Fu

AI总结:

本文提出RQ-RAG方法,通过让模型学习对查询进行重写、分解和消歧,提升检索增强生成在模糊或复杂问题上的表现,并在多个问答数据集上超越已有最优方法。

摘要:

大型语言模型(LLMs)展现出卓越的能力,但容易产生不准确或幻觉性的回答。这一局限源于它们依赖庞大的预训练数据集,使其在未见场景中容易出错。为应对这些挑战,检索增强生成(RAG)通过将外部相关文档纳入回答生成过程来解决该问题,从而在利用LLMs的上下文学习能力的同时使用非参数化知识。然而,现有的RAG实现主要关注用于上下文检索的初始输入,忽视了模糊或复杂查询中需要进一步澄清或分解才能准确回答的细微之处。为此,本文提出学习为检索增强生成优化查询(RQ-RAG),致力于通过赋予模型显式重写、分解和消歧的能力来增强模型。实验结果表明,当将我们的方法应用于7B Llama2模型时,在三个单跳问答数据集上平均超越先前最优方法(SOTA)1.9%,并且在处理复杂的多跳问答数据集时也表现出更强的性能。代码可在 https://github.com/chanchimin/RQ-RAG 获取。

英文摘要:

Large Language Models (LLMs) exhibit remarkable capabilities but are prone to generating inaccurate or hallucinatory responses. This limitation stems from their reliance on vast pretraining datasets, making them susceptible to errors in unseen scenarios. To tackle these challenges, Retrieval-Augmented Generation (RAG) addresses this by incorporating external, relevant documents into the response generation process, thus leveraging non-parametric knowledge alongside LLMs' in-context learning abilities. However, existing RAG implementations primarily focus on initial input for context retrieval, overlooking the nuances of ambiguous or complex queries that necessitate further clarification or decomposition for accurate responses. To this end, we propose learning to Refine Query for Retrieval Augmented Generation (RQ-RAG) in this paper, endeavoring to enhance the model by equipping it with capabilities for explicit rewriting, decomposition, and disambiguation. Our experimental results indicate that our method, when applied to a 7B Llama2 model, surpasses the previous state-of-the-art (SOTA) by an average of 1.9\% across three single-hop QA datasets, and also demonstrates enhanced performance in handling complex, multi-hop QA datasets. Our code is available at https://github.com/chanchimin/RQ-RAG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑