arXiv 2410.223532025-02-16cs.IRcs.AIcs.CL

RuleRAG：面向问答的、使用语言模型的规则引导检索增强生成

RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering

National University of Defense Technology（国防科技大学）；International Digital Economy Academy（国际数字经济学院）；University of Oxford（牛津大学）

Zhongwu Chen, Chengjin Xu, Dingmin Wang, Zhen Huang, Yong Dou, Xuhui Jiang, Jian Guo

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对RAG问答缺乏检索与推理引导的问题，提出RuleRAG，通过规则统一指导检索和生成，并构建RuleQA基准，实验显示检索与准确率大幅提升且具有泛化性。

摘要：

检索增强生成（RAG）已在知识密集型问答（QA）中展现出良好潜力。然而，现有方法仅考虑查询本身，既没有为检索器指定检索偏好，也没有告知生成器应如何参考检索到的文档来回答问题，这对QA性能构成了重大挑战。为解决这些问题，我们提出了使用语言模型的规则引导检索增强生成，该方法显式引入用于上下文学习的规则（RuleRAG-ICL），引导检索器沿规则方向回忆相关文档，并统一引导生成器依据相同规则进行归因推理。此外，大多数现有RAG数据集在构建时未考虑规则，而知识图谱（KG）被公认可提供高质量规则。因此，我们基于KG构建了五个具备规则意识的RAG问答基准RuleQA，以强调依据规则进行检索和推理的重要性。在RuleQA上的实验表明，RuleRAG-ICL使Recall@10的检索质量提升了89.2%，Exact Match的答案准确率提升了103.1%，而RuleRAG-FT带来了进一步提升。此外，在四个现有RAG数据集上的实验表明，通过向其提供RuleQA中的规则，RuleRAG同样有效，进一步证明了RuleRAG中规则引导的泛化能力。

英文摘要：

Retrieval-augmented generation (RAG) has shown promising potential in knowledge intensive question answering (QA). However, existing approaches only consider the query itself, neither specifying the retrieval preferences for the retrievers nor informing the generators of how to refer to the retrieved documents for the answers, which poses a significant challenge to the QA performance. To address these issues, we propose Rule-guided Retrieval-Augmented Generation with LMs, which explicitly introduces rules for in-context learning (RuleRAG-ICL) to guide retrievers to recall related documents in the directions of rules and uniformly guide generators to reason attributed by the same rules. Moreover, most existing RAG datasets were constructed without considering rules and Knowledge Graphs (KGs) are recognized as providing high-quality rules. Therefore, we construct five rule-aware RAG benchmarks for QA, RuleQA, based on KGs to stress the significance of retrieval and reasoning with rules. Experiments on RuleQA demonstrate RuleRAG-ICL improves the retrieval quality of +89.2% in Recall@10 and answer accuracy of +103.1% in Exact Match, and RuleRAG-FT yields more enhancement. In addition, experiments on four existing RAG datasets show RuleRAG is also effective by offering rules in RuleQA to them, further proving the generalization of rule guidance in RuleRAG.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。