arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.13694cs.CLcs.AIcs.IR
收藏

多源知识剪枝用于检索增强生成:基准与实证研究

Multi-Source Knowledge Pruning for Retrieval-Augmented Generation: A Benchmark and Empirical Study

State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学)

Shuo Yu, Mingyue Cheng, Qi Liu, Daoyu Wang, Jiqian Yang, Jie Ouyang, Yucong Luo, Chenyi Lei, Enhong Chen

AI总结:

针对多源知识RAG缺乏基准的问题,构建了结合结构化与非结构化知识的基准数据集,并提出多粒度剪枝框架PruningRAG,显著提升多种RAG变体性能。

摘要:

检索增强生成(RAG)日益被认为是缓解大型语言模型(LLM)幻觉的有效方法,其通过整合外部知识来实现。尽管已有众多研究,但大多数工作聚焦于单一类型的外部知识源。然而,在现实应用中,大多数场景涉及来自不同来源的多样化知识,但这一领域的研究相对较少。主要困境在于缺乏包含多种知识源的合适数据集,以及对相关问题的预先探索。为应对这些挑战,我们标准化了一个基准数据集,该数据集结合了来自多样且互补领域的结构化与非结构化知识。基于此数据集,我们进一步开发了一个即插即用的RAG框架,名为PruningRAG,其主要特点是使用多粒度剪枝策略来优化相关信息的整合,同时最小化误导性上下文。该框架在多种现有RAG变体上持续提升了性能,展示了其鲁棒性和广泛适用性。基于标准化的数据集和PruningRAG,我们还报告了一系列实验结果以及富有洞察力的发现。我们的数据集和代码已公开(https://github.com/USTCAGI/PruningRAG),旨在推动RAG社区的未来研究。

英文摘要:

Retrieval-augmented generation (RAG) is increasingly recognized as an effective approach to mitigating the hallucination of large language models (LLMs) through the integration of external knowledge. While numerous efforts, most studies focus on a single type of external knowledge source. However, in real-world applications, most situations involve diverse knowledge from various sources, yet this area has been less explored. The main dilemma is the lack of a suitable dataset containing multiple knowledge sources and pre-exploration of the associated issues. To address these challenges, we standardize a benchmark dataset that combines structured and unstructured knowledge across diverse and complementary domains. Based on this dataset, we further develop a plug-and-play RAG framework, \textbf{PruningRAG}, whose main characteristic is the use of multi-granularity pruning strategies to optimize the integration of relevant information while minimizing misleading context. It consistently improves performance across various existing RAG variants, demonstrating its robustness and broad applicability. Building upon the standardized dataset and PruningRAG, we also report a series of experimental results, as well as insightful findings. Our dataset and code are publicly available\footnote{https://github.com/USTCAGI/PruningRAG}, with the aim of advancing future research in the RAG community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑