从糟粕中分离精华:在检索增强生成中筛选分歧观点
Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
AI总结:
本文提出WinnowRAG框架,通过查询感知聚类和批评LLM迭代筛选噪声文档,在保留有用知识的同时提升检索增强生成的准确性,且无需模型微调。
摘要:
检索增强生成(RAG)通过整合外部知识源来增强大型语言模型(LLMs),以解决其在获取最新或专业信息方面的局限性。提高检索到相关信息可能性的一种自然策略是增加检索文档的数量。然而,涉及更多文档可能引入显著噪声,因为许多文档可能不相关或具有误导性,从而降低生成响应的整体准确性。为了克服处理大量文档所带来的挑战,我们提出了WinnowRAG,这是一个新颖的RAG框架,旨在系统性地过滤噪声文档,同时保留有价值的内容——我们将这一过程称为“winnowing(筛选)”。WinnowRAG分两个阶段运行:在第一阶段,我们执行查询感知聚类,将相似文档分组并形成不同的主题簇。每个簇分配给一个LLM智能体以生成独特的答案。在第二阶段,我们执行winnowing,其中批评LLM评估多个智能体的输出,并迭代地将有用文档与噪声文档分离。为了在丢弃智能体时保留有用文档,我们提出了两种策略性合并技术,以确保仅使用相关知识生成最终响应。至关重要的是,WinnowRAG与模型无关,不需要任何模型微调,使其易于适应各种任务。在各种真实数据集上的大量实验表明,WinnowRAG相对于最先进的基线方法具有有效性。
英文摘要:
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge sources to address their limitations in accessing up-to-date or specialized information. A natural strategy to increase the likelihood of retrieving relevant information is to expand the number of retrieved documents. However, involving more documents could introduce significant noise, as many documents may be irrelevant or misleading, thereby reducing the overall accuracy of the generated responses. To overcome the challenge associated with handling a larger number of documents, we propose WinnowRAG, a novel RAG framework designed to systematically filter out noisy documents while preserving valuable content -- a process we refer to as winnowing. WinnowRAG operates in two stages: In Stage I, we perform query-aware clustering to group similar documents and form distinct topic clusters. Each cluster is assigned to an LLM agent for generating a unique answer. In Stage II, we perform winnowing, wherein a critic LLM evaluates the outputs of multiple agents and iteratively separates useful documents from noisy ones. To retain useful documents when discarding agents, we propose two strategic merging techniques to ensure that only relevant knowledge is used for generating the final response. Crucially, WinnowRAG is model-agnostic and does not require any model fine-tuning, making it easily adaptable to various tasks. Extensive experiments on various realistic datasets demonstrate the effectiveness of WinnowRAG over state-of-the-art baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。