使用相关信息增益改进检索增强生成（RAG）
Better RAG using Relevant Information Gain
AI总结：
针对检索增强生成（RAG）中上下文窗口有限导致冗余信息占用空间的问题，提出基于相关信息增益的优化指标，通过该指标自然实现检索段落多样性与相关性平衡，在RGB问答任务上性能优于现有方法。
摘要：
扩展大型语言模型（LLMs）内存的常用方法是检索增强生成（RAG），该方法将从更大内存中检索到的文本插入LLM的上下文窗口。然而，上下文窗口通常限制在几千个token，这限制了可用于为模型响应提供信息的检索段落数量。因此，通过确保检索段落之间的一定多样性来避免用冗余信息占用上下文窗口空间非常重要。同时，信息还应与当前任务相关。大多数鼓励检索结果多样性的现有方法（如最大边际相关性（MMR））通过纳入明确权衡多样性和相关性的目标来实现这一点。我们提出了一种基于相关信息增益的新型简单优化指标，该指标是衡量一组检索结果中与查询相关的总信息的概率指标。通过优化该指标，多样性自然从我们的系统中产生。当作为RAG系统检索组件的即插即用替代品时，该方法在检索增强生成基准（RGB）的问答任务上取得了最先进的性能，优于直接优化相关性和多样性的现有指标。
英文摘要：
A common way to extend the memory of large language models (LLMs) is by retrieval augmented generation (RAG), which inserts text retrieved from a larger memory into an LLM's context window. However, the context window is typically limited to several thousand tokens, which limits the number of retrieved passages that can inform a model's response. For this reason, it's important to avoid occupying context window space with redundant information by ensuring a degree of diversity among retrieved passages. At the same time, the information should also be relevant to the current task. Most prior methods that encourage diversity among retrieved results, such as Maximal Marginal Relevance (MMR), do so by incorporating an objective that explicitly trades off diversity and relevance. We propose a novel simple optimization metric based on relevant information gain, a probabilistic measure of the total information relevant to a query for a set of retrieved results. By optimizing this metric, diversity organically emerges from our system. When used as a drop-in replacement for the retrieval component of a RAG system, this method yields state-of-the-art performance on question answering tasks from the Retrieval Augmented Generation Benchmark (RGB), outperforming existing metrics that directly optimize for relevance and diversity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
