RAG中相关性是否从检索器传播到生成器？
Is Relevance Propagated from Retriever to Generator in RAG?
AI总结：
本研究针对信息检索任务的主题重叠相关性，利用IR测试集实证探究RAG中检索文档的主题相关性与下游任务效用的关系，发现二者呈弱正相关且相关性随上下文规模增大而降低，同时更优的检索模型通常能提升RAG下游性能。
摘要：
检索增强生成（Retrieval Augmented Generation, RAG）是一种整合外部知识的框架，通常以从文档集合中检索到的一组文档作为大语言模型（LLM）提示的一部分，有望提升问答等下游任务的性能。标准检索任务的目标是最大化排名靠前的文档集合的相关性，而RAG系统的目标则是最大化这些文档的总效用——文档效用指的是将其作为额外上下文信息加入LLM提示后，是否能提升下游任务表现。现有研究探讨了RAG上下文的相关性在知识密集型语言任务（KILT）中的作用，这类任务中相关性本质上体现为答案包含性。与之不同，在本研究中，相关性对应信息检索任务中查询与文档之间的主题重叠程度。具体而言，我们利用一个IR测试集，实证探究由主题相关文档构成的RAG上下文是否能提升下游性能。我们的实验得出以下发现：（a）相关性与效用之间存在较小的正相关；（b）这种相关性随上下文规模增大（k-shot中k值更高）而降低；（c）更有效的检索模型通常能带来更好的下游RAG性能。
英文摘要：
Retrieval Augmented Generation (RAG) is a framework for incorporating external knowledge, usually in the form of a set of documents retrieved from a collection, as a part of a prompt to a large language model (LLM) to potentially improve the performance of a downstream task, such as question answering. Different from a standard retrieval task's objective of maximising the relevance of a set of top-ranked documents, a RAG system's objective is rather to maximise their total utility, where the utility of a document indicates whether including it as a part of the additional contextual information in an LLM prompt improves a downstream task. Existing studies investigate the role of the relevance of a RAG context for knowledge-intensive language tasks (KILT), where relevance essentially takes the form of answer containment. In contrast, in our work, relevance corresponds to that of topical overlap between a query and a document for an information seeking task. Specifically, we make use of an IR test collection to empirically investigate whether a RAG context comprised of topically relevant documents leads to improved downstream performance. Our experiments lead to the following findings: (a) there is a small positive correlation between relevance and utility; (b) this correlation decreases with increasing context sizes (higher values of k in k-shot); and (c) a more effective retrieval model generally leads to better downstream RAG performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
