零冗余知识下的检索增强生成：Zero-RAG
Zero-RAG: Towards Retrieval-Augmented Generation with Zero Redundant Knowledge
AI总结：
Zero-RAG通过修剪冗余知识和优化检索流程，提升LLM在减少冗余信息下的生成性能。
摘要：
检索增强生成在解决大型语言模型的幻觉问题上表现出显著成效，通常使用大量外部语料库来补充LLM的知识。然而，随着LLM的发展，其内部知识显著扩展，导致外部语料库与LLM之间出现显著的知识冗余。一方面，密集检索的索引成本与语料库大小密切相关，因此显著冗余的知识加剧了密集检索的工作量。另一方面，外部语料库中的冗余知识对LLM无益，我们的探索性分析表明，它反而会损害RAG在LLM能够自行回答的问题上的性能。为了解决这些问题，我们提出了Zero-RAG来应对这些挑战。具体来说，我们首先提出了Mastery-Score指标，用于识别RAG语料库中的冗余知识以进行修剪。修剪后，回答“掌握”的问题主要依赖于LLM的内部知识。为了更好地利用内部能力，我们提出了查询路由和噪声容忍微调，以避免无关文档的干扰，从而进一步提高LLM在修剪后的语料库中对内部知识的利用。实验结果表明，Zero-RAG将维基百科语料库修剪了30%，并加快了检索阶段的速度22%，而不会影响RAG的性能。
英文摘要：
Retrieval-Augmented Generation has shown remarkable results to address Large Language Models' hallucinations, which usually uses a large external corpus to supplement knowledge to LLMs. However, with the development of LLMs, the internal knowledge of LLMs has expanded significantly, thus causing significant knowledge redundancy between the external corpus and LLMs. On the one hand, the indexing cost of dense retrieval is highly related to the corpus size and thus significant redundant knowledge intensifies the dense retrieval's workload. On the other hand, the redundant knowledge in the external corpus is not helpful to LLMs and our exploratory analysis shows that it instead hurts the RAG performance on those questions which the LLM can answer by itself. To address these issues, we propose Zero-RAG to tackle these challenges. Specifically, we first propose the Mastery-Score metric to identify redundant knowledge in the RAG corpus to prune it. After pruning, answers to "mastered" questions rely primarily on internal knowledge of the LLM. To better harness the internal capacity, we propose Query Router and Noise-Tolerant Tuning to avoid the irrelevant documents' distraction and thus further improve the LLM's utilization of internal knowledge with pruned corpus. Experimental results show that Zero-RAG prunes the Wikipedia corpus by 30\% and accelerates the retrieval stage by 22\%, without compromising RAG's performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
