HyperRAG：通过重用重排序器 KV-Cache 增强 Retrieval-Augmented Generation 中的质量-效率权衡
HyperRAG: Enhancing Quality-Efficiency Tradeoffs in Retrieval-Augmented Generation with Reranker KV-Cache Reuse
AI总结：
提出HyperRAG系统，通过重用文档侧KV-cache进行高效重排序器推理并辅以系统级优化，在提升RAG流水线吞吐量的同时保证甚至提高了下游生成质量。
摘要：
Retrieval-Augmented Generation (RAG) 已成为一种强大的范式，通过将外部知识整合到生成过程中来增强大型语言模型（LLMs）的性能。RAG 流水线的一个关键组件是重排序器，它从检索到的候选文档池中选择最相关的文档，并显著提高生成响应的质量。虽然重排序器在 RAG 流水线中优化了检索文档的选择，但它们引入了阻碍高吞吐量和低延迟的计算挑战。为了解决这个问题，我们提出了 HyperRAG，这是一个通过利用 KV-cache 重用进行高效重排序器推理，来优化 RAG 流水线中质量与效率权衡的系统。通过重用文档侧的 KV-cache，HyperRAG 同时实现了高质量生成和系统级效率。为了充分发挥 KV-cache 重用的优势，HyperRAG 结合了一系列旨在提高效率和可扩展性的系统级优化。实验表明，HyperRAG 在使用 decoder-only 重排序器时实现了 2 - 3 倍的吞吐量提升，同时与传统 RAG 服务相比，还提供了更高的下游性能。
英文摘要：
Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing the performance of large language models (LLMs) by integrating external knowledge into the generation process. A key component of RAG pipelines is the reranker, which selects the most relevant documents from a pool of retrieved candidates and significantly improves the quality of the generated responses. While rerankers refine the selection of retrieved documents in RAG pipelines, they introduce computational challenges that hinder high throughput and low latency. To address this problem, we propose HyperRAG, a system that optimizes the trade-off between quality and efficiency in RAG pipelines by leveraging KV-cache reuse for efficient reranker inference. By reusing document-side KV-cache, HyperRAG achieves both high-quality generation and system-level efficiency. To fully realize the benefits of KV-cache reuse, HyperRAG incorporates a range of system-level optimizations designed to enhance efficiency and scalability. Experiments show that HyperRAG achieves a 2 - 3 throughput improvement with decoder-only rerankers while also delivering higher downstream performance compared with traditional RAG service.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
