OSCAR: 在线软压缩与重排序
OSCAR: Online Soft Compression And Reranking
AI总结：
OSCAR通过在线软压缩和重排序方法提升RAG管道效率，实现推理速度提升2-5倍且准确率无损失。
摘要：
检索增强生成（RAG）通过整合外部知识来增强大型语言模型（LLMs），从而提高准确性和相关性。然而，随着检索规模的扩大，扩展RAG管道仍然计算成本高昂。为此，我们引入了OSCAR，一种新颖的查询依赖的在线软压缩方法，该方法在减少计算开销的同时保持性能。与传统的硬压缩方法不同，后者缩短检索文本，或软压缩方法，后者将文档映射到连续嵌入中，OSCAR在推理时间动态压缩检索信息，消除了存储开销，并允许更高的压缩率。此外，我们扩展了OSCAR，使其能够同时执行重排序，进一步优化RAG管道的效率。我们的实验表明，与最先进的性能相比，推理速度提高了2-5倍，而对于从1B到24B参数的LLM来说，准确率几乎没有损失。模型可在：https://huggingface.co/collections/naver/oscar-67d446a8e3a2551f57464295获取。
英文摘要：
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge, leading to improved accuracy and relevance. However, scaling RAG pipelines remains computationally expensive as retrieval sizes grow. To address this, we introduce OSCAR, a novel query-dependent online soft compression method that reduces computational overhead while preserving performance. Unlike traditional hard compression methods, which shorten retrieved texts, or soft compression approaches, which map documents to continuous embeddings offline, OSCAR dynamically compresses retrieved information at inference time, eliminating storage overhead and enabling higher compression rates. Additionally, we extend OSCAR to simultaneously perform reranking, further optimizing the efficiency of the RAG pipeline. Our experiments demonstrate state-of-the-art performance with a 2-5x speed-up in inference and minimal to no loss in accuracy for LLMs ranging from 1B to 24B parameters. The models are available at: https://huggingface.co/collections/naver/oscar-67d446a8e3a2551f57464295.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
