arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.14403cs.IR
收藏

一种用于设备端检索增强生成的统一模型和文档表示

A Unified Model and Document Representation for On-Device Retrieval-Augmented Generation

University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);Google(谷歌)

Julian Killingback, Ofer Meshi, Henry Li, Hamed Zamani, Maryam Karimzadehgan

AI总结:

本文提出一种统一模型,通过共享表示压缩RAG上下文,减少存储需求并提升设备端性能。

摘要:

传统检索增强生成(RAG)方法假设检索和生成在远离终端用户的强大服务器上进行。虽然这减少了本地硬件限制,但引入了隐私问题、持续的维护和存储成本、增加的延迟以及对互联网连接的依赖。设备端RAG通过在本地执行整个流程来解决这些挑战,使其适合查询敏感个人信息如财务文件、联系方式和医疗记录。然而,设备端部署需要在有限的内存和磁盘空间之间取得平衡。具体而言,提供给生成模型的上下文大小必须受到限制,以管理KV缓存和注意力内存使用,同时存储的嵌入表示大小必须最小化以保存磁盘空间。在本文中,我们提出了一种统一模型,该模型压缩RAG上下文并利用相同的表示进行检索。这种方法相比使用单独表示的方法减少了磁盘使用,同时显著减少了生成所需的上下文大小。平均而言,我们的模型仅需传统RAG读者的1/10上下文,而存储需求与多向量检索模型相当。这种方法是首个使用共享模型和表示统一检索和上下文压缩的模型。我们相信这项工作将启发进一步整合不同模型以优化设备端性能。

英文摘要:

Traditional Retrieval-Augmented Generation (RAG) approaches generally assume that retrieval and generation occur on powerful servers removed from the end user. While this reduces local hardware constraints, it introduces significant drawbacks: privacy concerns regarding data access, recurring maintenance and storage costs, increased latency, and the necessity of an internet connection. On-device RAG addresses these challenges by executing the entire pipeline locally, making it ideal for querying sensitive personal information such as financial documents, contact details, and medical history. However, on-device deployment necessitates a delicate balance between limited memory and disk space. Specifically, the context size provided to the generative model must be restricted to manage KV cache and attention memory usage, while the size of stored embeddings must be minimized to preserve disk space. In this work, we propose a unified model that compresses the RAG context and utilizes the same representations for retrieval. This approach minimizes disk utilization compared to using separate representations, while significantly reducing the context size required for generation. With an average of 1/10 of the context, our model matches the performance of a traditional RAG reader without increasing storage requirements compared to a multi-vector retrieval model. This approach represents the first model to unify retrieval and context compression using a shared model and representation. We believe this work will inspire further consolidation of distinct models to optimize on-device performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑