不要忘记私有检索:面向大型语言模型的分布式私有相似性搜索
Don't forget private retrieval: distributed private similarity search for large language models
AI总结:
提出基于多方计算的私有检索增强生成(PRAG)方法,通过MPC友好型倒排文件近似搜索协议实现分布式私有数据上的安全top-k检索,确保查询与数据库内容隐私。
摘要:
尽管大型语言模型(LLMs)的灵活能力使其能够基于已学知识回答一系列查询,但用于增强生成的信息检索是让LLMs回答未包含在预训练数据中的信息的重要工具。这类私有信息正越来越多地由组织和个人在广泛的分布式环境中生成。使用查询和文档的神经嵌入执行此类信息检索总是会泄露有关查询和数据库内容的信息,除非两者都存储在本地。我们提出了私有检索增强生成(PRAG),这是一种使用多方计算(MPC)安全地将查询传输到包含私有构建数据库的分布式服务器集以返回top-k和近似top-k文档的方法。这是首个确保没有服务器能观察到客户端查询或看到数据库内容的密集信息检索方法。该方法引入了一种新颖的MPC友好型倒排文件近似搜索(IVF)协议,允许在分布式和私有数据上以亚线性通信复杂度进行快速文档搜索。这项工作开辟了新途径,使得用于LLMs的数据可以在无需集中化或放弃隐私的情况下被访问和使用。
英文摘要:
While the flexible capabilities of large language models (LLMs) allow them to answer a range of queries based on existing learned knowledge, information retrieval to augment generation is an important tool to allow LLMs to answer questions on information not included in pre-training data. Such private information is increasingly being generated in a wide array of distributed contexts by organizations and individuals. Performing such information retrieval using neural embeddings of queries and documents always leaked information about queries and database content unless both were stored locally. We present Private Retrieval Augmented Generation (PRAG), an approach that uses multi-party computation (MPC) to securely transmit queries to a distributed set of servers containing a privately constructed database to return top-k and approximate top-k documents. This is a first-of-its-kind approach to dense information retrieval that ensures no server observes a client's query or can see the database content. The approach introduces a novel MPC friendly protocol for inverted file approximate search (IVF) that allows for fast document search over distributed and private data in sublinear communication complexity. This work presents new avenues through which data for use in LLMs can be accessed and used without needing to centralize or forgo privacy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。