arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00196cs.CRcs.AIcs.CL
收藏

您的推理请求将变成一个黑箱:面向云上大型语言模型的保密推理

Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models

Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China(高可信软件技术重点实验室(北京大学));School of Computer Science, Peking University, Beijing, China(北京大学计算机学院);Tencent, Shenzhen, China(腾讯(深圳));Macau university, China(澳门大学)

Chung-ju Huang, Huiqiang Zhao, Yuanpeng He, Lijian Li, Wenpin Jiao, Zhi Jin, Peixuan Chen, Leye Wang

AI总结:

Talaria通过在客户端控制的CVM中执行敏感操作并使用ReMO协议保护隐私,实现云上LLM的保密推理,同时保持性能和效率。

摘要:

随着对云托管的大规模语言模型(LLMs)的依赖增加,敏感的客户端数据,如提示和响应,可能会暴露给服务提供商,从而导致隐私泄露。现有方法无法同时确保隐私、保持模型性能和保持计算效率。为了解决这一挑战,我们提出了Talaria,一个保密推理框架,通过将LLM管道划分为保护客户端数据,而不会影响云模型的知识产权或推理质量。Talaria在客户端控制的保密虚拟机(CVM)中执行敏感且权重无关的操作,同时将权重相关的计算卸载到云GPU上。这些环境之间的交互通过我们的可逆掩码外包(ReMO)协议得到保障,该协议使用混合掩码技术在外包计算前可逆地模糊中间数据。广泛的评估表明,Talaria能够防御最先进的令牌推理攻击,将令牌重建精度从超过97.5%降低到平均1.34%,同时保持无损机制,确保输出与原始模型相同,而不会显著降低效率和可扩展性。据我们所知,这是首个确保客户端的提示和响应对云不可用,同时保持模型隐私、性能和效率的工作。

英文摘要:

The increasing reliance on cloud-hosted Large Language Models (LLMs) exposes sensitive client data, such as prompts and responses, to potential privacy breaches by service providers. Existing approaches fail to ensure privacy, maintain model performance, and preserve computational efficiency simultaneously. To address this challenge, we propose Talaria, a confidential inference framework that partitions the LLM pipeline to protect client data without compromising the cloud's model intellectual property or inference quality. Talaria executes sensitive, weight-independent operations within a client-controlled Confidential Virtual Machine (CVM) while offloading weight-dependent computations to the cloud GPUs. The interaction between these environments is secured by our Reversible Masked Outsourcing (ReMO) protocol, which uses a hybrid masking technique to reversibly obscure intermediate data before outsourcing computations. Extensive evaluations show that Talaria can defend against state-of-the-art token inference attacks, reducing token reconstruction accuracy from over 97.5% to an average of 1.34%, all while being a lossless mechanism that guarantees output identical to the original model without significantly decreasing efficiency and scalability. To the best of our knowledge, this is the first work that ensures clients' prompts and responses remain inaccessible to the cloud, while also preserving model privacy, performance, and efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑