PipeLLM：基于推测性流水线加密的快速且保密的大语言模型服务
PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption
AI总结：
PipeLLM通过推测性流水线加密重叠加密与GPU计算，解决机密计算中加密带来的高开销，实现快速且保密的LLM服务，实验显示吞吐量开销小于19.6%。
摘要：
在GPU（如NVIDIA H100）上的机密计算通过实现强隔离和数据加密，缓解了外包大语言模型（LLM）的安全风险。然而，这种加密带来了显著的性能开销，在服务OPT-30B和OPT-66B时，吞吐量分别下降高达52.8%和88.2%。为了解决这一挑战，我们引入了PipeLLM，一个用户透明的运行时系统。PipeLLM通过流水线化重叠加密和GPU计算来消除开销——这一想法受CPU指令流水线的启发——从而有效隐藏加密引起的延迟增加。主要的技术挑战在于，与CPU不同，加密模块在GPU请求数据之前无法预先知道需要加密的具体数据。为此，我们提出了推测性流水线加密，通过分析LLM的服务模式来预测需要加密的数据。此外，我们开发了一种高效、低成本的流水线放弃方法，用于处理预测错误的情况。我们在NVIDIA H100 GPU上的实验表明，与没有机密计算的原始系统（如vLLM、PEFT和FlexGen）相比，PipeLLM在各种LLM规模（从13B到175B）下仅产生适度的开销（吞吐量下降小于19.6%）。
英文摘要：
Confidential computing on GPUs, like NVIDIA H100, mitigates the security risks of outsourced Large Language Models (LLMs) by implementing strong isolation and data encryption. Nonetheless, this encryption incurs a significant performance overhead, reaching up to 52.8 percent and 88.2 percent throughput drop when serving OPT-30B and OPT-66B, respectively. To address this challenge, we introduce PipeLLM, a user-transparent runtime system. PipeLLM removes the overhead by overlapping the encryption and GPU computation through pipelining - an idea inspired by the CPU instruction pipelining - thereby effectively concealing the latency increase caused by encryption. The primary technical challenge is that, unlike CPUs, the encryption module lacks prior knowledge of the specific data needing encryption until it is requested by the GPUs. To this end, we propose speculative pipelined encryption to predict the data requiring encryption by analyzing the serving patterns of LLMs. Further, we have developed an efficient, low-cost pipeline relinquishing approach for instances of incorrect predictions. Our experiments on NVIDIA H100 GPU show that compared with vanilla systems without confidential computing (e.g., vLLM, PEFT, and FlexGen), PipeLLM incurs modest overhead (less than 19.6 percent in throughput) across various LLM sizes, from 13B to 175B.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
