arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.23969cs.DCcs.CRcs.PF
收藏

序列化桥接:理解与恢复Blackwell GPU机密计算下的LLM服务性能

The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing

Hang Yin, Kevin Wang

AI总结:

本文发现Intel TDX加GPU-CC下LLM服务吞吐量下降13-27%的主因是机密VM-GPU桥接的序列化开销,而非GPU计算本身,并通过调度优化恢复高达92%的性能损失。

摘要:

GPU机密计算(GPU-CC)现在保留了GPU本地性能:在NVIDIA B300上,BF16矩阵乘法的性能为非机密性能的0.998倍。然而,在Intel TDX加GPU-CC下,LLM服务仍然损失13-27%的吞吐量,且KV缓存恢复延迟可能增加一倍以上。本文在两个Blackwell平台(RTX Pro 6000和B300 HGX)上研究了这一差距,并确定了其主要原因:机密VM-GPU桥接,而非GPU计算。我们发现,GPU-CC将主机/设备移动转变为序列化的、高设置成本的通道。安全拷贝在上下文内无法获得CUDA流并发性,异步传输在运行时边界阻塞,小规模跨域传输需支付固定开销。这违反了现代推理运行时的假设,即DMA应是廉价、并发和异步的。在vLLM密集解码中,差距围绕慢44倍的小分配和拷贝操作闭合;针对性补丁排除了其他解释。一个调度标志恢复了57%的差距,而工作线程排空在合格的高并发运行中恢复了高达92%的差距。相同的桥接模型解释了+131%的KV恢复惩罚和34倍的模型加载减速。Blackwell还改变了机密租户单元。我们验证了B300上的机密多GPU NVSwitch租户,包括CVM内510 GB/s的NVLink P2P和并发隔离租户,并识别了生产级机密AI平台中剩余的织物认证差距。

英文摘要:

GPU Confidential Computing (GPU-CC) now preserves GPU-local performance: on NVIDIA B300, BF16 matmul runs at 0.998x of non-confidential performance. Yet LLM serving under Intel TDX plus GPU-CC still loses 13-27% of throughput, and KV-cache restore latency can more than double. This paper studies that gap on two Blackwell platforms, RTX Pro 6000 and B300 HGX, and identifies its dominant cause: the confidential VM-GPU bridge, not GPU compute. We find that GPU-CC turns host/device movement into a serialized, high-setup-cost channel. Secure copies do not gain CUDA-stream concurrency within a context, asynchronous transfers block at the runtime boundary, and small crossings pay a fixed toll. This violates the assumptions of modern inference runtimes, where DMA is expected to be cheap, concurrent, and asynchronous. In vLLM dense decode, the gap closes around 44x-slower small alloc-and-copy operations; targeted patches reject alternative explanations. A scheduling flag recovers 57% of the gap, while a worker-thread drain recovers up to 92% in qualified high-concurrency runs. The same bridge model explains a +131% KV-restore penalty and a 34x model-load slowdown. Blackwell also changes the confidential tenancy unit. We qualify confidential multi-GPU NVSwitch tenants on B300, including 510 GB/s NVLink P2P inside a CVM and concurrent isolated tenants, and identify the remaining fabric-attestation gap for production confidential AI platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑