SwiftDiffusion:基于附加模块的高效扩散模型服务
SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules
AI总结:
针对扩散模型附加模块(ControlNet、LoRA)导致的服务延迟问题,SwiftDiffusion通过解耦部署ControlNet、BAL技术和潜在并行优化,在H800 GPU上服务SDXL模型时实现7.8倍延迟降低和1.6倍吞吐量提升。
摘要:
使用扩散模型的文本到图像(T2I)生成已成为当今AI云服务中的热门应用。生产环境中的T2I服务通常包含一个服务流程,其中基础扩散模型通过各种“附加”模块(尤其是ControlNet和LoRA)增强图像生成控制能力。与单独服务基础模型相比,这些附加模块引入了显著的加载和计算开销,导致延迟增加。本文提出SwiftDiffusion,这是一个通过整体方法高效服务T2I流程的系统。SwiftDiffusion将ControlNet与基础模型解耦,并将其作为独立扩展的服务部署在专用GPU上,实现ControlNet缓存、并行化和共享。为缓解LoRA服务的高加载开销,SwiftDiffusion采用有界异步LoRA加载(BAL)技术,允许LoRA加载与初始基础模型执行重叠最多k步,同时不影响图像质量。此外,SwiftDiffusion通过新颖的潜在并行技术优化基础模型执行。这些设计共同使SwiftDiffusion优于最先进的T2I服务系统,在H800 GPU上服务SDXL模型时实现高达7.8倍的延迟降低和1.6倍的吞吐量提升,且不牺牲图像质量。
英文摘要:
Text-to-image (T2I) generation using diffusion models has become a blockbuster service in today's AI cloud. A production T2I service typically involves a serving workflow where a base diffusion model is augmented with various "add-on" modules, notably ControlNet and LoRA, to enhance image generation control. Compared to serving the base model alone, these add-on modules introduce significant loading and computational overhead, resulting in increased latency. In this paper, we present SwiftDiffusion, a system that efficiently serves a T2I workflow through a holistic approach. SwiftDiffusion decouples ControNet from the base model and deploys it as a separate, independently scaled service on dedicated GPUs, enabling ControlNet caching, parallelization, and sharing. To mitigate the high loading overhead of LoRA serving, SwiftDiffusion employs a bounded asynchronous LoRA loading (BAL) technique, allowing LoRA loading to overlap with the initial base model execution by up to k steps without compromising image quality. Furthermore, SwiftDiffusion optimizes base model execution with a novel latent parallelism technique. Collectively, these designs enable SwiftDiffusion to outperform the state-of-the-art T2I serving systems, achieving up to 7.8x latency reduction and 1.6x throughput improvement in serving SDXL models on H800 GPUs, without sacrificing image quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。