arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.02031cs.DCcs.AIcs.LG
收藏

SwiftDiffusion:基于附加模块的高效扩散模型服务

SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules

HKUST(香港科技大学);Alibaba Group(阿里巴巴集团)

Suyi Li, Lingyun Yang, Xiaoxiao Jiang, Hanfeng Lu, Dakai An, Zhipeng Di, Weiyi Lu, Jiawei Chen, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, Wei Wang

AI总结:

针对扩散模型附加模块(ControlNet、LoRA)导致的服务延迟问题,SwiftDiffusion通过解耦部署ControlNet、BAL技术和潜在并行优化,在H800 GPU上服务SDXL模型时实现7.8倍延迟降低和1.6倍吞吐量提升。

摘要:

使用扩散模型的文本到图像(T2I)生成已成为当今AI云服务中的热门应用。生产环境中的T2I服务通常包含一个服务流程,其中基础扩散模型通过各种“附加”模块(尤其是ControlNet和LoRA)增强图像生成控制能力。与单独服务基础模型相比,这些附加模块引入了显著的加载和计算开销,导致延迟增加。本文提出SwiftDiffusion,这是一个通过整体方法高效服务T2I流程的系统。SwiftDiffusion将ControlNet与基础模型解耦,并将其作为独立扩展的服务部署在专用GPU上,实现ControlNet缓存、并行化和共享。为缓解LoRA服务的高加载开销,SwiftDiffusion采用有界异步LoRA加载(BAL)技术,允许LoRA加载与初始基础模型执行重叠最多k步,同时不影响图像质量。此外,SwiftDiffusion通过新颖的潜在并行技术优化基础模型执行。这些设计共同使SwiftDiffusion优于最先进的T2I服务系统,在H800 GPU上服务SDXL模型时实现高达7.8倍的延迟降低和1.6倍的吞吐量提升,且不牺牲图像质量。

英文摘要:

Text-to-image (T2I) generation using diffusion models has become a blockbuster service in today's AI cloud. A production T2I service typically involves a serving workflow where a base diffusion model is augmented with various "add-on" modules, notably ControlNet and LoRA, to enhance image generation control. Compared to serving the base model alone, these add-on modules introduce significant loading and computational overhead, resulting in increased latency. In this paper, we present SwiftDiffusion, a system that efficiently serves a T2I workflow through a holistic approach. SwiftDiffusion decouples ControNet from the base model and deploys it as a separate, independently scaled service on dedicated GPUs, enabling ControlNet caching, parallelization, and sharing. To mitigate the high loading overhead of LoRA serving, SwiftDiffusion employs a bounded asynchronous LoRA loading (BAL) technique, allowing LoRA loading to overlap with the initial base model execution by up to k steps without compromising image quality. Furthermore, SwiftDiffusion optimizes base model execution with a novel latent parallelism technique. Collectively, these designs enable SwiftDiffusion to outperform the state-of-the-art T2I serving systems, achieving up to 7.8x latency reduction and 1.6x throughput improvement in serving SDXL models on H800 GPUs, without sacrificing image quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑