arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.06646cs.CV
收藏

Hybrid SD:面向Stable Diffusion模型的边云协同推理

Hybrid SD: Edge-Cloud Collaborative Inference for Stable Diffusion Models

ByteDance Inc.(字节跳动公司)

Chenqian Yan, Songwei Liu, Hongjian Liu, Xurui Peng, Xiaojian Wang, Fangmin Chen, Lean Fu, Xing Mei

AI总结:

针对全尺寸Stable Diffusion模型部署成本高、边缘紧凑型模型质量差的问题,提出无需训练的边云协同推理框架Hybrid SD,结合云端大模型语义规划与边缘小模型细节优化,实现高参数效率与66%云成本降低。

摘要:

Stable Diffusion模型(SDM,稳定扩散模型)在图像合成方面展现出了卓越的性能。然而,其庞大的模型规模与密集的计算需求阻碍了广泛应用,部署通常需要成本高昂的云服务器。另一方面,尽管有许多为边缘设备定制的紧凑型模型能够降低这类需求,但与全尺寸SDM相比,它们往往会在语义完整性和视觉质量上有所折损。为弥补这一差距,我们提出了Hybrid SD——一种创新的、无需训练的SDM推理框架,专为边云协同推理设计。Hybrid SD将扩散过程的早期步骤分配给部署在云服务器上的大模型执行,以增强语义规划;此外,可整合部署在边缘设备上的小型高效模型,用于后期阶段的视觉细节细化。考虑到边缘设备计算与存储能力的多样性,我们对SDM的U-Net进行了结构化剪枝,并训练了一个轻量级VAE。实验评估表明,我们的压缩模型在边缘设备上实现了当前最优的参数效率(2.258亿参数),同时图像质量具备竞争力。此外,Hybrid SD通过边云协同推理将云成本降低了66%。

英文摘要:

Stable Diffusion Models (SDMs) have shown remarkable proficiency in image synthesis. However, their broad application is impeded by their large model sizes and intensive computational requirements, which typically require expensive cloud servers for deployment. On the flip side, while there are many compact models tailored for edge devices that can reduce these demands, they often compromise on semantic integrity and visual quality when compared to full-sized SDMs. To bridge this gap, we introduce Hybrid SD, an innovative, training-free SDMs inference framework designed for edge-cloud collaborative inference. Hybrid SD distributes the early steps of the diffusion process to the large models deployed on cloud servers, enhancing semantic planning. Furthermore, small efficient models deployed on edge devices can be integrated for refining visual details in the later stages. Acknowledging the diversity of edge devices with differing computational and storage capacities, we employ structural pruning to the SDMs U-Net and train a lightweight VAE. Empirical evaluations demonstrate that our compressed models achieve state-of-the-art parameter efficiency (225.8M) on edge devices with competitive image quality. Additionally, Hybrid SD reduces the cloud cost by 66% with edge-cloud collaborative inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑