arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.21760cs.CV
收藏

通过基于条件引导调度的混合数据-流水线并行加速扩散

Accelerating Diffusion via Hybrid Data-Pipeline Parallelism Based on Conditional Guidance Scheduling

School of Computing, KAIST(计算机学院,韩国科学技术院)

Euisoo Jung, Byunghyun Kim, Hyunjin Kim, Seonghye Cho, Jae-Gil Lee

AI总结:

本文提出了一种混合并行框架,通过条件引导调度和自适应流水线切换,提升扩散模型的生成效率和质量。

摘要:

扩散模型在高保真图像、视频和音频生成中取得了显著进展,但推断仍计算成本高昂。然而,目前基于分布式并行的扩散加速方法存在明显的生成伪影,并未能实现与GPU数量成比例的显著加速。因此,我们提出了一种混合并行框架,结合了新的数据并行策略,基于条件的分区,与最优流水线调度方法,自适应并行切换,以减少生成延迟并在条件扩散模型中实现高质量生成。关键思想是(i)利用条件和无条件去噪路径作为新的数据分区视角,(ii)根据这两个路径之间的去噪差异自适应地启用最优流水线并行。我们的框架在使用两块NVIDIA RTX 3090 GPU时,分别在SDXL和SD3上实现了2.31倍和2.07倍的延迟降低,同时保持图像质量。这一结果证实了我们的方法在基于U-Net的扩散模型和基于DiT的流匹配架构中的通用性。我们的方法在高分辨率合成设置下也优于现有方法。代码可在https://github.com/kaist-dmlab/Hybridiff上获得。

英文摘要:

Diffusion models have achieved remarkable progress in high-fidelity image, video, and audio generation, yet inference remains computationally expensive. Nevertheless, current diffusion acceleration methods based on distributed parallelism suffer from noticeable generation artifacts and fail to achieve substantial acceleration proportional to the number of GPUs. Therefore, we propose a hybrid parallelism framework that combines a novel data parallel strategy, condition-based partitioning, with an optimal pipeline scheduling method, adaptive parallelism switching, to reduce generation latency and achieve high generation quality in conditional diffusion models. The key ideas are to (i) leverage the conditional and unconditional denoising paths as a new data-partitioning perspective and (ii) adaptively enable optimal pipeline parallelism according to the denoising discrepancy between these two paths. Our framework achieves $2.31\times$ and $2.07\times$ latency reductions on SDXL and SD3, respectively, using two NVIDIA RTX~3090 GPUs, while preserving image quality. This result confirms the generality of our approach across U-Net-based diffusion models and DiT-based flow-matching architectures. Our approach also outperforms existing methods in acceleration under high-resolution synthesis settings. Code is available at https://github.com/kaist-dmlab/Hybridiff.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑