arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2505.18521cs.CV
收藏

改进的不可混扩散:通过降低可混性加速扩散训练

Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility

UC Berkeley(加州大学伯克利分校);UT Austin(德克萨斯大学奥斯汀分校)

Yiheng Li, Feng Liang, Dan Kondratyuk, Masayoshi Tomizuka, Kurt Keutzer, Chenfeng Xu

AI总结:

提出通过降低任意层级的可混性来加速扩散模型训练,实现多种实现方式(如KNN噪声选择和图像缩放),在多种任务上获得高达4倍以上的训练加速。

摘要:

扩散模型的高昂训练成本阻碍了其部署。最近提出的不可混扩散(Immiscible Diffusion)表明,通过线性分配减少噪声空间中的扩散轨迹混合可以简化去噪过程,从而加速训练。为了将不可混扩散扩展到高批量大小和高维度下低效的线性分配之外,我们将这一概念细化到更广泛的任意层级的可混性降低,并通过任意实现方式。具体地,我们实证证明了去噪过程相对于不可混扩散的双射性质,确保了其生成多样性的保持。此外,我们提供了深入分析,并逐步展示了不可混性如何简化去噪并提高效率。超越线性分配,我们提出了一系列实现方式,包括K近邻(KNN)噪声选择和图像缩放来降低可混性,在多种模型和任务(包括无条件/条件生成、图像编辑和机器人规划)上实现了高达4倍以上的训练加速。此外,我们对不可混性的分析为最优传输(OT)如何增强扩散训练提供了新的视角。通过将轨迹可混性识别为基本瓶颈,我们相信这项工作为未来高效扩散训练的研究建立了一个潜在的新方向。代码可在该 https URL 获取。

英文摘要:

The substantial training cost of diffusion models hinders their deployment. Immiscible Diffusion recently showed that reducing diffusion trajectory mixing in the noise space via linear assignment accelerates training by simplifying denoising. To extend immiscible diffusion beyond the inefficient linear assignment under high batch sizes and high dimensions, we refine this concept to a broader miscibility reduction at any layer and by any implementation. Specifically, we empirically demonstrate the bijective nature of the denoising process with respect to immiscible diffusion, ensuring its preservation of generative diversity. Moreover, we provide thorough analysis and show step-by-step how immiscibility eases denoising and improves efficiency. Extending beyond linear assignment, we propose a family of implementations including K-nearest neighbor (KNN) noise selection and image scaling to reduce miscibility, achieving up to >4x faster training across diverse models and tasks including unconditional/conditional generation, image editing, and robotics planning. Furthermore, our analysis of immiscibility offers a novel perspective on how optimal transport (OT) enhances diffusion training. By identifying trajectory miscibility as a fundamental bottleneck, we believe this work establishes a potentially new direction for future research into high-efficiency diffusion training. The code is available at https://github.com/yhli123/Immiscible-Diffusion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑