arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.09292cs.CVcs.AI
收藏

面向视觉的高效扩散模型:综述

Efficient Diffusion Models for Vision: A Survey

Anwaar Ulhaq, Naveed Akhtar

AI总结:

本综述聚焦视觉领域扩散模型的计算效率问题,梳理了近期提升模型效率的关键设计方案,旨在推动实用化高效扩散模型研究,并对该方向未来发展进行了展望。

摘要:

扩散模型(Diffusion Models, DMs)已在内容生成任务中展现出当前最优性能,且无需采用对抗训练。这类模型的训练分为两个步骤:首先,前向扩散过程逐步向数据(通常为图像)中添加噪声;随后,反向扩散过程逐步去除噪声,将其转化为所建模目标分布的样本。DMs的灵感源于非平衡热力学,且固有计算复杂度较高的特性。由于需要在高维空间中频繁进行函数求值与梯度计算,这类模型在训练和推理阶段都会产生可观的计算开销。这不仅会阻碍基于扩散的建模技术的普及,还会限制扩散模型在实际应用中的适配。更值得注意的是,由于能源过度消耗与环境隐患,计算模型的效率正迅速成为重要关切。这些因素推动了学界涌现出大量聚焦于设计计算高效型DMs的研究成果。在本综述中,我们梳理了视觉领域扩散模型的最新进展,重点关注影响DMs计算效率的关键设计维度,尤其着重介绍了近期提出的、能实现更高效DMs的设计方案。与其他从宏观视角讨论扩散模型的近期综述不同,本综述旨在通过梳理学界中能产出更具实用性模型的设计策略,推动该研究方向的发展,惠及更广泛的研究群体。我们还从计算效率的角度,对视觉领域扩散模型的未来发展方向进行了展望。

英文摘要:

Diffusion Models (DMs) have demonstrated state-of-the-art performance in content generation without requiring adversarial training. These models are trained using a two-step process. First, a forward - diffusion - process gradually adds noise to a datum (usually an image). Then, a backward - reverse diffusion - process gradually removes the noise to turn it into a sample of the target distribution being modelled. DMs are inspired by non-equilibrium thermodynamics and have inherent high computational complexity. Due to the frequent function evaluations and gradient calculations in high-dimensional spaces, these models incur considerable computational overhead during both training and inference stages. This can not only preclude the democratization of diffusion-based modelling, but also hinder the adaption of diffusion models in real-life applications. Not to mention, the efficiency of computational models is fast becoming a significant concern due to excessive energy consumption and environmental scares. These factors have led to multiple contributions in the literature that focus on devising computationally efficient DMs. In this review, we present the most recent advances in diffusion models for vision, specifically focusing on the important design aspects that affect the computational efficiency of DMs. In particular, we emphasize the recently proposed design choices that have led to more efficient DMs. Unlike the other recent reviews, which discuss diffusion models from a broad perspective, this survey is aimed at pushing this research direction forward by highlighting the design strategies in the literature that are resulting in practicable models for the broader research community. We also provide a future outlook of diffusion models in vision from their computational efficiency viewpoint.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑