arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.09174cs.CVcs.AIcs.LGeess.IV
收藏

改进扩散模型:无混叠重采样与增强的旋转等变性

Advancing Diffusion Models: Alias-Free Resampling and Enhanced Rotational Equivariance

University of California San Francisco(加利福尼亚大学旧金山分校);Department of Neurology(神经学系)

Md Fahim Anjum

AI总结:

本文指出扩散模型中不当重采样会引入混叠伪影,通过在UNet架构中无参数增加地引入无混叠重采样层,有效提升了图像合成质量与旋转等变性,并实现了免训练的用户可控图像旋转。

摘要:

图像生成的最新进展,特别是通过扩散模型,在图像合成质量上取得了令人瞩目的提升。尽管如此,扩散模型仍然面临模型引入的伪影和图像保真度有限稳定性的挑战。在这项工作中,我们假设该问题的主要原因是扩散模型中引入混叠的不当重采样操作,而基于图像处理理论的仔细的无混叠重采样可以提升模型的图像合成性能。我们提出将无混叠重采样层集成到扩散模型的UNet架构中,且不增加额外的可训练参数,从而保持计算效率。然后,我们评估这些理论驱动的修改是否提升了图像质量和旋转等变性。我们在基准数据集(包括CIFAR-10、MNIST和MNIST-M)上的实验结果显示,图像质量取得了一致的提升,特别是在FID和KID分数方面。此外,我们提出了一种改进的扩散过程,能够实现用户控制的生成图像旋转,而无需额外训练。我们的研究结果强调了诸如无混叠重采样等理论驱动的增强方法在生成模型中提升图像质量同时保持模型效率的潜力,并为未来将其整合到视频生成扩散模型中的研究方向开辟了道路,从而能够更深入地探索无混叠重采样在生成建模中的应用。

英文摘要:

Recent advances in image generation, particularly via diffusion models, have led to impressive improvements in image synthesis quality. Despite this, diffusion models are still challenged by model-induced artifacts and limited stability in image fidelity. In this work, we hypothesize that the primary cause of this issue is the improper resampling operation that introduces aliasing in the diffusion model and a careful alias-free resampling dictated by image processing theory can improve the model's performance in image synthesis. We propose the integration of alias-free resampling layers into the UNet architecture of diffusion models without adding extra trainable parameters, thereby maintaining computational efficiency. We then assess whether these theory-driven modifications enhance image quality and rotational equivariance. Our experimental results on benchmark datasets, including CIFAR-10, MNIST, and MNIST-M, reveal consistent gains in image quality, particularly in terms of FID and KID scores. Furthermore, we propose a modified diffusion process that enables user-controlled rotation of generated images without requiring additional training. Our findings highlight the potential of theory-driven enhancements such as alias-free resampling in generative models to improve image quality while maintaining model efficiency and pioneer future research directions to incorporate them into video-generating diffusion models, enabling deeper exploration of the applications of alias-free resampling in generative modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑