arXiv 2406.094162024-11-28cs.CV

通过多分辨率扩散模型和时间依赖层归一化缓解图像生成中的失真

Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalization

ByteDance（字节跳动）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）

Qihao Liu, Zhanpeng Zeng, Ju He, Qihang Yu, Xiaohui Shen, Liang-Chieh Chen

备注：: Introducing DiMR, a new diffusion backbone that surpasses all existing image generation models of various sizes on ImageNet 256 with only 505M parameters. Project page: https://qihao067.github.io/projects/DiMR

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对Transformer架构在图像生成中因分块化导致的视觉保真度与计算复杂度权衡及失真问题，提出整合多分辨率网络和时间依赖层归一化的扩散模型增强方法，在ImageNet基准上取得更优FID分数。

摘要：

本文通过整合新型多分辨率网络和时间依赖层归一化，提出了对扩散模型的创新增强。扩散模型因其在高保真图像生成中的有效性而备受关注。传统方法依赖卷积U-Net架构，而近期基于Transformer的设计已展现出更优性能和可扩展性。然而，通过“分块化”对输入数据进行标记的Transformer架构，由于自注意力操作与标记长度相关的二次特性，面临视觉保真度与计算复杂度的权衡。较大的分块尺寸虽能提高注意力计算效率，但难以捕捉细粒度视觉细节，导致图像失真。为解决这一挑战，我们提出用多分辨率网络（DiMR）增强扩散模型，该框架跨多分辨率细化特征，从低到高分辨率逐步提升细节。此外，我们引入时间依赖层归一化（TD-LN），这是一种参数高效的方法，将时间依赖参数融入层归一化以注入时间信息并实现更优性能。我们的方法在类别条件ImageNet生成基准上的有效性得到验证，DiMR-XL变体优于现有扩散模型，在ImageNet 256×256和512×512上创下1.70和2.89的最新SOTA FID分数。项目页面：https://qihao067.github.io/projects/DiMR

英文摘要：

This paper presents innovative enhancements to diffusion models by integrating a novel multi-resolution network and time-dependent layer normalization. Diffusion models have gained prominence for their effectiveness in high-fidelity image generation. While conventional approaches rely on convolutional U-Net architectures, recent Transformer-based designs have demonstrated superior performance and scalability. However, Transformer architectures, which tokenize input data (via "patchification"), face a trade-off between visual fidelity and computational complexity due to the quadratic nature of self-attention operations concerning token length. While larger patch sizes enable attention computation efficiency, they struggle to capture fine-grained visual details, leading to image distortions. To address this challenge, we propose augmenting the Diffusion model with the Multi-Resolution network (DiMR), a framework that refines features across multiple resolutions, progressively enhancing detail from low to high resolution. Additionally, we introduce Time-Dependent Layer Normalization (TD-LN), a parameter-efficient approach that incorporates time-dependent parameters into layer normalization to inject time information and achieve superior performance. Our method's efficacy is demonstrated on the class-conditional ImageNet generation benchmark, where DiMR-XL variants outperform prior diffusion models, setting new state-of-the-art FID scores of 1.70 on ImageNet 256 x 256 and 2.89 on ImageNet 512 x 512. Project page: https://qihao067.github.io/projects/DiMR

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。