arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.01709cs.CVcs.AI
收藏

上采样引导:无需训练即可扩展扩散模型

Upsample Guidance: Scale Up Diffusion Models without Training

Seoul National University(首尔大学);Korea Institute for Advanced Study(韩国高等研究院)

Juno Hwang, Yong-Hyun Park, Junghyo Jo

AI总结:

本文提出上采样引导技术,仅通过在采样过程中添加一个项,即可让预训练扩散模型无需训练或外部模型直接生成更高分辨率图像,并验证其适用于像素空间、潜在空间和视频扩散模型。

摘要:

扩散模型已在图像、视频和音频等多种生成任务中展现出卓越性能,但在直接生成高分辨率样本方面仍面临困难。此前针对该问题的解决方案包括修改网络架构、进行额外训练或将采样过程划分为多个阶段。这些方法的局限性在于无法直接原样利用预训练模型,需要额外工作。本文提出上采样引导技术,通过在采样过程中仅添加一个项,即可使预训练扩散模型(如 $512^2$)生成更高分辨率图像(如 $1536^2$)。值得注意的是,该技术无需任何额外训练,也不依赖外部模型。我们证明上采样引导可应用于多种模型,包括像素空间、潜在空间和视频扩散模型。我们还观察到,合理选择引导尺度可以提升图像质量、保真度和提示词对齐程度。

英文摘要:

Diffusion models have demonstrated superior performance across various generative tasks including images, videos, and audio. However, they encounter difficulties in directly generating high-resolution samples. Previously proposed solutions to this issue involve modifying the architecture, further training, or partitioning the sampling process into multiple stages. These methods have the limitation of not being able to directly utilize pre-trained models as-is, requiring additional work. In this paper, we introduce upsample guidance, a technique that adapts pretrained diffusion model (e.g., $512^2$) to generate higher-resolution images (e.g., $1536^2$) by adding only a single term in the sampling process. Remarkably, this technique does not necessitate any additional training or relying on external models. We demonstrate that upsample guidance can be applied to various models, such as pixel-space, latent space, and video diffusion models. We also observed that the proper selection of guidance scale can improve image quality, fidelity, and prompt alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑