Currently, applying diffusion models in pixel space of high resolution images is difficult. Instead, existing approaches focus on diffusion in lower dimensional spaces (latent diffusion), or have multiple super-resolution levels of generation referred to as cascades. The downside is that these approaches add additional complexity to the diffusion framework.
This paper aims to improve denoising diffusion for high resolution images while keeping the model as simple as possible. The paper is centered around the research question: How can one train a standard denoising diffusion models on high resolution images, and still obtain performance comparable to these alternate approaches?
The four main findings are: 1) the noise schedule should be adjusted for high resolution images, 2) It is sufficient to scale only a particular part of the architecture, 3) dropout should be added at specific locations in the architecture, and 4) downsampling is an effective strategy to avoid high resolution feature maps. Combining these simple yet effective techniques, we achieve state-of-the-art on image generation among diffusion models without sampling modifiers on ImageNet.
The University of Tokyo（东京大学）；Preferred Networks Inc.（Preferred Networks公司）
Diffusion models have shown great results in image generation and in image editing. However, current approaches are limited to low resolutions due to the computational cost of training diffusion models for high-resolution generation. We propose an approach that uses a pre-trained low-resolution diffusion model to edit images in the megapixel range. We first use Blended Diffusion to edit the image at a low resolution, and then upscale it in multiple stages, using a super-resolution model and Blended Diffusion. Using our approach, we achieve higher visual fidelity than by only applying off the s…
Korea Advanced Institute of Science and Technology(KAIST)（韩国科学技术院（KAIST））
Recently, diffusion model have demonstrated impressive image generation performances, and have been extensively studied in various computer vision tasks. Unfortunately, training and evaluating diffusion models consume a lot of time and computational resources. To address this problem, here we present a novel pyramidal diffusion model that can generate high resolution images starting from much coarser resolution images using a {\em single} score function trained with a positional embedding. This enables a neural network to be much lighter and also enables time-efficient image generation without…
From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task
School of Advanced Interdisciplinary Studies, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）
We show that cascaded diffusion models are capable of generating high fidelity images on the class-conditional ImageNet generation benchmark, without any assistance from auxiliary image classifiers to boost sample quality. A cascaded diffusion model comprises a pipeline of multiple diffusion models that generate images of increasing resolution, beginning with a standard diffusion model at the lowest resolution, followed by one or more super-resolution diffusion models that successively upsample the image and add higher resolution details. We find that the sample quality of a cascading pipeline…
Perceptually Regularized Diffusion Model for Image Super-Resolution
University of North Carolina（北卡罗来纳大学）；Texas Tech University（德克萨斯理工大学）；Case Western Reserve University（凯斯西储大学）；University of Houston（休斯顿大学）；University of Kentucky（肯塔基大学）
Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models
State Key Laboratory of Complex and Critical Software Environment（复杂与关键软件环境国家重点实验室）；Beihang University（北京航空航天大学）；School of Computer Science and Engineering（计算机科学与工程学院）
University of Hawaii Manoa（夏威夷大学马诺阿分校）；Nimbus AI LLC（Nimbus人工智能有限责任公司）
Forecasting future weather and climate is inherently difficult. Machine learning offers new approaches to increase the accuracy and computational efficiency of forecasts, but current methods are unable to accurately model uncertainty in high-dimensional predictions. Score-based diffusion models offer a new approach to modeling probability distributions over many dependent variables, and in this work, we demonstrate how they provide probabilistic forecasts of weather and climate variables at unprecedented resolution, speed, and accuracy. We apply the technique to day-ahead solar irradiance fore…
Physics-informed neural networks (NN) are an emerging technique to improve spatial resolution and enforce physical consistency of data from physics models or satellite observations. A super-resolution (SR) technique is explored to reconstruct high-resolution images ($4\times$) from lower resolution images in an advection-diffusion model of atmospheric pollution plumes. SR performance is generally increased when the advection-diffusion equation constrains the NN in addition to conventional pixel-based constraints. The ability of SR techniques to also reconstruct missing data is investigated by …
Diffusion models have shown promising results on single-image super-resolution and other image- to-image translation tasks. Despite this success, they have not outperformed state-of-the-art GAN models on the more challenging blind super-resolution task, where the input images are out of distribution, with unknown degradations. This paper introduces SR3+, a diffusion-based model for blind super-resolution, establishing a new state-of-the-art. To this end, we advocate self-supervised training with a combination of composite, parameterized degradations for self-supervised training, and noise-cond…
Bridging Diffusion Models and 3D Representations: A 3D Consistent Super-Resolution Framework
University of Maryland, College Park（马里兰大学 College Park 分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
