Diffusion probabilistic models (DPMs) have demonstrated a very promising ability in high-resolution image synthesis. However, sampling from a pre-trained DPM is time-consuming due to the multiple evaluations of the denoising network, making it more and more important to accelerate the sampling of DPMs. Despite recent progress in designing fast samplers, existing methods still cannot generate satisfying images in many applications where fewer steps (e.g., $<$10) are favored. In this paper, we develop a unified corrector (UniC) that can be applied after any existing DPM sampler to increase the order of accuracy without extra model evaluations, and derive a unified predictor (UniP) that supports arbitrary order as a byproduct. Combining UniP and UniC, we propose a unified predictor-corrector framework called UniPC for the fast sampling of DPMs, which has a unified analytical form for any order and can significantly improve the sampling quality over previous methods, especially in extremely few steps. We evaluate our methods through extensive experiments including both unconditional and conditional sampling using pixel-space and latent-space DPMs. Our UniPC can achieve 3.87 FID on CIFAR10 (unconditional) and 7.51 FID on ImageNet 256$\times$256 (conditional) with only 10 function evaluations. Code is available at https://github.com/wl-zhao/UniPC.
University of Michigan（密歇根大学）；Princeton University（普林斯顿大学）
Time-varying stochastic optimization problems frequently arise in machine learning practice (e.g. gradual domain shift, object tracking, strategic classification). Although most problems are solved in discrete time, the underlying process is often continuous in nature. We exploit this underlying continuity by developing predictor-corrector algorithms for time-varying stochastic optimizations. We provide error bounds for the iterates, both in presence of pure and noisy access to the queries from the relevant derivatives of the loss function. Furthermore, we show (theoretically and empirically i…
Using insight from numerical approximation of ODEs and the problem formulation and solution methodology of TD learning through a Galerkin relaxation, I propose a new class of TD learning algorithms. After applying the improved numerical methods, the parameter being approximated has a guaranteed order of magnitude reduction in the Taylor Series error of the solution to the ODE for the parameter $θ(t)$ that is used in constructing the linearly parameterized value function. Predictor-Corrector Temporal Difference (PCTD) is what I call the translated discrete time Reinforcement Learning(RL) algori…
Neural CDEs as Correctors for Learned Time Series Models
Department of Mechanical Engineering（机械工程系）；Iowa State University（爱荷华州立大学）；Sibley School of Mechanical and Aerospace Engineering（机械与航空航天工程学院）；Cornell University（康奈尔大学）
Lehigh University（理海大学）；The Ohio State University（俄亥俄州立大学）；Drake University（德雷克大学）；Cornell University（康奈尔大学）
Multiple-objective optimization (MOO) aims to simultaneously optimize multiple conflicting objectives and has found important applications in machine learning, such as minimizing classification loss and discrepancy in treating different populations for fairness. At optimality, further optimizing one objective will necessarily harm at least another objective, and decision-makers need to comprehensively explore multiple optima (called Pareto front) to pinpoint one final solution. We address the efficiency of finding the Pareto front. First, finding the front from scratch using stochastic multi-g…
We present a predictor-corrector framework, called PicCoLO, that can transform a first-order model-free reinforcement or imitation learning algorithm into a new hybrid method that leverages predictive models to accelerate policy learning. The new "PicCoLOed" algorithm optimizes a policy by recursively repeating two steps: In the Prediction Step, the learner uses a model to predict the unseen future gradient and then applies the predicted estimate to update the policy; in the Correction Step, the learner runs the updated policy in the environment, receives the true gradient, and then corrects t…
Department of Statistics, University of Oxford（1 统计系，牛津大学）；Inria, CNRS, École Normale Supérieure, PSL Research University（2 Inria，法国国家科学研究中心，巴黎高等师范学院，PSL研究大学）；Department of Mathematical Sciences, University of Copenhagen（3 数学科学系，哥本哈根大学）
Diffusion-corrected Autoregressive Fourier Neural Operator for Droplet Evolution Prediction
New Jersey Institute of Technology（新泽西理工学院）；University of Georgia（佐治亚大学）；University at Buffalo（纽约州立大学布法罗分校）；Hanyang University（汉阳大学）；Northeastern University（东北大学）
Synthesizing PET images from High-field and Ultra-high-field MR images Using Joint Diffusion Attention Model
Inner Mongolia University（内蒙古大学）；Inner Mongolia Medical University（内蒙古医科大学）；SIAT, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Inner Mongolia Medical University Affiliated Hospital（内蒙古医科大学附属医院）；Shenzhen Children’s Hospital（深圳市儿童医院）
