Score-based diffusion models (SBDM) have recently emerged as state-of-the-art approaches for image generation. Existing SBDMs are typically formulated in a finite-dimensional setting, where images are considered as tensors of finite size. This paper develops SBDMs in the infinite-dimensional setting, that is, we model the training data as functions supported on a rectangular domain. In addition to the quest for generating images at ever-higher resolutions, our primary motivation is to create a well-posed infinite-dimensional learning problem that we can discretize consistently on multiple resolution levels. We thereby intend to obtain diffusion models that generalize across different resolution levels and improve the efficiency of the training process. We demonstrate how to overcome two shortcomings of current SBDM approaches in the infinite-dimensional setting. First, we modify the forward process using trace class operators to ensure that the latent distribution is well-defined in the infinite-dimensional setting and derive the reverse processes for finite-dimensional approximations. Second, we illustrate that approximating the score function with an operator network is beneficial for multilevel training. After deriving the convergence of the discretization and the approximation of multilevel training, we demonstrate some practical benefits of our infinite-dimensional SBDM approach on a synthetic Gaussian mixture example, the MNIST dataset, and a dataset generated from a nonlinear 2D reaction-diffusion equation.
An Unconditional Representation of the Conditional Score in Infinite-Dimensional Linear Inverse Problems
School of Engineering Science(工程科学学院);Lappeenranta-Lahti University of Technology(拉普兰塔-拉赫蒂技术大学);Vienna University of Technology (TU Wien)(维也纳技术大学);Department of Mathematics and Statistics(数学与统计学系);University of Helsinki(赫尔辛基大学);Department of Computational and Applied Mathematics(计算与应用数学系);Rice University(里士满大学)
Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection
National University of Singapore(新加坡国立大学);Singapore University of Technology and Design(新加坡科技设计大学);University of Science(科学大学);Vietnam National University, Ho Chi Minh City(胡志明市越南国家大学)
Multilevel Determinants of Overweight and Obesity Among U.S. Children Aged 10-17: Comparative Evaluation of Statistical and Machine Learning Approaches Using the 2021 National Survey of Children's Health
Mila(米拉(学习算法研究所));Université de Montréal(蒙特利尔大学);ETH Zürich(苏黎世联邦理工学院);Helmholtz AI(亥姆霍兹人工智能研究所);Technical University of Munich(慕尼黑工业大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)
Set Invariance with Probability One for Controlled Diffusion: Score-based Approach
Department of Aerospace Engineering, Iowa State University(航空航天工程系,爱荷华州立大学);Department of Applied Mathematics, University of California, Santa Cruz(应用数学系,加州大学圣克鲁兹分校);Department of Electrical Engineering and Computer Sciences, University of California, Berkeley(电气工程与计算机科学系,加州大学伯克利分校)