Generative diffusion models and many stochastic models in science and engineering naturally live in infinite dimensions before discretisation. To incorporate observed data for statistical and learning tasks, one needs to condition on observations. While recent work has treated conditioning linear processes in infinite dimensions, conditioning non-linear processes in infinite dimensions has not been explored. This paper conditions function valued stochastic processes without prior discretisation. To do so, we use an infinite-dimensional version of Girsanov's theorem to condition a function-valued stochastic process, leading to a stochastic differential equation (SDE) for the conditioned process involving the score. We apply this technique to do time series analysis for shapes of organisms in evolutionary biology, where we discretise via the Fourier basis and then learn the coefficients of the score function with score matching methods.
Conditional image generation has paved the way for several breakthroughs in image editing, generating stock photos and 3-D object generation. This continues to be a significant area of interest with the rise of new state-of-the-art methods that are based on diffusion models. However, diffusion models provide very little control over the generated image, which led to subsequent works exploring techniques like classifier guidance, that provides a way to trade off diversity with fidelity. In this work, we explore techniques to condition diffusion models with carefully crafted input noise artifact…
Diffusion Model Conditioning on Gaussian Mixture Model and Negative Gaussian Mixture Gradient
University of Macau（澳门大学）；Great Bay University（大湾区大学）；Great Bay Institute for Advanced Study（大湾区高等研究院）；University of Science and Technology of China（中国科学技术大学）
Generating visual layouts is an essential ingredient of graphic design. The ability to condition layout generation on a partial subset of component attributes is critical to real-world applications that involve user interaction. Recently, diffusion models have demonstrated high-quality generative performances in various domains. However, it is unclear how to apply diffusion models to the natural representation of layouts which consists of a mix of discrete (class) and continuous (location, size) attributes. To address the conditioning layout generation problem, we introduce DLT, a joint discre…
We present Bit Diffusion: a simple and generic approach for generating discrete data with continuous state and continuous time diffusion models. The main idea behind our approach is to first represent the discrete data as binary bits, and then train a continuous diffusion model to model these bits as real numbers which we call analog bits. To generate samples, the model first generates the analog bits, which are then thresholded to obtain the bits that represent the discrete variables. We further propose two simple techniques, namely Self-Conditioning and Asymmetric Time Intervals, which lead …
Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution
Virginia Tech（弗吉尼亚理工大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；University of California, Irvine（加利福尼亚大学欧文分校）；Tulane University（杜兰大学）；Battelle（巴特尔公司）；Duke University（杜克大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；The Ohio State University（俄亥俄州立大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation
Institute for AI, University of Stuttgart（斯图加特大学人工智能研究所）；IMPRS-IS（国际马克斯·普朗克智能系统研究学院）；Institute of Theoretical Chemistry, University of Stuttgart（斯图加特大学理论化学研究所）
We show that cascaded diffusion models are capable of generating high fidelity images on the class-conditional ImageNet generation benchmark, without any assistance from auxiliary image classifiers to boost sample quality. A cascaded diffusion model comprises a pipeline of multiple diffusion models that generate images of increasing resolution, beginning with a standard diffusion model at the lowest resolution, followed by one or more super-resolution diffusion models that successively upsample the image and add higher resolution details. We find that the sample quality of a cascading pipeline…
We present 3DiM, a diffusion model for 3D novel view synthesis, which is able to translate a single input view into consistent and sharp completions across many views. The core component of 3DiM is a pose-conditional image-to-image diffusion model, which takes a source view and its pose as inputs, and generates a novel view for a target pose as output. 3DiM can generate multiple views that are 3D consistent using a novel technique called stochastic conditioning. The output views are generated autoregressively, and during the generation of each novel view, one selects a random conditioning view…
FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version
FPT Software AI Center, FPT Corporation（FPT软件人工智能中心，FPT公司）；Department of Computer Science, Aalborg University（奥尔堡大学计算机科学系）；Quantum AI and Cyber Security Institute, FPT Corporation（FPT公司量子人工智能与网络安全研究所）
TimeAutoDiff: A Unified Framework for Generation, Imputation, Forecasting, and Time-Varying Metadata Conditioning of Heterogeneous Time Series Tabular Data
