arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.07261cs.LGstat.ML
收藏

在哪里扩散,如何扩散,以及如何返回:多变量扩散的自动学习

Where to Diffuse, How to Diffuse, and How to Get Back: Automated Learning for Multivariate Diffusions

Courant Institute of Mathematical Sciences(库朗数学科学研究所);New York University(纽约大学);Center for Data Science(数据科学中心)

Raghav Singhal, Mark Goldstein, Rajesh Ranganath

摘要:

基于扩散的生成模型(DBGM)将数据扰动到目标噪声分布,并反转该过程以生成样本。噪声处理或推理扩散处理的选择影响似然性和样本质量。例如,使用辅助变量扩展推理过程可以提高样本质量。虽然有许多这样的多元扩散探索,每个新的需要大量的模型特定的分析,阻碍快速原型和评估。本文研究了多元扩散模型。对于任何数量的辅助变量,我们提供了一个最大化MDM可能性下限的方法,而不需要任何特定于模型的分析。然后,我们演示了如何为指定的目标噪声分布的扩散参数化,这两点一起使推理扩散过程的优化。优化扩散将简单的实验从几个众所周知的过程扩展到对所有线性扩散的自动搜索。为了证明这些想法,我们引入了两个新的特定扩散,并在MNIST,CIFAR10和ImageNet32数据集上学习扩散过程。我们发现,对于给定的数据集和模型架构,学习的MDM匹配或超过相对于扩散的固定选择的每维位数(BPD)。

英文摘要:

Diffusion-based generative models (DBGMs) perturb data to a target noise distribution and reverse this process to generate samples. The choice of noising process, or inference diffusion process, affects both likelihoods and sample quality. For example, extending the inference process with auxiliary variables leads to improved sample quality. While there are many such multivariate diffusions to explore, each new one requires significant model-specific analysis, hindering rapid prototyping and evaluation. In this work, we study Multivariate Diffusion Models (MDMs). For any number of auxiliary variables, we provide a recipe for maximizing a lower-bound on the MDMs likelihood without requiring any model-specific analysis. We then demonstrate how to parameterize the diffusion for a specified target noise distribution; these two points together enable optimizing the inference diffusion process. Optimizing the diffusion expands easy experimentation from just a few well-known processes to an automatic search over all linear diffusions. To demonstrate these ideas, we introduce two new specific diffusions as well as learn a diffusion process on the MNIST, CIFAR10, and ImageNet32 datasets. We show learned MDMs match or surpass bits-per-dims (BPDs) relative to fixed choices of diffusions for a given dataset and model architecture.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑