Continuous normalizing flows (CNFs) are an attractive generative modeling technique, but they have been held back by limitations in their simulation-based maximum likelihood training. We introduce the generalized conditional flow matching (CFM) technique, a family of simulation-free training objectives for CNFs. CFM features a stable regression objective like that used to train the stochastic flow in diffusion models but enjoys the efficient inference of deterministic flow models. In contrast to both diffusion models and prior CNF training algorithms, CFM does not require the source distribution to be Gaussian or require evaluation of its density. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference, as evaluated in our experiments. Furthermore, we show that when the true OT plan is available, our OT-CFM method approximates dynamic OT. Training CNFs with CFM improves results on a variety of conditional and unconditional generation tasks, such as inferring single cell dynamics, unsupervised image translation, and Schrödinger bridge inference.
Univ. Bretagne-Sud（南布列塔尼大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；IRISA（法国雷恩信息与自动化系统研究所）；School of Mathematics, The University of Edinburgh（爱丁堡大学数学学院）；The Maxwell Institute for the Mathematical Sciences（麦克斯韦数学科学研究所）；Univ Rennes（雷恩大学）；IRMAR（雷恩数学研究所）；École Polytechnique（巴黎综合理工学院）；Univ Lyon（里昂大学）；ENS de Lyon（里昂高等师范学院）；UCB Lyon 1（里昂第一大学克洛德·贝尔纳大学）
Optimal transport distances have become a classic tool to compare probability distributions and have found many applications in machine learning. Yet, despite recent algorithmic developments, their complexity prevents their direct use on large scale datasets. To overcome this challenge, a common workaround is to compute these distances on minibatches i.e. to average the outcome of several smaller optimal transport problems. We propose in this paper an extended analysis of this practice, which effects were previously studied in restricted cases. We first consider a large variety of Optimal Tran…
Multisample Flow Matching: Straightening Flows with Minibatch Couplings
Center for Data Science, NYU（纽约大学数据科学中心）；Meta AI (FAIR)（Meta人工智能（基础人工智能研究团队））；Weizmann Institute of Science（魏茨曼科学研究所）；Courant Institute of Mathematical Sciences, NYU（纽约大学柯朗数学科学研究所）
Google Brain（谷歌大脑）；Institute for Quantum Computing, University of Waterloo（滑铁卢大学量子计算研究所）；UC Berkeley（加州大学伯克利分校）；Stanford University（斯坦福大学）；Princeton University（普林斯顿大学）；University of Waterloo（滑铁卢大学）
The dual tasks of quantum Hamiltonian learning and quantum Gibbs sampling are relevant to many important problems in physics and chemistry. In the low temperature regime, algorithms for these tasks often suffer from intractabilities, for example from poor sample- or time-complexity. With the aim of addressing such intractabilities, we introduce a generalization of quantum natural gradient descent to parameterized mixed states, as well as provide a robust first-order approximating algorithm, Quantum-Probabilistic Mirror Descent. We prove data sample efficiency for the dual tasks using tools fro…
The City College, The City University of New York（纽约城市大学城市学院）；The Graduate Center, The City University of New York（纽约城市大学研究生中心）；Borough of Manhattan Community College, The City University of New York（纽约城市大学曼哈顿社区学院）
In this work, we generalize semi-supervised generative adversarial networks (GANs) from classification problems to regression problems. In the last few years, the importance of improving the training of neural networks using semi-supervised training has been demonstrated for classification problems. We present a novel loss function, called feature contrasting, resulting in a discriminator which can distinguish between fake and real data based on feature statistics. This method avoids potential biases and limitations of alternative approaches. The generalization of semi-supervised GANs to the r…
Developing deep generative models that flexibly incorporate diverse measures of probability distance is an important area of research. Here we develop an unified mathematical framework of f-divergence generative model, f-GM, that incorporates both VAE and f-GAN, and enables tractable learning with general f-divergences. f-GM allows the experimenter to flexibly design the f-divergence function without changing the structure of the networks or the learning procedure. f-GM jointly models three components: a generator, a inference network and a density estimator. Therefore it simultaneously enable…
University of Toronto（多伦多大学）；University of Bremen（不来梅大学）；Vector Institute（向量研究所）
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforcing invertibility. However, prior work trained invertible residual networks for density estimation by relying on biased log-density estimates whose bias increased with the network's expressiveness. We give a tractable unbiased estimate of the log density using a "Russian roulette"…
Flow Matching from Viewpoint of Proximal Operators
The Institute of Statistical Mathematics, Japan（日本统计数学研究所）；RIKEN AIP, Japan（日本理化学研究所AIP）；Tohoku University, Japan（日本东北大学）；East China Normal University, China（中国华东师范大学）；University of Chicago, USA（美国芝加哥大学）
Switched Flow Matching: Eliminating Singularities via Switching ODEs
Research Institute of Intelligent Complex Systems, Fudan University（复旦大学智能复杂系统研究院）；Institutes of Brain Science, Fudan University（复旦大学脑科学研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Optimal Flow Matching: Learning Straight Trajectories in Just One Step
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Innopolis University（因诺波利斯大学）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Steklov Mathematical Institute of RAS（俄罗斯科学院斯捷克洛夫数学研究所）；Artificial Intelligence Research Institute（人工智能研究院）
Straighter Flow Matching via a Diffusion-Based Coupling Prior
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Department of Electrical and Systems Engineering University of Pennsylvania（宾夕法尼亚大学电气工程与系统工程系）
While deep learning has resulted in major breakthroughs in many application domains, the frameworks commonly used in deep learning remain fragile to artificially-crafted and imperceptible changes in the data. In response to this fragility, adversarial training has emerged as a principled approach for enhancing the robustness of deep learning with respect to norm-bounded perturbations. However, there are other sources of fragility for deep learning that are arguably more common and less thoroughly studied. Indeed, natural variation such as lighting or weather conditions can significantly degrad…
Zero-shot learning (ZSL) has been shown to be a promising approach to generalizing a model to categories unseen during training by leveraging class attributes, but challenges still remain. Recently, methods using generative models to combat bias towards classes seen during training have pushed the state of the art of ZSL, but these generative models can be slow or computationally expensive to train. Additionally, while many previous ZSL methods assume a one-time adaptation to unseen classes, in reality, the world is always changing, necessitating a constant adjustment for deployed models. Mode…
ETH Zurich（苏黎世联邦理工学院）；University Hospital of Zurich（苏黎世大学医院）
Supervised deep learning methods for segmentation require large amounts of labelled training data, without which they are prone to overfitting, not generalizing well to unseen images. In practice, obtaining a large number of annotations from clinical experts is expensive and time-consuming. One way to address scarcity of annotated examples is data augmentation using random spatial and intensity transformations. Recently, it has been proposed to use generative models to synthesize realistic training examples, complementing the random augmentation. So far, these methods have yielded limited gain…
An implicit goal in works on deep generative models is that such models should be able to generate novel examples that were not previously seen in the training data. In this paper, we investigate to what extent this property holds for widely employed variational autoencoder (VAE) architectures. VAEs maximize a lower bound on the log marginal likelihood, which implies that they will in principle overfit the training data when provided with a sufficiently expressive decoder. In the limit of an infinite capacity decoder, the optimal generative model is a uniform mixture over the training data. Mo…
