In text-to-speech (TTS) synthesis, diffusion models have achieved promising generation quality. However, because of the pre-defined data-to-noise diffusion process, their prior distribution is restricted to a noisy representation, which provides little information of the generation target. In this work, we present a novel TTS system, Bridge-TTS, making the first attempt to substitute the noisy Gaussian prior in established diffusion-based TTS methods with a clean and deterministic one, which provides strong structural information of the target. Specifically, we leverage the latent representation obtained from text input as our prior, and build a fully tractable Schrodinger bridge between it and the ground-truth mel-spectrogram, leading to a data-to-data process. Moreover, the tractability and flexibility of our formulation allow us to empirically study the design spaces such as noise schedules, as well as to develop stochastic and deterministic samplers. Experimental results on the LJ-Speech dataset illustrate the effectiveness of our method in terms of both synthesis quality and sampling efficiency, significantly outperforming our diffusion counterpart Grad-TTS in 50-step/1000-step synthesis and strong fast TTS models in few-step scenarios. Project page: https://bridge-tts.github.io/
The Schrödinger bridge problem (SBP) finds the most likely stochastic evolution between two probability distributions given a prior stochastic evolution. As well as applications in the natural sciences, problems of this kind have important applications in machine learning such as dataset alignment and hypothesis testing. Whilst the theory behind this problem is relatively mature, scalable numerical recipes to estimate the Schrödinger bridge remain an active area of research. We prove an equivalence between the SBP and maximum likelihood estimation enabling direct application of successful mach…
The Hong Kong University of Science and Technology（香港科技大学）；Wuhan University（武汉大学）；WeBank Co., Ltd.（微众银行股份有限公司）
We propose to learn a generative model via entropy interpolation with a Schrödinger Bridge. The generative learning task can be formulated as interpolating between a reference distribution and a target distribution based on the Kullback-Leibler divergence. At the population level, this entropy interpolation is characterized via an SDE on $[0,1]$ with a time-varying drift term. At the sample level, we derive our Schrödinger Bridge algorithm by plugging the drift term estimated by a deep score estimator and a deep density ratio estimator into the Euler-Maruyama method. Under some mild smoothness…
University of Oxford（牛津大学）；ENS（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL University（巴黎文理研究大学）
Score-based generative models exhibit state of the art performance on density estimation and generative modeling tasks. These models typically assume that the data geometry is flat, yet recent extensions have been developed to synthesize data living on Riemannian manifolds. Existing methods to accelerate sampling of diffusion models are typically not applicable in the Riemannian setting and Riemannian score-based methods have not yet been adapted to the important task of interpolation of datasets. To overcome these issues, we introduce \emph{Riemannian Diffusion Schrödinger Bridge}. Our propos…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Artificial Intelligence Research Institute（人工智能研究院）
Schrödinger Bridges (SB) have recently gained the attention of the ML community as a promising extension of classic diffusion models which is also interconnected to the Entropic Optimal Transport (EOT). Recent solvers for SB exploit the pervasive bridge matching procedures. Such procedures aim to recover a stochastic process transporting the mass between distributions given only a transport plan between them. In particular, given the EOT plan, these procedures can be adapted to solve SB. This fact is heavily exploited by recent works giving rise to matching-based SB solvers. The cornerstone he…
Probabilistic Super-Resolution for Urban Micrometeorology via a Schrödinger Bridge
Research Institute for Value-Added-Information Generation（价值加成信息研究所）；Japan Agency for Marine-Earth Science and Technology（日本海洋地球科学技术机构）；Supercomputing Research Center（超级计算研究中心）；Institute of Integrated Research（整合研究所）；Institute of Science Tokyo（东京科学研究所）
Compared to the existing function-based models in deep generative modeling, the recently proposed diffusion models have achieved outstanding performance with a stochastic-process-based approach. But a long sampling time is required for this approach due to many timesteps for discretization. Schrödinger bridge (SB)-based models attempt to tackle this problem by training bidirectional stochastic processes between distributions. However, they still have a slow sampling speed compared to generative models such as generative adversarial networks. And due to the training of the bidirectional stochas…
Exact Solutions to the Quantum Schrödinger Bridge Problem
Machine Learning Research, Pfizer Bristol-Myers Squibb, ML Research Cambridge（机器学习研究部，辉瑞百灵益泰，ML研究剑桥）；Pfizer Worldwide Research Development and Medical（辉瑞全球研发与医疗）；Machine Learning Research group, Cambridge, MA, USA（机器学习研究组，剑桥，马萨诸塞州，美国）
Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers
NVIDIA Corporation（NVIDIA公司）；University of Oxford（牛津大学）；Donders Institute for Brain, Cognition, and Behaviour（大脑与行为研究所）；AITHYRA, Research Institute for Biomedical AI（生物医学人工智能研究所）
University of Oxford（牛津大学）；PSL University（巴黎文理研究大学）；ENS（巴黎高等师范学院）
Denoising diffusion models have recently emerged as a powerful class of generative models. They provide state-of-the-art results, not only for unconditional simulation, but also when used to solve conditional simulation problems arising in a wide range of inverse problems. A limitation of these models is that they are computationally intensive at generation time as they require simulating a diffusion process over a long time horizon. When performing unconditional simulation, a Schrödinger bridge formulation of generative modeling leads to a theoretically grounded algorithm shortening generatio…
Optimal Multimarginal Schrödinger Bridge: Minimum Spanning Tree over Measure-valued Vertices
Department of Applied Mathematics, University of California Santa Cruz（应用数学系，加州大学圣克鲁兹分校）；Department of Aerospace Engineering, Iowa State University（航空航天工程系，爱荷华州立大学）
