In this work we analyze regularized optimal transport problems in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, the aim is to find a transport plan, which is another Radon measure on the product of the sets, that has these two measures as marginals and minimizes the sum of a certain linear cost function and a regularization term. We focus on regularization terms where a Young's function applied to the (density of the) transport plan is integrated against a product measure. This forces the transport plan to belong to a certain Orlicz space. The predual problem is derived and proofs for strong duality and existence of primal solutions of the regularized problem are presented. Existence of (pre-)dual solutions is shown for the special case of $L^p$ regularization for $p\geq 2$. Moreover, two results regarding $Γ$-convergence are stated: The first is concerned with marginals that do not lie in the appropriate Orlicz space and guarantees $Γ$-convergence to the original Kantorovich problem, when smoothing the marginals. The second results gives convergence of a regularized and discretized problem to the unregularized, continuous problem.
We investigate the continuous optimal transport problem in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider regularization of the problem with so-called Young's functions, which forces the optimal transport plan to be a function in the corresponding Orlicz space rather than a Radon measure. We derive the predual problem and show strong duality and existence of primal solutions to…
We analyze continuous optimal transport problems in the so-called Kantorovich form, where we seek a transport plan between two marginals that are probability measures on compact subsets of Euclidean space. We consider the case of regularization with the negative entropy with respect to the Lebesgue measure, which has attracted attention because it can be solved by the very simple Sinkhorn algorithm. We first analyze the regularized problem in the context of classical Fenchel duality and derive a strong duality result for a predual problem in the space of continuous functions. However, this pro…
Optimal transport has recently proved to be a useful tool in various machine learning applications needing comparisons of probability measures. Among these, applications of distributionally robust optimization naturally involve Wasserstein distances in their models of uncertainty, capturing data shifts or worst-case scenarios. Inspired by the success of the regularization of Wasserstein distances in optimal transport, we study in this paper the regularization of Wasserstein distributionally robust optimization. First, we derive a general strong duality result of regularized Wasserstein distrib…
In 2012, Pflug and Pichler proved, under regularity assumptions, that the value function in Multistage Stochastic Programming (MSP) is Lipschitz continuous w.r.t. the Nested Distance, which is a distance between scenario trees (or discrete time stochastic processes with finite support). The Nested Distance is a refinement of the Wasserstein distance to account for proximity of the filtrations of discrete time stochastic processes. The computation of the Nested Distance between two scenario trees amounts to the computation of an exponential (in the horizon $T$) number of optimal transport probl…
We introduce a new class of convex-regularized Optimal Transport losses, which generalizes the classical Entropy-regularization of Optimal Transport and Sinkhorn divergences, and propose a generalized Sinkhorn algorithm. Our framework unifies many regularizations and numerical methods previously appeared in the literature. We show the existence of the maximizer for the dual problem, complementary slackness conditions, providing a complete characterization of solutions for such class of variational problems. As a consequence, we study structural properties of these losses, including continuity,…
We propose to tackle the problem of understanding the effect of regularization in Sinkhorn algotihms. In the case of Gaussian distributions we provide a closed form for the regularized optimal transport which enables to provide a better understanding of the effect of the regularization from a statistical framework.
The optimal transport problem has recently developed into a powerful framework for various applications in estimation and control. Many of the recent advances in the theory and application of optimal transport are based on regularizing the problem with an entropy term, which connects it to the Schrödinger bridge problem and thus to stochastic optimal control. Moreover, the entropy regularization makes the otherwise computationally demanding optimal transport problem feasible even for large scale settings. This has lead to an accelerated development of optimal transport based methods in a broad…
We consider a multimarginal optimal transport, which includes as a particular case the Wasserstein barycenter problem. In this problem one has to find an optimal coupling between $m$ probability measures, which amounts to finding a tensor of the order $m$. We propose an accelerated method based on accelerated alternating minimization and estimate its complexity to find the approximate solution to the problem. We use entropic regularization with sufficiently small regularization parameter and apply accelerated alternating minimization to the dual problem. A novel primal-dual analysis is used to…
We study MinMax solution methods for a general class of optimization problems related to (and including) optimal transport. Theoretically, the focus is on fitting a large class of problems into a single MinMax framework and generalizing regularization techniques known from classical optimal transport. We show that regularization techniques justify the utilization of neural networks to solve such problems by proving approximation theorems and illustrating fundamental issues if no regularization is used. We further study the relation to the literature on generative adversarial nets, and analyze …
The optimal transport (OT) problem is a classical optimization problem having the form of linear programming. Machine learning applications put forward new computational challenges in its solution. In particular, the OT problem defines a distance between real-world objects such as images, videos, texts, etc., modeled as probability distributions. In this case, the large dimension of the corresponding optimization problem does not allow applying classical methods such as network simplex or interior-point methods. This challenge was overcome by introducing entropic regularization and using the e…
Stanford University（斯坦福大学）；Chinese University of Hong Kong（香港中文大学）
Distributionally robust optimization has been shown to offer a principled way to regularize learning models. In this paper, we find that Tikhonov regularization is distributionally robust in an optimal transport sense (i.e., if an adversary chooses distributions in a suitable optimal transport neighborhood of the empirical measure), provided that suitable martingale constraints are also imposed. Further, we introduce a relaxation of the martingale constraints which not only provides a unified viewpoint to a class of existing robust methods but also leads to new regularization tools. To realize…
We investigate the problem of optimal transport in the so-called Beckmann form, i.e. given two Radon measures on a compact set, we seek an optimal flow field which is a vector valued Radon measure on the same set that describes a flow between these two measures and minimizes a certain linear cost function. We consider $L^α$ regularization of the problem, which guarantees uniqueness and forces the solution to be an integrable function rather than a Radon measure. This regularization naturally gives rise to a semi-smooth Newton scheme that can be used to solve the problem numerically. Besides mo…
This paper is concerned with an optimization problem governed by the Kantorovich optimal transportation problem. This gives rise to a bilevel optimization problem, which can be reformulated as a mathematical problem with complementarity constraints in the space of regular Borel measures. Because of the non-smoothness induced by the complementarity relations, problems of this type are frequently regularized. Here we apply a quadratic regularization of the Kantorovich problem. As the title indicates, this is the first part in a series of three papers. It addresses the existence of optimal soluti…
We give a new proof of the Caffarelli contraction theorem, which states that the Brenier optimal transport map sending the standard Gaussian measure onto a uniformly log-concave probability measure is Lipschitz. The proof combines a recent variational characterization of Lipschitz transport map by the second author and Juillet with a convexity property of optimizers in the dual formulation of the entropy-regularized optimal transport (or Schr{ö}dinger) problem.
Well-Posedness and Efficient Algorithms for Inverse Optimal Transport with Bregman Regularization
Yau Mathematical Sciences Center, Tsinghua University（丘成桐数学科学中心，清华大学）；Department of Mathematical Sciences, Tsinghua University（数学科学系，清华大学）；Department of Mathematics, Cornell University（数学系，康奈尔大学）
We investigate the problem of optimal transport in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider quadratic regularization of the problem, which forces the optimal transport plan to be a square integrable function rather than a Radon measure. We derive the dual problem and show strong duality and existence of primal and dual solutions to the regularized problem. Then we derive …
Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization
Operations Research and Information Engineering, Cornell University（运筹学与信息工程系，康奈尔大学）；Department of Electrical Engineering and Computer Sciences, UC Berkeley（电气工程与计算机科学系，加州大学伯克利分校）；Automatic Control Lab, ETH Zürich（自动控制实验室，瑞士联邦理工学院（ETH） Zurich分校）；Risk Analytics and Optimization Chair, EPFL Lausanne（风险分析与优化 chair，瑞士联邦理工学院（EPFL） Lausanne分校）
This paper is concerned with an optimization problem that is constrained by the Kantorovich optimal transportation problem. This bilevel optimization problem can be reformulated as a mathematical problem with complementarity constraints in the space of regular Borel measures. Because of the non-smoothness induced by the complementarity relations, problems of this type are frequently regularized. Here we apply a quadratic regularization of the Kantorovich problem. As the title indicates, this is the second part in a series of three papers. While the existence of optimal solutions to both the bi…
Semi-discrete optimal transport problems, which evaluate the Wasserstein distance between a discrete and a generic (possibly non-discrete) probability measure, are believed to be computationally hard. Even though such problems are ubiquitous in statistics, machine learning and computer vision, however, this perception has not yet received a theoretical justification. To fill this gap, we prove that computing the Wasserstein distance between a discrete probability measure supported on two points and the Lebesgue measure on the standard hypercube is already #P-hard. This insight prompts us to se…
Ordinary differential equations for regularized variational problems involving semi-discrete optimal transport
Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Inria Saclay（法国国家信息与自动化研究所萨克莱分中心）；Institut Universitaire de France (IUF)（法兰西学院）；University of Alberta（阿尔伯塔大学）
Alfréd Rényi Institute of Mathematics（阿尔弗雷德·雷尼数学研究所）
Entropic regularization provides a generalization of the original optimal transport problem. It introduces a penalty term defined by the Kullback-Leibler divergence, making the problem more tractable via the celebrated Sinkhorn algorithm. Replacing the Kullback-Leibler divergence with a general $f$-divergence leads to a natural generalization. The case of divergences defined by superlinear functions was recently studied by Di Marino and Gerolin. Using convex analysis, we extend the theory developed so far to include all $f$-divergences defined by functions of Legendre type, and prove that unde…
This work studies how the introduction of the entropic regularization term in unbalanced Optimal Transport (OT) models may alter their homogeneity with respect to the input measures. We observe that in common settings (including balanced OT and unbalanced OT with Kullback-Leibler divergence to the marginals), although the optimal transport cost itself is not homogeneous, optimal transport plans and the so-called Sinkhorn divergences are indeed homogeneous. However, homogeneity does not hold in more general Unbalanced Regularized Optimal Transport (UROT) models, for instance those using the Tot…
This chapter describes techniques for the numerical resolution of optimal transport problems. We will consider several discretizations of these problems, and we will put a strong focus on the mathematical analysis of the algorithms to solve the discretized problems. We will describe in detail the following discretizations and corresponding algorithms: the assignment problem and Bertsekas auction's algorithm; the entropic regularization and Sinkhorn-Knopp's algorithm; semi-discrete optimal transport and Oliker-Prussner or damped Newton's algorithm, and finally semi-discrete entropic regularizat…
We propose a family of relaxations of the optimal transport problem which regularize the problem by introducing an additional minimization step over a small region around one of the underlying transporting measures. The type of regularization that we obtain is related to smoothing techniques studied in the optimization literature. When using our approach to estimate optimal transport costs based on empirical measures, we obtain statistical learning bounds which are useful to guide the amount of regularization, while maintaining good generalization properties. To illustrate the computational ad…
Technical University of Munich（慕尼黑工业大学）；European Institute for Molecular Imaging（欧洲分子成像研究所）；University of Münster（明斯特大学）
We propose a novel dynamic image reconstruction method from PET listmode data that could be particularly suited to tracking single or small numbers of cells. In contrast to conventional PET reconstruction our method combines the information from all detected events not only to reconstruct the dynamic evolution of the radionuclide distribution, but also to improve the reconstruction at each single time point by enforcing temporal consistency. This is achieved via optimal transport regularization where in principle, among all possible temporally evolving radionuclide distributions consistent wit…
Potsdam Institute for Climate Impact Research（波茨坦气候影响研究所）；University of Lübeck（吕贝克大学）
Functional lifting methods provide a tool for approximating solutions of difficult non-convex problems by embedding them into a larger space. In this work, we investigate a mathematically rigorous formulation based on embedding into the space of pointwise probability measures over a fixed range $Γ$. Interestingly, this approach can be derived as a generalization of the theory of dynamical optimal transport. Imposing the established continuity equation as a constraint corresponds to variational models with first-order regularization. By modifying the continuity equation, the approach can also b…
