During training, supervised object detection tries to correctly match the predicted bounding boxes and associated classification scores to the ground truth. This is essential to determine which predictions are to be pushed towards which solutions, or to be discarded. Popular matching strategies include matching to the closest ground truth box (mostly used in combination with anchors), or matching via the Hungarian algorithm (mostly used in anchor-free methods). Each of these strategies comes with its own properties, underlying losses, and heuristics. We show how Unbalanced Optimal Transport unifies these different approaches and opens a whole continuum of methods in between. This allows for a finer selection of the desired properties. Experimentally, we show that training an object detection model with Unbalanced Optimal Transport is able to reach the state-of-the-art both in terms of Average Precision and Average Recall as well as to provide a faster initial convergence. The approach is well suited for GPU implementation, which proves to be an advantage for large-scale models.
This study examines the time complexities of the unbalanced optimal transport problems from an algorithmic perspective for the first time. We reveal which problems in unbalanced optimal transport can/cannot be solved efficiently. Specifically, we prove that the Kantorovich Rubinstein distance and optimal partial transport in the Euclidean metric cannot be computed in strongly subquadratic time under the strong exponential time hypothesis. Then, we propose an algorithm that solves a more general unbalanced optimal transport problem exactly in quasi-linear time on a tree metric. The proposed alg…
We consider constrained policy optimization in Reinforcement Learning, where the constraints are in form of marginals on state visitations and global action executions. Given these distributions, we formulate policy optimization as unbalanced optimal transport over the space of occupancy measures. We propose a general purpose RL objective based on Bregman divergence and optimize it using Dykstra's algorithm. The approach admits an actor-critic algorithm for when the state or action space is large, and only samples from the marginals are available. We discuss applications of our approach and pr…
Optimal transport induces the Earth Mover's (Wasserstein) distance between probability distributions, a geometric divergence that is relevant to a wide range of problems. Over the last decade, two relaxations of optimal transport have been studied in depth: unbalanced transport, which is robust to the presence of outliers and can be used when distributions don't have the same total mass; entropy-regularized transport, which is robust to sampling noise and lends itself to fast computations using the Sinkhorn algorithm. This paper combines both lines of work to put robust optimal transport on so…
The Institute of Statistical Mathematics（统计数学研究所）；The University of Akron（阿克伦大学）；RIKEN AIP（理化学研究所先进智能项目中心）
Optimal transport (OT) is a popular and powerful tool for comparing probability measures. However, OT suffers a few drawbacks: (i) input measures required to have the same mass, (ii) a high computational complexity, and (iii) indefiniteness which limits its applications on kernel-dependent algorithmic approaches. To tackle issues (ii)--(iii), Le et al. (2022) recently proposed Sobolev transport for measures on a graph having the same total mass by leveraging the graph structure over supports. In this work, we consider measures that may have different total mass and are supported on a graph met…
UOTIP: Unbalanced Optimal Transport Map for Unpaired Inverse Problems
Sungkyunkwan University（成均馆大学）；IPAI (Interdisciplinary Program in Artificial Intelligence, Seoul National University)（人工智能跨学科项目（Seoul National University））；Department of Mathematical Sciences, Seoul National University（首尔国立大学数学科学系）
Optimal transport distances have found many applications in machine learning for their capacity to compare non-parametric probability distributions. Yet their algorithmic complexity generally prevents their direct use on large scale datasets. Among the possible strategies to alleviate this issue, practitioners can rely on computing estimates of these distances over subsets of data, {\em i.e.} minibatches. While computationally appealing, we highlight in this paper some limits of this strategy, arguing it can lead to undesirable smoothing effects. As an alternative, we suggest that the same min…
ETH Zurich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；Microsoft Research（微软研究院）
Comparing unpaired samples of a distribution or population taken at different points in time is a fundamental task in many application domains where measuring populations is destructive and cannot be done repeatedly on the same sample, such as in single-cell biology. Optimal transport (OT) can solve this challenge by learning an optimal coupling of samples across distributions from unpaired data. However, the usual formulation of OT assumes conservation of mass, which is violated in unbalanced scenarios in which the population size changes (e.g., cell proliferation or death) between measuremen…
WFR-MFM: One-Step Inference for Dynamic Unbalanced Optimal Transport
LMAM and School of Mathematical Sciences, Peking University（理学院和数学科学学院，北京大学）；Center for Machine Learning Research, Peking University（机器学习研究中心，北京大学）；Center for Quantitative Biology, Peking University（定量生物学研究中心，北京大学）；National Engineering Laboratory for Big Data Analysis and Applications（大数据分析与应用国家工程实验室）；AI for Science Institute（人工智能科学研究院）；Center for Data Science, Peking University（数据科学中心，北京大学）
Neighbor Embeddings Using Unbalanced Optimal Transport Metrics
Department of Mathematics, University of Texas at Arlington（德克萨斯大学阿灵顿分校数学系）；Division of Data Science, University of Texas at Arlington（德克萨斯大学阿灵顿分校数据科学 division）
Université Bretagne Sud, IRISA（南布列塔尼大学，IRISA）；CMAP, Ecole Polytechnique, IP Paris（巴黎综合理工学院应用数学中心，巴黎综合理工学院，巴黎IP）；LTCI, Télécom Paris, IP Paris（巴黎电信学院LTCI实验室，巴黎IP）；Univ. Lyon, UJM-Saint-Etienne, CNRS, UMR 5516（里昂大学，圣艾蒂安让·莫内大学，法国国家科学研究中心，UMR 5516）；Center for Computational Molecular Biology, Brown University（布朗大学计算分子生物学中心）；Department of Computer Science, Brown University（布朗大学计算机科学系）
Optimal transport (OT) compares probability distributions by computing a meaningful alignment between their samples. CO-optimal transport (COOT) takes this comparison further by inferring an alignment between features as well. While this approach leads to better alignments and generalizes both OT and Gromov-Wasserstein distances, we provide a theoretical result showing that it is sensitive to outliers that are omnipresent in real-world data. This prompts us to propose unbalanced COOT for which we provably show its robustness to noise in the compared datasets. To the best of our knowledge, this…
Unbalanced optimal transport (UOT) extends optimal transport (OT) to take into account mass variations to compare distributions. This is crucial to make OT successful in ML applications, making it robust to data normalization and outliers. The baseline algorithm is Sinkhorn, but its convergence speed might be significantly slower for UOT than for OT. In this work, we identify the cause for this deficiency, namely the lack of a global normalization of the iterates, which equivalently corresponds to a translation of the dual OT potentials. Our first contribution leverages this idea to develop a …
Optimal Transport (OT) has recently emerged as a central tool in data sciences to compare in a geometrically faithful way point clouds and more generally probability distributions. The wide adoption of OT into existing data analysis and machine learning pipelines is however plagued by several shortcomings. This includes its lack of robustness to outliers, its high computational costs, the need for a large number of samples in high dimension and the difficulty to handle data in distinct spaces. In this review, we detail several recently proposed approaches to mitigate these issues. We insist in…
Learning stochastic dynamics from snapshots through regularized unbalanced optimal transport
LMAM and School of Mathematical Sciences, Peking University（北京大学应用数学系和数学科学学院）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；NELBDA, Peking University（北京大学NELBDA）；Center for Quantitative Biology, Peking University（北京大学定量生物学中心）；AI for Science Institute, Beijing（北京人工智能科学研究院）
This paper addresses the problem of Unbalanced Optimal Transport (UOT) in which the marginal conditions are relaxed (using weighted penalties in lieu of equality) and no additional regularization is enforced on the OT plan. In this context, we show that the corresponding optimization problem can be reformulated as a non-negative penalized linear regression problem. This reformulation allows us to propose novel algorithms inspired from inverse problems and nonnegative matrix factorization. In particular, we consider majorization-minimization which leads in our setting to efficient multiplicativ…
Simulation-free Unbalanced Dynamic Optimal Transport with General Growth Penalty
Beijing International Center for Mathematical Research, Peking University（北京大学北京数学科学研究中心）；Center for Quantitative Biology, Peking University（北京大学定量生物学中心）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；National Engineering Laboratory for Big Data Analysis and Applications（大数据分析与应用国家工程实验室）；AI for Science Institute, Beijing（北京人工智能科学研究院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）
INRIA (PreMeDICaL & HeKA)（INRIA（PreMeDICaL与HeKA））；CNRS and ENS, PSL Université（CNRS与ENS，PSL大学）；Mathematics Institute, University of Warwick（数学研究所，沃里克大学）
Robust Barycenter Estimation using Semi-Unbalanced Neural Optimal Transport
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Artificial Intelligence Research Institute（人工智能研究院）；Georgia Institute of Technology（佐治亚理工学院）；Sungkyunkwan University（成均馆大学）
The Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）
This paper considers stochastic first-order algorithms for convex-concave minimax problems of the form $\min_{\bf x}\max_{\bf y}f(\bf x, \bf y)$, where $f$ can be presented by the average of $n$ individual components which are $L$-average smooth. For $μ_x$-strongly-convex-$μ_y$-strongly-concave setting, we propose a new method which could find a $\varepsilon$-saddle point of the problem in $\tilde{\mathcal O} \big(\sqrt{n(\sqrt{n}+κ_x)(\sqrt{n}+κ_y)}\log(1/\varepsilon)\big)$ stochastic first-order complexity, where $κ_x\triangleq L/μ_x$ and $κ_y\triangleq L/μ_y$. This upper bound is near optim…
WFR-FM: Simulation-Free Dynamic Unbalanced Optimal Transport
LMAM and School of Mathematical Sciences, Peking University（北京大学数学科学学院及数学及其应用实验室）；Center for Quantitative Biology, Peking University（北京大学定量生物学中心）；Beijing International Center for Mathematical Research, Peking University（北京大学北京国际数学研究中心）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；NSF-Simons Center for Multiscale Cell Fate Research, University of California, Irvine（加州大学尔湾分校NSF-Simons多尺度细胞命运研究中心）；Department of Developmental and Cell Biology, University of California, Irvine（加州大学尔湾分校发育与细胞生物学系）；Department of Mathematics, University of California, Irvine（加州大学尔湾分校数学系）；National Engineering Laboratory for Big Data Analysis and Applications, Beijing（北京大数据分析与应用技术国家工程实验室）；AI for Science Institute, Beijing（北京科学智能研究院）
Variational Regularized Unbalanced Optimal Transport: Single Network, Least Action
Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；LMAM and School of Mathematical Sciences, Peking University（北京大学LMAM与数学科学学院）；Center for Quantitative Biology, Peking University（北京大学定量生物学中心）；NELBDA, Peking University（北京大学NELBDA研究中心）；AI for Science Institute, Beijing（北京人工智能科学研究院）
Modeling Cell Dynamics and Interactions with Unbalanced Mean Field Schrödinger Bridge
LMAM and School of Mathematical Sciences, Peking University（北京大学理学院和数学科学学院）；Center for Quantitative Biology, Peking University（北京大学定量生物学中心）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；NELBDA, Peking University（北京大学NELBDA研究中心）；AI for Science Institute, Beijing（北京人工智能科学研究院）
Bocconi University（博科尼大学）；Politecnico di Torino（都灵理工大学）
We analyze the connection between minimizers with good generalizing properties and high local entropy regions of a threshold-linear classifier in Gaussian mixtures with the mean squared error loss function. We show that there exist configurations that achieve the Bayes-optimal generalization error, even in the case of unbalanced clusters. We explore analytically the error-counting loss landscape in the vicinity of a Bayes-optimal solution, and show that the closer we get to such configurations, the higher the local entropy, implying that the Bayes-optimal solution lays inside a wide flat regio…
Despite the great success of GANs in images translation with different conditioned inputs such as semantic segmentation and edge maps, generating high-fidelity realistic images with reference styles remains a grand challenge in conditional image-to-image translation. This paper presents a general image translation framework that incorporates optimal transport for feature alignment between conditional inputs and style exemplars in image translation. The introduction of optimal transport mitigates the constraint of many-to-one feature matching significantly while building up accurate semantic co…
Sinkhorn-CPD: Robust point cloud registration via unbalanced entropic optimal transport
LMIB & School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院与LMIB）；State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院数学科学国家重点实验室）；Beijing Key Laboratory of Artificial Intelligence Innovation and Application in the Machine Tool Industry, School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院北京市机床行业人工智能创新与应用重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Experts-Guided Unbalanced Optimal Transport for ISP Learning from Unpaired and/or Paired Data
Computer Vision Lab, CAIDAS & IFI, University of Würzburg（计算机视觉实验室，CAIDAS与IFI，乌尔姆大学）；Institute for Information Transmission Problems, RAS（信息传输问题研究所，俄罗斯科学院）；Moscow Institute of Physics and Technologies（莫斯科物理技术学院）；Artificial Intelligence Research Institute（人工智能研究 institute）
