Eustasio del Barrio, Jean-Michel Loubes
We propose to tackle the problem of understanding the effect of regularization in Sinkhorn algotihms. In the case of Gaussian distributions we provide a closed form for the regularized optimal transport which enables to provide a better understanding of the effect of the regularization from a statistical framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We prove a central limit theorem for the entropic transportation cost between subgaussian probability measures, centered at the population cost. This is the first result which allows for asymptotically valid inference for entropic optimal transport between measures which are not necessarily discrete. In the compactly supported case, we complement these results with new, faster, convergence rates for the expected entropic transportation cost between empirical measures. Our proof is based on strengthening convergence results for dual solutions to the entropic optimal transport problem.
Entropic regularization is a method for large-scale linear programming. Geometrically, one traces intersections of the feasible polytope with scaled toric varieties, starting at the Birch point. We compare this to log-barrier methods, with reciprocal linear spaces, starting at the analytic center. We revisit entropic regularization for unbalanced optimal transport, and we develop the use of optimal conic couplings. We compute the degree of the associated toric variety, and we explore algorithms like iterative scaling.
Extending Mean-Field Variational Inference via Entropic Regularization: Theory and Computation
本文提出$Ξ$-VI方法，通过熵正则化扩展均场推断，分析其理论性质与计算效率，并在模拟和真实数据中验证其优于传统均场方法。
The goal of regression and classification methods in supervised learning is to minimize the empirical risk, that is, the expectation of some loss function quantifying the prediction error under the empirical distribution. When facing scarce training data, overfitting is typically mitigated by adding regularization terms to the objective that penalize hypothesis complexity. In this paper we introduce new regularization techniques using ideas from distributionally robust optimization, and we give new probabilistic interpretations to existing techniques. Specifically, we propose to minimize the w…
We describe a graph-based semi-supervised learning framework in the context of deep neural networks that uses a graph-based entropic regularizer to favor smooth solutions over a graph induced by the data. The main contribution of this work is a computationally efficient, stochastic graph-regularization technique that uses mini-batches that are consistent with the graph structure, but also provides enough stochasticity (in terms of mini-batch data diversity) for convergence of stochastic gradient descent methods to good solutions. For this work, we focus on results of frame-level phone classifi…
Minimum intrinsic dimension scaling for entropic optimal transport
本研究受流形假设启发，提出熵最优传输的最小本征维度（MID）标度现象，推导仅由单尺度本征维度最小值决定收敛速率的统计界，证明其普适性并首次严格解释熵正则化的统计效应。
Sample complexity of unbalanced entropic OT
本文研究非平衡熵正则化最优传输在最优耦合层面的样本复杂度，通过平移不变对偶公式和几何估计导出经验耦合的高概率有限样本界，揭示了正则化在降低维度诅咒和减少样本需求方面的必要性。
A Sinkhorn-type Algorithm for Constrained Optimal Transport
针对带等式与不等式约束的最优传输问题，本文推导对应熵正则化公式，提出带理论保障的Sinkhorn型算法，证明其误差界与收敛速率，并加入动态正则化调度与二阶加速以提升收敛性能，拓展了熵最优传输的适用场景。
Entropic regularization is quickly emerging as a new standard in optimal transport (OT). It enables to cast the OT computation as a differentiable and unconstrained convex optimization problem, which can be efficiently solved using the Sinkhorn algorithm. However, entropy keeps the transportation plan strictly positive and therefore completely dense, unlike unregularized OT. This lack of sparsity can be problematic in applications where the transportation plan itself is of interest. In this paper, we explore regularizing the primal and dual OT formulations with a strongly convex term, which co…
Averaged Mirror Descent and Dual Gradient Methods: Convergent Algorithms for Entropic Gromov-Wasserstein Problems
针对熵正则化Gromov-Wasserstein问题，提出平均镜像下降算法并证明其对任意成本收敛，同时证明固定步长对偶梯度法的收敛性，在经典MD失败的案例上两者均有效。
A note on the relations between mixture models, maximum-likelihood and entropic optimal transport
本文证明混合模型的最大似然估计等价于熵正则化最优传输问题，并以高斯混合模型为例说明EM算法是该问题的块坐标下降法。
PINS: Proximal Iterations with Sparse Newton and Sinkhorn for Optimal Transport
PINS通过近端迭代结合稀疏牛顿和Sinkhorn算法，克服了最优传输中熵偏置 plateau 的限制，实现了更高的精度和效率。
Entropic Neural Optimal Transport via Diffusion Processes
本文提出一种基于薛定谔桥鞍点重构的端到端神经算法，用于大规模熵正则化最优传输，具有单步学习、快速推理和支持小熵正则化系数的优点。
We tackle the challenge of disentangled representation learning in generative adversarial networks (GANs) from the perspective of regularized optimal transport (OT). Specifically, a smoothed OT loss gives rise to an implicit transportation plan between the latent space and the data space. Based on this theoretical observation, we exploit a structured regularization on the transportation plan to encourage a prescribed latent subspace to be informative. This yields the formulation of a novel informative OT-based GAN. By convex duality, we obtain the equivalent view that this leads to perturbed g…
Sparse Domain Transfer via Elastic Net Regularization
针对域迁移中迁移映射的稀疏性需求，提出结合L1与L2正则的ENOT框架，通过对偶形式的稀疏化梯度求解稀疏稳定的迁移映射，可用于特征选择，在合成与真实数据上验证了有效性。
Doubly Regularized Entropic Wasserstein Barycenters
本文提出双正则化 $(λ,τ)$-熵 Wasserstein 重心，证明其去偏逼近、光滑稳定、$n^{-1/2}$ 估计速率，并给出指数收敛的无网格噪声粒子梯度下降算法。
Deep neural networks have established as a powerful tool for large scale supervised classification tasks. The state-of-the-art performances of deep neural networks are conditioned to the availability of large number of accurately labeled samples. In practice, collecting large scale accurately labeled datasets is a challenging and tedious task in most scenarios of remote sensing image analysis, thus cheap surrogate procedures are employed to label the dataset. Training deep neural networks on such datasets with inaccurate labels easily overfits to the noisy training labels and degrades the perf…
Bridging Maximum Likelihood and Optimal Transport for Efficient Inference and Model Selection in Stochastic Block Models
本文通过最优传输视角研究随机块模型，提出正则化与未正则化的半松弛Gromov-Wasserstein估计器，实现聚类与模型参数的联合推断及簇数自动选择。
Sinkhorn Treatment Effects: A Causal Optimal Transport Measure
本文提出Sinkhorn治疗效应，作为因果分布差异的熵最优传输度量，通过核方法构建去偏估计器，并提出聚合检验方法以提高测试效力。
Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates
该研究提出一种结合范数约束线性最小化神谕与熵权重更新的单循环多目标优化方法，扩展了MOON的谱构造，在确定性和随机设置下分别达到O(T^{-1/2})和O(T^{-1/4})的收敛率，实验验证了其性能。
Mini-batch optimal transport (m-OT) has been successfully used in practical applications that involve probability measures with a very high number of supports. The m-OT solves several smaller optimal transport problems and then returns the average of their costs and transportation plans. Despite its scalability advantage, the m-OT does not consider the relationship between mini-batches which leads to undesirable estimation. Moreover, the m-OT does not approximate a proper metric between probability measures since the identity property is not satisfied. To address these problems, we propose a n…
Asymptotic distribution and convergence rates of stochastic algorithms for entropic optimal transportation between probability measures
针对熵正则最优传输的 Sinkhorn 散度估计，论文提出基于半对偶形式的 Robbins-Monro 递推随机算法，证明其在离散与半离散情形下的几乎必然收敛、渐近正态性及期望超额风险收敛速率。
Sparse Partial Optimal Transport via Quadratic Regularization
本文提出了一种基于二次正则化的部分最优传输（QPOT）新方法，通过引入稀疏性解决了传统熵正则化方法产生密集传输计划的问题，并在合成和真实数据集及实际应用中验证了其优越的稀疏性和性能。
Neural Estimation for Scaling Entropic Multimarginal Optimal Transport
本文提出 Neural Entropic MOT (NEMOT) 框架，通过小批量训练神经网络将计算复杂度从数据集大小转移到小批量大小，显著提升了多边际熵最优传输的可扩展性，并实现数个数量级加速。
Some remarks on the connection between generative diffusion and optimal transport
本文揭示生成式扩散与最优传输间的三种联系，包括联合分布对应熵正则化最优传输解、模式分离的动力学特性及反向过程的变分原理。
Annealed Sinkhorn for Optimal Transport: convergence, regularization path and debiasing
提出Debiased Annealed Sinkhorn算法，通过减少松弛误差实现更快的退火调度，单次运行即可覆盖标准Sinkhorn算法的完整速度-精度Pareto前沿。
Optimal Transport (OT) has recently emerged as a central tool in data sciences to compare in a geometrically faithful way point clouds and more generally probability distributions. The wide adoption of OT into existing data analysis and machine learning pipelines is however plagued by several shortcomings. This includes its lack of robustness to outliers, its high computational costs, the need for a large number of samples in high dimension and the difficulty to handle data in distinct spaces. In this review, we detail several recently proposed approaches to mitigate these issues. We insist in…
The entropic optimal (self-)transport problem: Limit distributions for decreasing regularization with application to score function estimation
研究熵最优自运输问题的统计性质，分析正则化参数缩小时的极限分布，揭示经验熵最优自运输计划的分数函数收敛性及波动特性。
Entropic regularization provides a generalization of the original optimal transport problem. It introduces a penalty term defined by the Kullback-Leibler divergence, making the problem more tractable via the celebrated Sinkhorn algorithm. Replacing the Kullback-Leibler divergence with a general $f$-divergence leads to a natural generalization. The case of divergences defined by superlinear functions was recently studied by Di Marino and Gerolin. Using convex analysis, we extend the theory developed so far to include all $f$-divergences defined by functions of Legendre type, and prove that unde…
Correctly estimating the discrepancy between two data distributions has always been an important task in Machine Learning. Recently, Cuturi proposed the Sinkhorn distance which makes use of an approximate Optimal Transport cost between two distributions as a distance to describe distribution discrepancy. Although it has been successfully adopted in various machine learning applications (e.g. in Natural Language Processing and Computer Vision) since then, the Sinkhorn distance also suffers from two unnegligible limitations. The first one is that the Sinkhorn distance only gives an approximation…
The matching principles behind optimal transport (OT) play an increasingly important role in machine learning, a trend which can be observed when OT is used to disambiguate datasets in applications (e.g. single-cell genomics) or used to improve more complex methods (e.g. balanced attention in transformers or self-supervised learning). To scale to more challenging problems, there is a growing consensus that OT requires solvers that can operate on millions, not thousands, of points. The low-rank optimal transport (LOT) approach advocated in \cite{scetbon2021lowrank} holds several promises in tha…
Sparsity-Constrained Optimal Transport
本文针对最优传输提出显式基数约束方法，以控制传输计划的稀疏性，并将其应用于稀疏专家混合中的 token 与专家匹配，证明对应半对偶问题可用一阶梯度方法求解，在无正则化与二次正则化最优传输之间建立联系。
On Fitting Flow Models with Large Sinkhorn Couplings
针对无配对场景下流模型训练慢、推理成本高的问题，本文通过分片计算扩展批量规模，结合低熵正则化的大尺寸Sinkhorn耦合拟合流模型，在合成与图像生成任务中实现了性能提升。
Optimal transport (OT) has become a widely used tool in the machine learning field to measure the discrepancy between probability distributions. For instance, OT is a popular loss function that quantifies the discrepancy between an empirical distribution and a parametric model. Recently, an entropic penalty term and the celebrated Sinkhorn algorithm have been commonly used to approximate the original OT in a computationally efficient way. However, since the Sinkhorn algorithm runs a projection associated with the Kullback-Leibler divergence, it is often vulnerable to outliers. To overcome this…
Wasserstein barycenters provide a geometric notion of the weighted average of probability measures based on optimal transport. In this paper, we present a scalable algorithm to compute Wasserstein-2 barycenters given sample access to the input measures, which are not restricted to being discrete. While past approaches rely on entropic or quadratic regularization, we employ input convex neural networks and cycle-consistency regularization to avoid introducing bias. As a result, our approach does not resort to minimax optimization. We provide theoretical analysis on error bounds as well as empir…
Progressive Entropic Optimal Transport Solvers
针对熵最优传输求解器超参数难调优的问题，提出渐进式求解器ProgOT，结合动态OT思路与时间离散化分步求解，在大规模场景下更快更鲁棒，且具备传输映射估计的统计一致性。
Beyond entropic regularization: Debiased Gaussian estimators for discrete optimal transport and general linear programs
本文针对离散最优传输提出一种基于对偶惩罚函数的新正则化方法，通过线性组合两个不同调节参数的惩罚解抵消偏差，构造出以真实解为中心的无偏高斯估计量，并证明了朴素自助法的一致性。
Entropic Gromov-Wasserstein Distances: Stability and Algorithms
本文利用变分表示推导出求解二次Entropic GW距离的高效算法，通过加速梯度法和Sinkhorn迭代提供非渐近收敛保证，并在实验中验证了其最先进的性能。
Although optimal transport (OT) problems admit closed form solutions in a very few notable cases, e.g. in 1D or between Gaussians, these closed forms have proved extremely fecund for practitioners to define tools inspired from the OT geometry. On the other hand, the numerical resolution of OT problems using entropic regularization has given rise to many applications, but because there are no known closed-form solutions for entropic regularized OT problems, these approaches are mostly algorithmic, not informed by elegant closed forms. In this paper, we propose to fill the void at the intersecti…
Gromov-Wasserstein Distances: Entropic Regularization, Duality, and Sample Complexity
本文推导标准与熵正则化Gromov-Wasserstein距离的OT/EOT对偶形式，建立并证明其经验收敛速率的最优性，同时分析正则化稳定性及一维排列问题。
We compute exact second-order asymptotics for the cost of an optimal solution to the entropic optimal transport problem in the continuous-to-discrete, or semi-discrete, setting. In contrast to the discrete-discrete or continuous-continuous case, we show that the first-order term in this expansion vanishes but the second-order term does not, so that in the semi-discrete setting the difference in cost between the unregularized and regularized solution is quadratic in the inverse regularization parameter, with a leading constant that depends explicitly on the value of the density at the points of…
The diffeomorphic registration framework enables to define an optimal matching function between two probability measures with respect to a data-fidelity loss function. The non convexity of the optimization problem renders the choice of this loss function crucial to avoid poor local minima. Recent work showed experimentally the efficiency of entropy-regularized optimal transportation costs, as they are computationally fast and differentiable while having few minima. Following this approach, we provide in this paper a new framework based on Sinkhorn divergences, unbiased entropic optimal transpo…
We derive nearly tight and non-asymptotic convergence bounds for solutions of entropic semi-discrete optimal transport. These bounds quantify the stability of the dual solutions of the regularized problem (sometimes called Sinkhorn potentials) w.r.t. the regularization parameter, for which we ensure a better than Lipschitz dependence. Such facts may be a first step towards a mathematical justification of annealing or $\varepsilon$-scaling heuristics for the numerical resolution of regularized semi-discrete optimal transport. Our results also entail a non-asymptotic and tight expansion of the d…
The optimal transport map between the standard Gaussian measure and an $α$-strongly log-concave probability measure is $α^{-1/2}$-Lipschitz, as first observed in a celebrated theorem of Caffarelli. In this paper, we apply two classical covariance inequalities (the Brascamp-Lieb and Cramér-Rao inequalities) to prove a sharp bound on the Lipschitz constant of the map that arises from entropically regularized optimal transport. In the limit as the regularization tends to zero, we obtain an elegant and short proof of Caffarelli's original result. We also extend Caffarelli's theorem to the setting …
Sinkhorn algorithms for entropic vector quantile regression
本文提出两种熵正则化的向量分位数回归Sinkhorn算法，通过求解Schrödinger型系统或投影梯度步，实现线性收敛，并推导了对偶势函数的显式界。
The squared Wasserstein distance is a natural quantity to compare probability distributions in a non-parametric setting. This quantity is usually estimated with the plug-in estimator, defined via a discrete optimal transport problem which can be solved to $ε$-accuracy by adding an entropic regularization of order $ε$ and using for instance Sinkhorn's algorithm. In this work, we propose instead to estimate it with the Sinkhorn divergence, which is also built on entropic regularization but includes debiasing terms. We show that, for smooth densities, this estimator has a comparable sample comple…
Neural Estimation Of Entropic Optimal Transport
针对熵正则化最优传输的可扩展估计问题，提出用神经网络参数化半对偶表示并优化经验目标的方法，证明其非渐近误差界与极小极大最优速率。
Neural Entropic Optimal Transport and Gromov-Wasserstein Alignment
本文提出基于神经网络的熵最优传输与Gromov-Wasserstein对齐方法，通过小批量训练降低计算复杂度，实现大规模高维数据下的高效估计。
Lower Complexity Adaptation for Empirical Entropic Optimal Transport
本文证明经验熵最优传输的统计性能仅取决于两个测度中较简单者，遵循低复杂度自适应原则，并推广至熵Gromov-Wasserstein距离。
The notion of entropy-regularized optimal transport, also known as Sinkhorn divergence, has recently gained popularity in machine learning and statistics, as it makes feasible the use of smoothed optimal transportation distances for data analysis. The Sinkhorn divergence allows the fast computation of an entropically regularized Wasserstein distance between two probability distributions supported on a finite metric space of (possibly) high-dimension. For data sampled from one or two unknown probability distributions, we derive the distributional limits of the empirical Sinkhorn divergence and …
Central limit theorems and bootstrap for sparse regularized multimarginal optimal transport and barycenters
针对高维多样本Wasserstein重心估计，提出稀疏正则化多边际最优传输方法以避免熵正则化的质量过度扩散，并建立了相合性、中心极限定理与自助法有效性，数值实验验证其结构保持优势。
Sparsistency for Inverse Optimal Transport
本研究针对逆最优传输（iOT）不适定需正则化的问题，深入研究l1正则化理论，推导基础成本稀疏鲁棒恢复的充分条件（推广Lasso不可表示条件），并建立iOT与机器学习中图估计的联系。
Finite-sample bounds for regularized optimal transport
针对一般凸正则化（包括KL散度和L^p惩罚）的最优传输，推导了经验代价的非渐近偏差和方差界，改进了熵正则化结果并首次给出L^p正则化的完全定量结果。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。