我们研究用于求解熵正则化最优传输问题的Sinkhorn算法的收敛速率,其中至少一个概率测度μ在$\mathbb{R}^d$上具有密度。对于以$c_{\infty}$为界的半凹成本函数和正则化参数$λ>0$,我们得到了对偶次优性间隙的指数收敛保证,其收缩率是$λ/c_{\infty}$的多项式。这相对于通过Hilbert投影度量可达到的已知收缩率$1 - Θ(\exp(-c_{\infty}/λ))$而言,是一个指数级的改进。具体来说,当μ具有有界对数密度时,我们证明了收缩率值为$1-Θ(λ^2/c_\infty^2)$。在某些情况下,例如当μ是对数凹的且成本函数为$c(x,y)=-\langle x, y \rangle$时,该速率改进为$1-Θ(λ/c_\infty)$。后一速率与我们为各向同性高斯测度之间的传输所推导出的速率一致,表明在$λ/c_\infty$依赖关系上的紧致性。我们的结果完全是非渐近的,并且对问题的所有参数都是显式的。
英文摘要:
We study the convergence rate of Sinkhorn's algorithm for solving entropy-regularized optimal transport problems when at least one of the probability measures, $μ$, admits a density over $\mathbb{R}^d$. For a semi-concave cost function bounded by $c_{\infty}$ and a regularization parameter $λ> 0$, we obtain exponential convergence guarantees on the dual sub-optimality gap with contraction rate polynomial in $λ/c_{\infty}$. This represents an exponential improvement over the known contraction rate $1 - Θ(\exp(-c_{\infty}/λ))$ achievable via Hilbert's projective metric. Specifically, we prove a contraction rate value of $1-Θ(λ^2/c_\infty^2)$ when $μ$ has a bounded log-density. In some cases, such as when $μ$ is log-concave and the cost function is $c(x,y)=-\langle x, y \rangle$, this rate improves to $1-Θ(λ/c_\infty)$. The latter rate matches the one that we derive for the transport between isotropic Gaussian measures, indicating tightness in the dependency in $λ/c_\infty$. Our results are fully non-asymptotic and explicit in all the parameters of the problem.
Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich
School of Medicine, Nankai University(南开大学医学院);College of Artificial Intelligence, Nankai University(南开大学人工智能学院)
Sharp Stability of a String with Local Degenerate Kelvin-Voigt Damping
School of Mathematics, Tianjin University(天津大学数学学院);Department of Mathematics and Statistics, University of Minnesota(明尼苏达大学数学与统计系);School of Mathematics and Statistics, Beijing KeyLaboratory on MCAACI, Beijing Institute of Technology(北京理工大学数学与统计学院)
A Unified Analysis of First-Order Methods for Smooth Games via Integral Quadratic Constraints
University of Toronto(多伦多大学);Vector Institute(向量研究所);Northeastern University(东北大学)
积分二次约束理论可以证明含有非线性或不确定元素的关联系统的指数收敛性。在这项工作中,我们适应的IQC理论研究一阶方法光滑和强单调的游戏,并展示了如何设计定制的二次约束,以获得紧的上界的收敛速度。在此框架下,我们恢复了梯度法GD的已有界,导出了邻近点法PPM和乐观梯度法OG的更精确界,并给出了负动量法NM的整体收敛速度的\emph{for the first time},其迭代复杂度为$\mathcal{O}(κ^{1.5})$,与已知的下界相匹配.此外,对于时变系统,我们证明了具有最佳步长的梯度法在二次型李雅普诺夫函数下达到最快的可证明最坏情况收敛速度。最后,我们进一步扩展我们的分析,随机对策和研究乘性噪声对不同算法的影响。我们表明,这是不可能的一个步骤的内存实现加速的算法,如果它只查询梯度一次,每批(与随机强凸优化设置,这样的加速已被证明)。然而,我们展示了一种算法,它实现了每批两个梯度查询的加速。