arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.01792stat.MLcs.LG
收藏

小批量最佳运输距离;分析和应用

Minibatch optimal transport distances; analysis and applications

Univ. Bretagne-Sud(南布列塔尼大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);IRISA(法国雷恩信息与自动化系统研究所);School of Mathematics, The University of Edinburgh(爱丁堡大学数学学院);The Maxwell Institute for the Mathematical Sciences(麦克斯韦数学科学研究所);Univ Rennes(雷恩大学);IRMAR(雷恩数学研究所);École Polytechnique(巴黎综合理工学院);Univ Lyon(里昂大学);ENS de Lyon(里昂高等师范学院);UCB Lyon 1(里昂第一大学克洛德·贝尔纳大学)

Kilian Fatras, Younes Zine, Szymon Majewski, Rémi Flamary, Rémi Gribonval, Nicolas Courty

摘要:

最优运输距离已成为比较概率分布的经典工具,并在机器学习中有许多应用。然而,尽管最近的算法的发展,其复杂性阻止其直接用于大规模数据集。为了克服这一挑战,一个常见的解决方法是计算这些距离的小批量,即平均几个较小的最佳运输问题的结果。在本文中,我们提出了一个扩展的分析这种做法,其影响以前研究在有限的情况下。我们首先考虑各种各样的最优传输内核。我们特别认为,minibatch策略具有吸引人的属性,如无偏估计量,梯度和浓度约束周围的期望,但也有限制:minibatch OT不是一个距离。为了恢复一些丢失的距离公理,我们引入了一个去偏的minibatch OT函数,并研究其统计和优化特性。除了这种理论分析,我们还对梯度流、生成对抗网络(GANs)或颜色转移进行了实证实验,以突出这种策略的实际意义。

英文摘要:

Optimal transport distances have become a classic tool to compare probability distributions and have found many applications in machine learning. Yet, despite recent algorithmic developments, their complexity prevents their direct use on large scale datasets. To overcome this challenge, a common workaround is to compute these distances on minibatches i.e. to average the outcome of several smaller optimal transport problems. We propose in this paper an extended analysis of this practice, which effects were previously studied in restricted cases. We first consider a large variety of Optimal Transport kernels. We notably argue that the minibatch strategy comes with appealing properties such as unbiased estimators, gradients and a concentration bound around the expectation, but also with limits: the minibatch OT is not a distance. To recover some of the lost distance axioms, we introduce a debiased minibatch OT function and study its statistical and optimisation properties. Along with this theoretical analysis, we also conduct empirical experiments on gradient flows, generative adversarial networks (GANs) or color transfer that highlight the practical interest of this strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑