Optimal Transport (OT) is a mathematical framework that first emerged in the eighteenth century and has led to a plethora of methods for answering many theoretical and applied questions. The last decade has been a witness to the remarkable contributions of this classical optimization problem to machine learning. This paper is about where and how optimal transport is used in machine learning with a focus on the question of scalable optimal transport. We provide a comprehensive survey of optimal transport while ensuring an accessible presentation as permitted by the nature of the topic and the context. First, we explain the optimal transport background and introduce different flavors (i.e., mathematical formulations), properties, and notable applications. We then address the fundamental question of how to scale optimal transport to cope with the current demands of big and high dimensional data. We conduct a systematic analysis of the methods used in the literature for scaling OT and present the findings in a unified taxonomy. We conclude with presenting some open challenges and discussing potential future research directions. A live repository of related OT research papers is maintained in https://github.com/abdelwahed/OT_for_big_data.git
On Scalable and Efficient Computation of Large Scale Optimal Transport
Georgia Institute of Technology(佐治亚理工学院)
最优传输(OT)自然出现在许多机器学习应用中,但沉重的计算负担限制了它的广泛应用。为了解决可扩展性问题,我们提出了一个基于隐式生成学习的框架,称为SPOT(Scalable Push-forward of Optimal Transport)。具体来说,我们近似的最优运输计划的推进的参考分布,并铸造的最优运输问题成一个极大极小问题。然后,我们可以有效地解决OT问题使用原始对偶随机梯度型算法。我们还表明,我们可以恢复密度的最佳运输计划,使用神经常微分方程。在模拟数据集和真实数据集上的数值实验表明,SPOT算法具有良好的收敛性和鲁棒性。SPOT还使我们能够从最佳传输计划中高效地进行采样,这有利于下游应用,如领域自适应。
A scalable deep learning approach for solving high-dimensional dynamic optimal transport
North China Electric Power University(华北电力大学);Tsinghua University(清华大学);Yaqi Lake Beijing Institute of Mathematical Sciences and Applications(雁栖湖北京数学科学与应用研究院);Peking University(北京大学)
Scalable Approximate Algorithms for Optimal Transport Linear Models
Swiss Data Science Center(瑞士数据科学中心);Paul Scherrer Institute(保罗谢勒研究所);Ecole Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);PSI Center for Neutron and Muon Sciences CNM(PSI中子与缪子科学中心);Air Quality Research Center, University of California, Davis(加州大学戴维斯分校空气质量研究中心);Laboratory for Environmental Spectrochemistry, Ecole Polytechnique Fédérale de Lausanne(洛桑联邦理工学院环境光谱学实验室)
Wasserstein Wormhole: Scalable Optimal Transport Distance with Transformers
Sloan Kettering Institute(斯隆凯特琳研究所);Memorial Sloan Kettering Cancer Center(纪念斯隆凯特琳癌症中心);Tri-Institutional Training Program in Computational Biology and Medicine(三机构计算生物学与医学培训项目);Weill Cornell Medicine(威尔康奈尔医学院);Columbia University(哥伦比亚大学);Yale University(耶鲁大学);Howard Hughes Medical Institute(霍华德·休斯医学研究所)
SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);School of Life Science and Technology, China Pharmaceutical University(中国药科大学生命科学与技术学院);Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology, Shenzhen(深圳先进技术大学计算机科学与控制工程学院)
Scalable Bi-causal Optimal Transport via KL Relaxation and Policy Gradients
Department of Applied Mathematics and Statistics, Data Science and AI Institute, and Mathematical Institute for Data Science, Johns Hopkins University(应用数学与统计学系、数据科学与人工智能学院以及数据科学数学研究所,约翰霍普金斯大学);Department of Statistics, Oxford-Man Institute of Quantitative Finance, and Nuffield College, University of Oxford(统计系、牛津-曼定量金融研究所以及牛津大学努尔菲尔德学院);Management Science & Engineering Department, Stanford University(管理科学与工程系,斯坦福大学);School of Mathematical Sciences, Peking University(北京大学数学科学学院)
Scalable Reinforcement Learning for Virtual Machine Scheduling
School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院)