arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.04006math.OCcs.LG
收藏

加速概率测度空间上的优化

Accelerating optimization over the space of probability measures

University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Eindhoven University of Technology(埃因霍温理工大学)

Shi Chen, Qin Li, Oliver Tse, Stephen J. Wright

AI总结:

本文针对概率测度空间的优化需求,提出一种类似欧氏空间动量方法的Hamiltonian-flow方法,并证明其在连续时间下可实现任意高阶收敛率。

摘要:

基于梯度的优化方法的加速具有重大的实际和理论意义,尤其在机器学习应用中。尽管大量关注集中在欧几里得空间内的优化,但机器学习中在概率测度空间上进行优化的需求,也促使人们探索该背景下的加速梯度方法。为此,我们引入了一种类似于欧几里得空间中基于动量方法的Hamiltonian-flow方法。我们证明,在连续时间设置下,基于该方法的算法可以实现任意高阶的收敛率。我们用数值例子补充了我们的发现。

英文摘要:

The acceleration of gradient-based optimization methods is a subject of significant practical and theoretical importance, particularly within machine learning applications. While much attention has been directed towards optimizing within Euclidean space, the need to optimize over spaces of probability measures in machine learning motivates exploration of accelerated gradient methods in this context too. To this end, we introduce a Hamiltonian-flow approach analogous to momentum-based approaches in Euclidean space. We demonstrate that, in the continuous-time setting, algorithms based on this approach can achieve convergence rates of arbitrarily high order. We complement our findings with numerical examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑