arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2102.11537math.OCcs.LG
收藏

再论欧拉数值积分在凸优化加速和稳定性中的作用

Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization

ETH Zürich(苏黎世联邦理工学院);INRIA Paris(法国国家信息与自动化研究所巴黎分部)

Peiyuan Zhang, Antonio Orvieto, Hadi Daneshmand, Thomas Hofmann, Roy Smith

摘要:

将最优化方法看作是常微分方程的数值积分器,为研究加速一阶优化器提供了一个发人深省的现代框架。在这些文献中,加速度通常被认为与积分器的质量(准确性,能量守恒,辛性)有关。在这项工作中,我们提出了一个新的常微分方程,质疑这种连接:显式和半隐式(又名辛)欧拉离散这个ODE导致加速算法的凸规划。虽然半隐式方法是众所周知的数值分析享受许多理想的功能,物理系统的整合,我们的研究结果表明,这些属性不一定与加速度。

英文摘要:

Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often supposed to be linked to the quality of the integrator (accuracy, energy preservation, symplecticity). In this work, we propose a novel ordinary differential equation that questions this connection: both the explicit and the semi-implicit (a.k.a symplectic) Euler discretizations on this ODE lead to an accelerated algorithm for convex programming. Although semi-implicit methods are well-known in numerical analysis to enjoy many desirable features for the integration of physical systems, our findings show that these properties do not necessarily relate to acceleration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑