再论欧拉数值积分在凸优化加速和稳定性中的作用
Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization
摘要:
将最优化方法看作是常微分方程的数值积分器,为研究加速一阶优化器提供了一个发人深省的现代框架。在这些文献中,加速度通常被认为与积分器的质量(准确性,能量守恒,辛性)有关。在这项工作中,我们提出了一个新的常微分方程,质疑这种连接:显式和半隐式(又名辛)欧拉离散这个ODE导致加速算法的凸规划。虽然半隐式方法是众所周知的数值分析享受许多理想的功能,物理系统的整合,我们的研究结果表明,这些属性不一定与加速度。
英文摘要:
Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often supposed to be linked to the quality of the integrator (accuracy, energy preservation, symplecticity). In this work, we propose a novel ordinary differential equation that questions this connection: both the explicit and the semi-implicit (a.k.a symplectic) Euler discretizations on this ODE lead to an accelerated algorithm for convex programming. Although semi-implicit methods are well-known in numerical analysis to enjoy many desirable features for the integration of physical systems, our findings show that these properties do not necessarily relate to acceleration.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。