arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.00673math.NAcs.LGcs.NAmath.OC
收藏

某些optimization算法的李雅普诺夫函数与微分方程的联系

The connections between Lyapunov functions for some optimization algorithms and differential equations

J. M. Sanz-Serna, Konstantinos C. Zygalakis

摘要:

本文研究了一类二阶阻尼微分方程的性质、离散化及其与$m$ -强凸函数和$L$ -光滑函数的加速优化算法的联系.特别是,使用线性矩阵不等式LMI框架开发的\emph{Fazlyab et. al. $(2018)$},我们推导出解析的(离散)李雅普诺夫函数的两个参数的家庭Nesterov最优化方法,它允许其收敛速度的完整表征。在适当的限制,这一系列的方法可以被看作是一个家庭的二阶常微分方程,我们构造(连续)李雅普诺夫函数的LMI框架的离散化。连续的李雅普诺夫函数也可以通过研究其离散对应物的极限行为来获得。最后,我们表明,大多数的常微分方程的家庭,如重球方法的典型离散化,不具备李雅普诺夫函数的性质类似于这里构造的Nesterov方法的李雅普诺夫函数。

英文摘要:

In this manuscript, we study the properties of a family of second-order differential equations with damping, its discretizations and their connections with accelerated optimization algorithms for $m$-strongly convex and $L$-smooth functions. In particular, using the Linear Matrix Inequality LMI framework developed by \emph{Fazlyab et. al. $(2018)$}, we derive analytically a (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods, which allows for the complete characterization of their convergence rate. In the appropriate limit, this family of methods may be seen as a discretization of a family of second-order ordinary differential equations for which we construct(continuous) Lyapunov functions by means of the LMI framework. The continuous Lyapunov functions may alternatively, be obtained by studying the limiting behaviour of their discrete counterparts. Finally, we show that the majority of typical discretizations of the family of ODEs, such as the Heavy ball method, do not possess Lyapunov functions with properties similar to those of the Lyapunov function constructed here for the Nesterov method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑