arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.11199math.OC
收藏

强凸函数最快一阶最优化方法的高分辨率建模

High-Resolution Modeling of the Fastest First-Order Optimization Method for Strongly Convex Functions

Boya Sun, Jemin George, Solmaz Kia

摘要:

基于梯度优化算法可以从极限常微分方程(ODE)的角度进行研究这一事实,本文推导了加速三动量(TM)算法的ODE表示.对于具有强凸代价的无约束最优化问题,TM算法具有比Nesterov加速梯度(NAG)方法更快的收敛速度,但具有相同的计算复杂性.我们表明,类似于NAG方法准确地捕捉TM方法的特点,我们需要使用高分辨率建模,以获得TM算法的ODE表示。我们使用李雅普诺夫分析研究的稳定性和收敛行为的建议的高分辨率常微分方程表示的TM算法。我们通过这种分析表明,这种ODE模型具有鲁棒性的TM算法的参数偏差。我们比较了TM方法的ODE表示的速度与NAG方法,以确认其更快的收敛速度。我们的研究也导致了NAG方法的ODE模型的最差收敛速率的更严格的限制。最后,我们讨论了使用积分二次约束(IQC)的方法来建立一个估计的收敛速率的TM算法。一个数值例子证明了我们的结果。

英文摘要:

Motivated by the fact that the gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs), here we derive an ODE representation of the accelerated triple momentum (TM) algorithm. For unconstrained optimization problems with strongly convex cost, the TM algorithm has a proven faster convergence rate than the Nesterov's accelerated gradient (NAG) method but with the same computational complexity. We show that similar to the NAG method to capture accurately the characteristics of the TM method, we need to use a high-resolution modeling to obtain the ODE representation of the TM algorithm. We use a Lyapunov analysis to investigate the stability and convergence behavior of the proposed high-resolution ODE representation of the TM algorithm. We show through this analysis that this ODE model has robustness to deviation from the parameters of the TM algorithm. We compare the rate of the ODE representation of the TM method with that of the NAG method to confirm its faster convergence. Our study also leads to a tighter bound on the worst rate of convergence for the ODE model of the NAG method. Lastly, we discuss the use of the integral quadratic constraint (IQC) method to establish an estimate on the rate of convergence of the TM algorithm. A numerical example demonstrates our results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑