arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.15033math.OC
收藏

具有二次上界的凸函数的最优一阶方法

Optimal first-order methods for convex functions with a quadratic upper bound

Baptiste Goujaud, Adrien Taylor, Aymeric Dieuleveut

摘要:

我们分析了一阶最优化方法在光滑和凸函数的函数类上的最坏情况收敛保证。这个类包含允许一个简单的二次上界的凸函数。它的研究动机是它在小扰动下的稳定性。我们提供了一个彻底的分析一阶方法,包括最坏情况下的收敛保证几个算法,并证明其中一些实现了最佳的最坏情况下的保证类。我们支持我们的分析,最坏情况下的保证,使用性能估计问题的数值验证。从这个分析中可以得出一些观察结果,特别是关于最优性(分别是)。和自适应性)的重球方法(分别。重球与线搜索)。最后,我们展示了如何利用我们的分析来获得更复杂的函数类的收敛保证。总的来说,这项研究带来了对标准一阶方法具有最坏情况保证的函数类的选择的见解。

英文摘要:

We analyze worst-case convergence guarantees of first-order optimization methods over a function class extending that of smooth and convex functions. This class contains convex functions that admit a simple quadratic upper bound. Its study is motivated by its stability under minor perturbations. We provide a thorough analysis of first-order methods, including worst-case convergence guarantees for several algorithms, and demonstrate that some of them achieve the optimal worst-case guarantee over the class. We support our analysis by numerical validation of worst-case guarantees using performance estimation problems. A few observations can be drawn from this analysis, particularly regarding the optimality (resp. and adaptivity) of the heavy-ball method (resp. heavy-ball with line-search). Finally, we show how our analysis can be leveraged to obtain convergence guarantees over more complex classes of functions. Overall, this study brings insights on the choice of function classes over which standard first-order methods have working worst-case guarantees.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑