Polyak动量的普适平均-情形最优性
Universal Average-Case Optimality of Polyak Momentum
摘要：
Polyak momentum（PM），也称为重球方法，是一种广泛使用的最优化方法，在二次目标上具有渐近最优的最坏情况复杂度。然而，其显着的经验成功并不能完全解释这种最优性，作为最坏情况下的分析-相反的平均情况-是不代表的预期复杂性的算法。在这项工作中，我们建立了一个新的PM和平均情况下的分析之间的联系。我们的主要贡献是证明，任何最佳的平均情况下的方法收敛于PM的迭代次数，在温和的假设。这为这种经典方法带来了一个新的视角，表明PM是渐近最坏情况和平均情况下的最优。
英文摘要：
Polyak momentum (PM), also known as the heavy-ball method, is a widely used optimization method that enjoys an asymptotic optimal worst-case complexity on quadratic objectives. However, its remarkable empirical success is not fully explained by this optimality, as the worst-case analysis -- contrary to the average-case -- is not representative of the expected complexity of an algorithm. In this work we establish a novel link between PM and the average-case analysis. Our main contribution is to prove that any optimal average-case method converges in the number of iterations to PM, under mild assumptions. This brings a new perspective on this classical method, showing that PM is asymptotically both worst-case and average-case optimal.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
