发表机构
Johns Hopkins University; Massachusetts Institute of Technology; Georgia Institute of Technology(约翰斯·霍普金斯大学; 麻省理工学院; 佐治亚理工学院)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
本文针对光滑凸优化中固定步长一阶方法,通过代数证明确立了OGM、OGM-G及Lemniscate方法在三种性能度量下的最优性及其唯一性。
AI 中文摘要
本文考虑高维$L$-光滑凸函数最小化中固定步长一阶方法的设计问题。对于通过最终函数值的次优性(相对于到最小化器的初始平方距离)来衡量的最坏情况性能优化,我们提供了优化梯度法(OGM)最优性的代数证明,并确立了其在所有固定步长一阶方法中的唯一性。对于最终平方梯度范数(相对于初始次优性)的替代度量,我们证明OGM-G方法是最优的,并且在固定步长一阶方法中具有唯一性。最后,对于衡量最终平方梯度范数(相对于到最小化器的初始平方距离)的设置,我们展示了最近提出的Lemniscate方法是最优且唯一的。我们的证明依赖于下界论证的代数化简,而非传统的信息论界限,后者先前仅能确立OGM的最优性而无法证明其唯一性。
英文摘要
This paper considers the design of optimal fixed-step first-order methods for high-dimensional minimization of $L$-smooth convex functions. For optimizing worst-case performance measured via suboptimality of the final function value (relative to the initial squared distance to a minimizer), we provide an algebraic proof of the optimality of the optimized gradient method (OGM) and establish its uniqueness among all fixed-step first-order methods. For the alternative measure of final squared gradient norm (relative to initial suboptimality), we prove the OGM-G method is optimal and uniquely so among fixed-step first-order methods. Finally, for the setting measuring the final squared gradient norm (relative to the initial squared distance to a minimizer), we show the recently proposed Lemniscate method is optimal and uniquely so. Our proofs rely on algebraic reductions for lower bound arguments rather than traditional information-theoretic bounds, which were previously only able to establish OGM's optimality but not uniqueness.