UGM：用于光滑强凸优化中加速梯度方法的统一框架与新视角
UGM: A Unified Framework and New Perspectives for Accelerated Gradient Methods in Smooth and Strongly Convex Optimization
浏览论文内容
中文总结 AI 辅助
本文提出名为UGM的统一框架，将重球法与梯度下降结合，开发增强型加速梯度算法，经实验验证其在两类优化问题上性能优于基线方法。
中文摘要 AI 辅助
本文提出了一种名为UGM的加速梯度方法统一框架，它包含了大量用于最小化L-光滑和μ-强凸函数的加速及传统梯度类方法。我们证明该框架的迭代更新可被内在解释为重球法与普通梯度下降的混合组合，此解释揭示经典加速梯度方法本质上是将保守的梯度下降步骤融入快速但不稳定的重球动力学中，从而在加速性与稳定性间取得良好权衡。我们进一步利用李雅普诺夫函数建立统一收敛分析，基于该分析开发了一系列增强型加速梯度算法，这些算法利用梯度信息与迭代变量间的内积关系优化迭代更新。对无约束二次优化和逻辑回归的大量数值实验验证，所提算法在典型结构条件下较现有基线方法实现了更优性能。
英文摘要
In this paper, we propose a unified framework for accelerated gradient methods, dubbed UGM, which subsumes a wide range of accelerated and conventional gradient-type methods designed for minimizing $L$-smooth and $μ$-strongly convex functions. We demonstrate that the iteration update of the proposed framework can be intrinsically interpreted as a hybrid combination of the heavy-ball method and vanilla gradient descent. This interpretation reveals that classical accelerated gradient methods essentially integrate a conservative gradient descent step into the fast yet unstable heavy-ball dynamics, which achieves a favorable trade-off between acceleration and stability. We further establish a unified convergence analysis using Lyapunov functions. Guided by our analysis, we develop a family of enhanced accelerated gradient algorithms that leverage the inner product relationship between gradient information and iterative variables to optimize iterative updates. Extensive numerical experiments on unconstrained quadratic optimization and logistic regression validate that the proposed algorithms achieve superior performance compared with existing baseline methods under typical structural conditions.
