arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.26546math.OC
收藏

Nesterov加速梯度方法中梯度重启的非存在性

On the Nonoccurrence of Gradient Restart in Nesterov Accelerated Gradient Method

Tsinghua University(清华大学);Yanqi Lake Beijing Institute of Mathematical Sciences and Applications(北京雁栖湖应用数学研究院);Hunan University(湖南大学);The Hong Kong Polytechnic University(香港理工大学)

Chenglong Bao, Liang Chen, Jiahong Li, Defeng Sun

AI总结:

本文针对Nesterov加速梯度方法及其连续时间模型,构造了梯度重启从未发生的光滑强凸目标函数,否定了梯度重启可保证连续动力学线性收敛的开放问题。

摘要:

梯度重启通常可提升无约束凸优化问题的加速梯度方法的实际性能,但该提升的一般性理论保证仍不明确。本文构造了具有Lipschitz连续梯度的光滑强凸目标函数及对应初始点,对于Nesterov加速梯度方法及其常微分方程模型,梯度重启从未发生。由此得出结论:仅强凸性与光滑性无法保证梯度重启能改善加速方法或其连续时间模型的收敛行为。特别地,对于连续时间示例,目标函数间隙渐近为$1/(\pi t^3)$。因此,我们对Su、Boyd和Candès[《J. Mach. Learn. Res.》,2016,17(153),1--43]提出的开放问题——梯度重启是否能保证光滑强凸目标下连续动力学的线性收敛——给出否定答案。

英文摘要:

Gradient restarting often improves the practical performance of accelerated gradient methods for unconstrained convex optimization problems, but a general theoretical guarantee of such improvement remains unclear. In this paper, we construct smooth, strongly convex objectives with Lipschitz-continuous gradients and corresponding initial points for which gradient restart never occurs, both for Nesterov's accelerated gradient method and for its ordinary differential equation model. From these examples, we conclude that strong convexity and smoothness alone do not guarantee that gradient restart improves the convergence behavior of the accelerated method or its continuous-time model. In particular, for the continuous-time example, the objective gap is asymptotic to $1/(πt^3)$. We therefore obtain a negative answer to the open problem raised by Su, Boyd, and Cand{è}s [\textit{J. Mach. Learn. Res.}, 2016, 17(153), 1--43] of whether gradient restart guarantees linear convergence of the continuous dynamics for smooth strongly convex objectives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑