arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

超越Nesterov的改进梯度下降下界

Improved Gradient Descent Lower Bounds Beyond Nesterov

Yuhan Ye, Kaizhao Liu

arXiv 2609.02855首次发表:更新:

发表机构

MIT(麻省理工学院)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究在光滑凸优化中,突破经典梯度下降一阶神谕下界,得到更优的非任意时间与任意时间梯度下降下界,还明确了两种场景收敛指数的严格区分。

AI 中文摘要

我们研究在光滑凸优化中,通过预定步长可将梯度下降(GD)加速到何种程度。超越Nemirovsky和Yudin提出的经典一阶神谕下界Ω(n⁻²),我们证明了非任意时间下界为Ω(n⁻¹·⁶³⁴²),任意时间下界为Ω(n⁻¹·²⁴⁰⁸)。这些结果分别改进了Ma和Chen近期提出的Ω(n⁻¹·⁹³²)非任意时间下界,以及Tsai等人提出的Ω(n⁻⁴/³)任意时间下界。结合银调度(silver schedules)实现的非任意时间O(n⁻^log₂(1+√2))速率,我们的任意时间下界在两种场景的可达收敛指数间建立了严格区分。

英文摘要

We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization. Going beyond the classical $Ω(n^{-2})$ first-order oracle lower bound of Nemirovsky and Yudin (1983), we prove an $Ω(n^{-1.6342})$ non-anytime lower bound and an $Ω(n^{-1.2408})$ anytime lower bound. These improve the recent $Ω(n^{-1.932})$ non-anytime lower bound of Ma and Chen (2026) and the $Ω(n^{-4/3})$ anytime lower bound of Tsai et al. (2026), respectively. Both results continue to hold when the stepsizes may be negative. Our anytime lower bound also shows that the $O(n^{-\log_2(1+\sqrt{2})})$ rate of non-anytime silver schedules (Altschuler and Parrilo, 2025; Grimmer et al., 2025) is unattainable in the anytime setting. This establishes a strict separation between the two settings.

Comments34 pages, 6 figures. This version extends the lower bounds to stepsize schedules that may include negative stepsizes

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑