arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

梯度下降(非)任意时刻加速的更强下界

Stronger Lower Bounds for (Non-)Anytime Acceleration of Gradient Descent

Minchan Jung, Hanseul Cho, Chulhee Yun

arXiv 2609.04032首次发表:更新:

发表机构

Korea Science Academy of KAIST; Graduate School of AI, KAIST(韩国科学技术院科学高等研究院; 韩国科学技术院人工智能研究生院)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究针对梯度下降的非任意时刻与任意时刻设置,建立了比此前更强的收敛率下界,缩小了与对应上界之间的差距。

AI 中文摘要

在凸优化领域,固定步长梯度下降(GD)的速率最优收敛率对于L-利普希茨光滑凸目标函数,已有公认结果为Θ(N⁻¹)。令人惊讶的是,近期多项研究表明,采用非恒定、非自适应的确定性步长调度可加速普通GD。截至目前,非任意时刻与任意时刻设置下的最佳已知上界分别为O(N⁻¹.²⁷¹)(Altschuler和Parrilo,2025;Grimmer等人,2023)与O(N⁻¹.¹¹⁹)(Zhang等人,2025);而同期非任意时刻与任意时刻设置下的最佳报道下界(或壁垒)分别为Ω(N⁻¹.⁶³⁵)与Ω(N⁻¹.²⁴¹)(Ye和Liu,2026)。本文通过在两种设置下建立GD收敛率的更强下界缩小上述差距:非任意时刻速率界的下界为Ω(N⁻¹.⁴⁵⁰),任意时刻速率壁垒的下界为Ω(N⁻¹.¹⁸⁴)。

英文摘要

The rate-optimal convergence rate of gradient descent (GD) with a fixed step-size is well known to be $Θ(N^{-1})$ for $L$-Lipschitz smooth convex objectives in the prior art in convex optimization. Surprisingly, several recent works show that we can accelerate vanilla GD by applying a nonconstant, nonadaptive, deterministic step-size schedule. The best-known upper bounds so far in the non-anytime & anytime setups are $O(N^{-1.271})$ [Altschuler and Parrilo, 2025, Grimmer et al., 2023] and $O(N^{-1.119})$ [Zhang et al., 2025], respectively. On the other hand, the best reported lower bounds (or barriers) up to date in the non-anytime & anytime setups are $Ω(N^{-1.635})$ and $Ω(N^{-1.241})$ [Ye and Liu, 2026], respectively. We narrow these gaps by establishing stronger lower bounds for GD's convergence rate in both settings: $Ω(N^{-1.450})$ for the non-anytime rate bound and $Ω(N^{-1.184})$ for the anytime rate barrier.

Comments38 pages, 2 figures. Minor change in abstract, intro, & Fig. 1

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑