随机调度下的重球方法

中文摘要 AI 辅助

我们研究了预定义的随机参数调度如何在一般光滑凸目标上加速重球方法。我们的分析区分了两个层次的随机化：在边界确定的时间间隔内随机选择梯度评估时间，以及额外随机化时间边界本身。对于确定性的时间边界，我们构造了固定时间和随时调度，实现了期望的最后迭代函数值间隙为$\mathcal{O}(1/K^{4/3})$阶；随时调度也几乎必然满足相同的速率。然后我们使用随机时间边界，获得了改进的最后迭代速率$\mathcal{O}(1/K^{3/2})$，无论是在期望意义上还是几乎必然意义上。据我们所知，这是重球方法在一般光滑凸函数上的首个全局非渐近收敛保证，其在多项式意义上优于经典的$\mathcal{O}(1/K)$速率。我们的结果表明，重球方法实现了严格优于普通梯度下降采用银步长调度所能达到的最佳已知结果$\mathcal{O}(1/K^{\log_2(1+\sqrt{2})})$的收敛速率。

英文摘要

We study how predefined randomized parameter schedules accelerate the heavy-ball method on general smooth convex objectives. Our analysis distinguishes two levels of randomization: sampling gradient-evaluation times within intervals whose boundaries are deterministic, and additionally randomizing the time boundaries themselves. With deterministic time boundaries, we construct fixed-time and anytime schedules that achieve an expected last-iterate function value gap of order $\mathcal{O}(1/K^{4/3})$; the anytime schedule also satisfies the same rate almost surely. We then use randomized time boundaries and obtain the improved last-iterate rate $\mathcal{O}(1/K^{3/2})$, both in expectation and almost surely. To the best of our knowledge, this is the first global nonasymptotic convergence guarantee for the heavy-ball method on general smooth convex functions that improves polynomially over the classical $\mathcal{O}(1/K)$ rate. Our result shows that the heavy-ball method achieves a strictly better convergence rate than the best known result $\mathcal{O}(1/K^{\log_2(1+\sqrt{2})})$ attainable by plain gradient descent with silver stepsize schedules.

发表机构

• University of Minnesota（明尼苏达大学）

机构由 AI 辅助整理，请以论文原文为准。