发表机构
Tsinghua University（清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究为配备Polyak步长和Armijo线搜索的随机重球方法建立了统一的收敛性分析，覆盖强凸、凸和非凸目标，并在插值或强增长条件下强化为几乎必然收敛，为该方法提供了更全面的理论保证。
AI 中文摘要
Polyak步长（PS）和Armijo线搜索（ALS）在随机优化中受到越来越多的关注，具有令人鼓舞的经验性能和理论保证。然而，它们对于随机重球（SHB）方法的收敛理论仍然有限。在这项工作中，我们为配备PS和ALS的SHB开发了统一的收敛性分析。为此，我们引入了一种与Polyak步长密切相似的修正Armijo规则，以及一种解耦分析，该分析隔离了由动量引起的历史依赖性。对于具有标准PS和ALS的SHB，我们在强凸、凸和非凸目标上建立了期望收敛，无需插值或对动量参数的限制性条件。在插值或强增长条件下，我们进一步将结果加强到几乎必然收敛速率和最后迭代收敛。此外，对于超出插值的一般设置，我们证明了对于具有PS和ALS的递减变体的SHB，几乎必然收敛到精确最优值或平稳点。这些结果为随机重球方法的Polyak步长和Armijo线搜索提供了更全面的理论视角。
英文摘要
Polyak step size (PS) and Armijo line search (ALS) have received increasing attention in stochastic optimization, with encouraging empirical performance and theoretical guarantees. However, their convergence theory for stochastic heavy ball (SHB) methods remains limited. In this work, we develop a unified convergence analysis for SHB equipped with PS and ALS. To this end, we introduce a modified Armijo rule that closely parallels the Polyak step size, together with a decoupling analysis that isolates the historical dependence induced by momentum. For SHB with standard PS and ALS, we establish expected convergence for strongly convex, convex, and non-convex objectives without interpolation or restrictive conditions on the momentum parameter. Under interpolation or strong growth, we further strengthen the results to almost sure rates and last-iterate convergence. Moreover, for general settings beyond interpolation, we prove almost sure convergence to the exact optimum or to stationarity for SHB with diminishing variants of PS and ALS. These results provide a more comprehensive theoretical view of Polyak step size and Armijo line search for stochastic heavy ball methods.
