$(H_0,H_1)$-光滑与重尾噪声下的加速随机方法

中文摘要 AI 辅助

我们针对重尾噪声下的凸$(H_0,H_1)$-光滑优化问题，开发了一种加速随机方法。无偏随机梯度估计器具有有限的$p$阶噪声矩（$1<p\le2$），且包含常数项、依赖梯度的项和依赖间隙的项。我们的方法将加速更新与裁剪、投影和阶段重启相结合。每次迭代仅使用单个随机梯度样本，无需小批量处理。尽管存在裁剪偏差，我们仍证明了以高概率收敛到任意期望精度。确定性项对$H_0$和$H_1$均保持平方根依赖关系。在我们的三组分噪声模型中，强凸性仅对常数分量产生多项式精度代价；若无该分量，即使$p<2$，依赖关系也是对数级的。更一般地，对于噪声矩随$(f(x)-f^\star)^\alpha$缩放的情况，在凸目标下当$\alpha=p$时随机代价变为对数级，在强凸条件下当$\alpha=p/2$时亦然。

英文摘要

We develop an accelerated stochastic method for convex $(H_0,H_1)$-smooth optimization under heavy-tailed noise. The unbiased oracle has a finite $p$-th noise moment, $1<p\le2$, with constant, gradient-dependent, and gap-dependent terms. Our method combines accelerated updates with clipping, projection, and phase restarts. Each iteration uses a single stochastic-gradient sample, without minibatching. We prove high-probability convergence to any desired accuracy despite clipping bias. The deterministic terms retain square-root dependence on both $H_0$ and $H_1$. In our three-component noise model, strong convexity leaves a polynomial accuracy cost only for the constant component; without it, the dependence is logarithmic even for $p<2$. More generally, for noise moments scaling as $(f(x)-f^\star)^α$, the stochastic cost becomes logarithmic at $α=p$ for convex objectives and $α=p/2$ under strong convexity.

发表机构

• MSU AI Center（莫斯科国立大学人工智能中心）
• HSE University（高等经济大学）
• Innopolis University（伊诺波利斯大学）

机构由 AI 辅助整理，请以论文原文为准。