AI 中文总结
针对距离相关矩下的非凸SGD,证明了有限时域平稳性界并匹配BG-0下界,无需修改算法,并给出高概率界。
AI 中文摘要
均匀噪声矩界排除了随机梯度变异性随迭代增加的情况。我们研究了在距离相关条件矩下,针对光滑、下界、可能非凸的目标函数的普通单样本随机梯度下降。仅基于二阶矩,直接的下降-位移论证给出了$T^{-1/3}$期望平均梯度平方平稳性,并采用依赖于时域的步长。一个显式的预言机复杂度推论匹配了已知的光滑Blum-Gladyshev (BG-0)下界,包括$Lb_2\Delta^3\varepsilon^{-6}$和$L\Delta\sigma^2\varepsilon^{-4}$随机项,其中$\Delta$是初始目标间隙,$\sigma^2+b_2\\|x-x_1\\|^2$界定方差。因此,未修改的SGD在该二阶矩类中达到了极小极大随机复杂度。对于$p>2$,可预测的局部化和Hilbert空间Fuk-Nagaev不等式产生了一个高概率界,分离了对数方差和多项式稀有冲击贡献。局部化半径由递归推导:无需有界迭代假设、裁剪、归一化、动量或增加批量大小。我们还给出了递增置信度速率、恢复根$T$平稳性的目标间隙增长细化,以及随机$L^p$-Lipschitz示例。广泛的BG-0最优性陈述与较小的均方光滑类区分开来,在后者中额外的预言机结构允许更快的算法。
英文摘要
Uniform noise-moment bounds exclude stochastic gradients whose variability increases with the iterate. We study ordinary, single-sample stochastic gradient descent for smooth, lower-bounded, possibly nonconvex objectives under distance-dependent conditional moments. Under second moments alone, a direct descent--displacement argument yields $T^{-1/3}$ expected average squared-gradient stationarity with a horizon-dependent stepsize. An explicit oracle-complexity corollary matches the known smooth Blum--Gladyshev (BG-0) lower bound, including the $Lb_2Δ^3\varepsilon^{-6}$ and $LΔσ^2\varepsilon^{-4}$ stochastic terms, where $Δ$ is the initial objective gap and $σ^2+b_2\|x-x_1\|^2$ bounds the variance. Thus unchanged SGD attains the minimax stochastic complexity in this second-moment class. For $p>2$, predictable localization and a Hilbert-space Fuk--Nagaev inequality yield a high-probability bound separating logarithmic variance and polynomial rare-shock contributions. The localization radius is derived from the recursion: no bounded-iterate assumption, clipping, normalization, momentum, or increasing batch size is needed. We also give increasing-confidence rates, an objective-gap-growth refinement recovering root-$T$ stationarity, and stochastic $L^p$-Lipschitz examples. The broad BG-0 optimality statement is distinguished from the smaller mean-square-smooth class, in which additional oracle structure permits faster algorithms.