常步长SGD的锐利平稳高斯逼近
Sharp Stationary Gaussian Approximation for Constant-Stepsize SGD
浏览论文内容
中文总结 AI 辅助
针对常步长SGD，证明了其不变分布与高斯分布的锐利逼近，结合分块比较与收缩技术，并通过四状态例子验证下界。
中文摘要 AI 辅助
我们证明了具有由外部一致遍历马尔可夫链生成的有界加性噪声的常步长SGD的不变律的锐利高斯逼近。对于具有Lipschitz Hessian和非退化长期噪声协方差的光滑强凸目标，以步长平方根归一化的中心化迭代在1-Wasserstein距离上与其极限高斯分布$O(\sqrt{\alpha})$-接近。该证明结合了分块高斯比较与长期收缩。一个四状态例子给出了匹配的下界，尽管单次噪声边际是对称的且所有非零滞后自协方差为零。在该例子中，相邻的三阶混合矩产生了主要修正。
英文摘要
We prove a sharp Gaussian approximation for the invariant law of constant-stepsize SGD with bounded additive noise generated by an exogenous uniformly ergodic Markov chain. For a smooth, strongly convex objective with a Lipschitz Hessian and nondegenerate long-run noise covariance, the centered iterate normalized by the square root of the stepsize is $O(\sqrtα)$-close in 1-Wasserstein distance to its limiting Gaussian. The proof combines blockwise Gaussian comparison with long-run contraction. A four-state example gives a matching lower bound although the one-time noise marginal is symmetric and every nonzero-lag autocovariance vanishes. In this example, an adjacent third-order mixed moment produces the leading correction.
发表机构
- PIT IN Corp.（PIT IN 公司）
机构由 AI 辅助整理，请以论文原文为准。
