发表机构
University of Chicago（芝加哥大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究证明时间依赖可使SGD的平稳误差矩发散但仍保持高斯极限，通过构造设计、理论推导和实验验证，建立了超越矩分析的精确概率律近似。
AI 中文摘要
时间依赖性可以将随机梯度下降的高斯近似与其平稳矩分离开来。对于未修改的最小二乘SGD，我们构造了一个具有标准高斯边缘分布的设计，其平稳误差的每个正阶矩都是无穷大的。具有相同边缘分布的独立观测反而给出有限的平稳方差。两种机制都保持高斯小步长极限。我们的通用理论从有限的二阶设计矩建立路径收缩，然后利用得分消去和局部化获得平稳高斯和Ornstein-Uhlenbeck极限。独立高斯回归误差在相同设计可积性下产生精确的条件高斯律和全变差收敛。更强的设计条件确定一个正的一阶全变差常数和具有$o(a)$误差的确定性协方差修正。一个标量覆盖展开将此修正转化为其推断后果。实验检验了依赖得分下的分布误差、覆盖率和校准。这些结果共同建立了超越基于矩的平稳分析的概率律精确近似。
英文摘要
Temporal dependence can separate the Gaussian approximation of stochastic gradient descent from its stationary moments. For unmodified least-squares SGD, we construct a design with standard Gaussian marginals whose stationary error has every positive moment infinite. Independent observations with the same marginals instead give finite stationary variance. Both regimes retain a Gaussian small-step limit. Our general theory establishes pathwise contraction from a finite second design moment, then uses score cancellation and localization to obtain stationary Gaussian and Ornstein--Uhlenbeck limits. Independent Gaussian regression errors yield an exact conditional Gaussian law and total-variation convergence under the same design integrability. Stronger design conditions identify a positive first-order total-variation constant and a deterministic covariance correction with $o(a)$ error. A scalar coverage expansion translates this correction into its inference consequence. Experiments examine distributional error, coverage, and calibration with dependent scores. Together, these results establish precise probability-law approximation beyond moment-based stationary analysis.
