发表机构
Fudan University; Shanghai Center for Mathematical Sciences（复旦大学; 上海数学科学中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究证明标准SGD在光滑凸目标上的时间一致收敛速率可任意接近但达不到$\sqrt{\log n/n}$，并给出精确的充要条件刻画可达速率边界。
AI 中文摘要
我们研究了无约束光滑凸目标下标准随机梯度下降（SGD）原始迭代的时间一致收敛性。我们证明，在标准噪声假设下，时间一致收敛速率可以任意接近 $\sqrt{\log n / n}$，但永远无法达到该速率。更具体地说，我们证明：对于每个正的、最终非递减的序列 $h$，若满足 $h(n) = o(\sqrt{n})$，则存在一个以概率至少 $1-\alpha$ 同时对所有 $n$ 成立且在整个问题类上一致的 $h(n)/\sqrt{n}$ 阶界，当且仅当 \\[ \sum_{j = 1}^{\infty} \frac{1}{h(2^j)^2} < \infty. \\] 构造性的充分性结果来自一个二进制的无水平线调度以及一个加性条件重启不等式。必要性部分适用于每个确定性非负调度，并且即使对于带高斯噪声的一维解析光滑凸目标也成立。
英文摘要
We study the time-uniform convergence of the raw iterate of standard stochastic gradient descent (SGD) for unconstrained smooth convex objectives. We prove that, under standard noise assumptions, the time-uniform convergence rate gets arbitrarily close to $\sqrt{\log n / n}$ but never reaches it. More specifically, we prove that for every positive, eventually nondecreasing sequence $h$ satisfying $h(n) = o(\sqrt{n})$, a bound of order $h(n)/\sqrt{n}$, holding simultaneously for all $n$ with probability at least $1-α$ and uniformly over the problem class, is achievable if and only if \[ \sum_{j = 1}^{\infty} \frac{1}{h(2^j)^2} < \infty. \] The constructive sufficiency result follows from a dyadic horizon-free schedule together with an additive conditional-restart inequality. The necessity counterpart applies to every deterministic nonnegative schedule and holds even for a one-dimensional analytic smooth convex objective with Gaussian noise.
Comments28 pages, including appendix
