一致稳定性的锐尾
The Sharp Tail of Uniform Stability
查看机构详情
- Johns Hopkins University（约翰斯·霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文针对一致稳定性的泛化差距尾部分布问题，构造确定性γ-一致稳定学习问题填补了无对数上界与现有下界间的空白，确定了一致稳定性的最优高概率依赖关系。
中文摘要 AI 辅助
一致稳定性控制单个训练样本能在任意测试点改变多少损失。一种新的无对数上界表明，损失在[0,L]区间内的γ-一致稳定算法，以概率1-δ的泛化差距最多为O(γlog(1/δ)+L√(log(1/δ)/n))。实际有界损失学习算法能否实现与log(1/δ)的线性依赖关系仍未解决。已知构造仅对辅助弱依赖随机变量实现该线性依赖，其逐点范围随n增长；已知学习下界仅在恒定概率下成立。我们填补这一空白：对每个n、稳定性水平γ和损失界L，我们构造一个确定性γ-一致稳定学习问题，其尾部分布同时满足1≤p≤cn时，P(R(A_S)-R_S(A_S)≥c'min{L,γp+L√(p/n)})≥e^{-p}。该构造是带常数标签的普通有界绝对损失回归，关键在于多尺度的稀有Rademacher特征集合：逐坐标斜坡在sup范数下稳定，而奇对称最大值将唯一极端特征转化为γp量级的差距且不违反损失界；几何间隔的斜坡将所有置信水平纳入同一问题。结合无对数上界，这确定了一致稳定性的最优高概率和矩依赖关系（仅差普适常数）。
英文摘要
Uniform stability controls how much one training example can change the loss at any test point. A new logarithmic-free upper bound shows that a $γ$-uniformly stable algorithm with loss in $[0,L]$ has generalization gap at most $O \left(γ\log(1/δ) +L\sqrt{\frac{\log(1/δ)}{n}}\right)$ with probability $1-δ$. Whether an actual bounded-loss learning algorithm can realize the linear dependence on $\log(1/δ)$ has remained open. The known construction realizes it only for auxiliary weakly dependent random variables whose pointwise range grows with $n$. The known learning lower bound holds only at constant probability. We close this gap. For every $n$, stability level $γ$, and loss bound $L$, we construct one deterministic $γ$-uniformly stable learning problem whose tail satisfies, simultaneously for $1\le p\le c n$, $\mathbb P \left( R(A_S)-R_S(A_S) \ge c'\min \left\{L,γp+L\sqrt{p/n}\right\} \right)\ge e^{-p}.$ The construction is ordinary bounded absolute-loss regression with constant labels. Its key is a multiscale collection of rare Rademacher features. A coordinatewise ramp is stable in sup norm, while an odd symmetrized maximum converts a unique extreme feature into a gap of order $γp$ without violating the loss bound. Geometrically spaced ramps put all confidence levels into the same problem. Together with the logarithmic-free upper bound, this determines the optimal high-probability and moment dependence of uniform stability up to universal constants.
