无对数项的一致稳定算法的矩与泛化界
Logarithmic-Free Moment and Generalization Bounds for Uniformly Stable Algorithms
- VinUniversity（范安大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究针对一致稳定算法的泛化界问题，移除了前人结果中的log n因子，通过Rademacher立方体估计与双副本随机化论证，得到了更紧的矩上界。
AI中文摘要：
一致稳定性是控制学习算法泛化误差的经典工具。Bousquet、Klochkov和Zhivotovskiy（2020）表明，该问题可简化为独立随机变量的弱相互作用函数之和的矩不等式，其界包含额外的log n因子，并提出该因子能否被移除的问题。我们肯定地回答了这个上界问题。具体而言，设Z=(Z₁,…,Zₙ)具有独立坐标，且gᵢ(Z)满足：对所有i=1,̄,n，E[gᵢ(Z)|Z_{-i}]=0，|E[gᵢ(Z)|Zᵢ]|≤M，且改变任意坐标Zⱼ（j≠i）时gᵢ的变化不超过β，其中Z_{-i}表示除Zᵢ外的所有坐标。我们证明，对每个p≥2，‖∑ᵢ=1ⁿgᵢ(Z)‖ₚ≤16pnβ+M√(2pn)。这移除了先前界中的log n因子，且在Bousquet等人构造的范围内，与他们的下界仅相差通用常数。我们的证明首先在Rademacher立方体上建立所需估计，再通过双副本随机化论证将其推广到任意乘积分布。
英文摘要：
Uniform stability is a classical tool for controlling the generalization error of a learning algorithm. Bousquet, Klochkov, and Zhivotovskiy (2020) showed that the problem can be reduced to a moment inequality for a sum of weakly interacting functions of independent random variables. Their bound contains an additional factor $\log n$, and they asked whether this factor can be removed. We answer this upper-bound question affirmatively. More specifically, let $Z=(Z_1,\ldots,Z_n)$ have independent coordinates and let $g_i(Z)$ satisfy $\mathbb E[g_i(Z)\mid Z_{-i}]=0, \ \left| \mathbb E[g_i(Z)\mid Z_i]\right|\le M, \ \text{for every } i = 1, \dots, n, $ where $Z_{-i}$ denotes all coordinates except $Z_i$. Assume additionally that changing any coordinate $Z_j$, $j\neq i$, changes $g_i$ by at most $β$, we prove that, for every $p\ge2$, for every $p\ge2$, $$ \left\| \sum_{i=1}^n g_i(Z)\right\|_p \le 16pnβ+M\sqrt{2pn}. $$ This removes the $\log n$ factor from the previous bound and matches the lower bound of Bousquet, Klochkov, and Zhivotovskiy up to universal constants in the range covered by their construction. Our proof first establishes the required estimate on the Rademacher cube, then transfers it to arbitrary product distributions by a two-copy randomization argument.
