发表机构
University of Oxford（牛津大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对过参数化分类中梯度下降的隐含偏那次优问题，提出早停梯度下降方法，在带标签翻转噪声的高斯混合分类任务中实现极小极大最优额外零一风险，且早停所需样本量远少于线性插值器。
AI 中文摘要
在过参数化分类任务中，即使潜在分布并非线性可分，训练数据也可能线性可分。在此场景下，针对逻辑损失的梯度下降（GD）在范数上发散，却在方向上收敛到最大间隔插值分类器，其隐含偏好在统计层面可能并非最优。本研究表明，早停可克服该次优性：在带有标签翻转噪声的高斯混合模型中，在合适的神谕时刻停止的GD，对于具有快速连续衰减的协方差谱（包括多项式和指数谱衰减），可达到极小极大最优的额外零一风险。本分析结合了早停迭代的严格上界与任意分类器的匹配统计下界，得到的最优速率通过实验得到验证。一项核心技术贡献是新的校准结果，可将额外逻辑损失转化为额外零一风险；该结果处理了标签翻转噪声导致的模型误设，消除了标准界中的平方根速率。我们还建立了线性插值器的下界，表明为达到相同的额外风险，插值可能需要比早停多指数级的样本量。
英文摘要
In overparameterised classification, training data can be linearly separable even when the underlying distribution is not. In this setting, gradient descent (GD) on the logistic loss diverges in norm while converging in direction to a max-margin interpolating classifier, whose implicit bias can be statistically suboptimal. In this work, we show that early stopping can overcome this suboptimality: in a Gaussian mixture model with label-flipping noise, GD stopped at an appropriate oracle time achieves minimax-optimal excess zero-one risk for covariance spectra with fast and continuous decay, including polynomial and exponential spectral decays. Our analysis combines a sharp upper bound for the early-stopped iterate with a matching statistical lower bound over arbitrary classifiers, yielding optimal rates that are validated by experiments. A central technical contribution is a new calibration result that converts excess logistic risk into excess zero-one risk; it handles the model misspecification induced by the label-flipping noise, and removes the square-root rate in standard bounds. We also establish a lower bound for linear interpolators, showing that interpolation can require exponentially more samples than early stopping to achieve the same excess risk.
