arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.04328cs.LG
收藏

双下降史前简史

A Brief Prehistory of Double Descent

University of Copenhagen(哥本哈根大学);Delft University of Technology(代尔夫特理工大学)

Marco Loog, Tom Viering, Alexander Mey, Jesse H. Krijthe, David M. J. Tax

摘要:

在他们发人深省的论文[1]中,Belkin等人说明并讨论了现代高复杂性学习者背景下的风险曲线形状。给定一个固定的训练样本大小$n$,这样的曲线显示了学习者的风险作为其复杂性$N$的一些(近似)度量的函数。当$N$表示特征数量时,这些曲线也称为特征曲线。[1]中的一个显著观察结果是,这些曲线可以显示他们所谓的双下降:随着$N$的增加,风险最初降低,达到最小值,然后增加,直到$N$等于$n$,此时训练数据完美拟合。进一步增加$N$,风险第二次也是最后一次降低,在$N=n$处产生峰值。这种双重的下降可能会让人感到惊讶,但与[1]报道的相反,它在历史上并没有被忽视。我们的信提请注意一些原始的,早期的发现,感兴趣的当代机器学习。

英文摘要:

In their thought-provoking paper [1], Belkin et al. illustrate and discuss the shape of risk curves in the context of modern high-complexity learners. Given a fixed training sample size $n$, such curves show the risk of a learner as a function of some (approximate) measure of its complexity $N$. With $N$ the number of features, these curves are also referred to as feature curves. A salient observation in [1] is that these curves can display, what they call, double descent: with increasing $N$, the risk initially decreases, attains a minimum, and then increases until $N$ equals $n$, where the training data is fitted perfectly. Increasing $N$ even further, the risk decreases a second and final time, creating a peak at $N=n$. This twofold descent may come as a surprise, but as opposed to what [1] reports, it has not been overlooked historically. Our letter draws attention to some original, earlier findings, of interest to contemporary machine learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑