arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.07571cs.LGstat.ML
收藏

弱特征的两个双下降模型

Two models of double descent for weak features

Halıcıoğlu Data Science Institute, UC San Diego(加州大学圣迭戈分校哈勒哲奥卢数据科学研究所);Columbia University(哥伦比亚大学)

Mikhail Belkin, Daniel Hsu, Ji Xu

摘要:

提出了“双下降“风险曲线来定性描述变参数机器学习模型的样本外预测精度。这篇文章提供了一个精确的数学分析,在两个简单的数据模型与最小二乘/最小范数预测这条曲线的形状。具体而言,当特征数$p$接近样本量$n$时,风险达到峰值,但当$p$增加到超过$n$时,风险降低到最小值。这种行为与“有先见之明”的模型形成对比,这些模型以先验的最优顺序选择特征。

英文摘要:

The "double descent" risk curve was proposed to qualitatively describe the out-of-sample prediction accuracy of variably-parameterized machine learning models. This article provides a precise mathematical analysis for the shape of this curve in two simple data models with the least squares/least norm predictor. Specifically, it is shown that the risk peaks when the number of features $p$ is close to the sample size $n$, but also that the risk decreases towards its minimum as $p$ increases beyond $n$. This behavior is contrasted with that of "prescient" models that select features in an a priori optimal order.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑