arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.10322cs.LG
收藏

理解机器学习中的双下降曲线

Understanding the double descent curve in Machine Learning

INESC-ID(葡萄牙国家系统与计算机工程研究所);Higher Technical Institute, University of Lisbon(里斯本大学高等技术学院)

Luis Sa-Couto, Jose Miguel Ramos, Miguel Almeida, Andreas Wichert

摘要:

偏差-方差理论用于指导应用机器学习算法时的模型选择。然而,现代实践表明,过度参数化的模型是成功的,这些模型被期望过拟合,但没有。这导致了贝尔金等人提出的性能双下降曲线的建议。虽然它似乎描述了一个真实的,有代表性的现象,该领域缺乏一个基本的理论理解正在发生什么,什么是模型选择的后果,以及何时预计会发生双下降。在本文中,我们开发了一个原则性的理解的现象,并勾勒出这些重要问题的答案。此外,我们报告了我们提出的假设正确预测的真实实验结果。

英文摘要:

The theory of bias-variance used to serve as a guide for model selection when applying Machine Learning algorithms. However, modern practice has shown success with over-parameterized models that were expected to overfit but did not. This led to the proposal of the double descent curve of performance by Belkin et al. Although it seems to describe a real, representative phenomenon, the field is lacking a fundamental theoretical understanding of what is happening, what are the consequences for model selection and when is double descent expected to occur. In this paper we develop a principled understanding of the phenomenon, and sketch answers to these important questions. Furthermore, we report real experimental results that are correctly predicted by our proposed hypothesis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑