arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2005.00180cs.LGstat.ML
收藏

高维广义线性模型的泛化误差

Generalization Error of Generalized Linear Models in High Dimensions

University of California, Los Angeles(加州大学洛杉矶分校);New York University(纽约大学)

Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit, Sundeep Rangan, Alyson K. Fletcher

摘要:

机器学习的核心问题是学习规则在以前看不见的数据上的可推广性。虽然基于神经网络的过参数化模型现在在机器学习应用中无处不在,但我们对其泛化能力的理解是不完整的。由于底层学习问题的非凸性,这一任务变得更加困难。我们提供了一个一般框架来表征单层神经网络的渐近泛化误差(即,广义线性模型)与任意非线性,使其适用于回归以及分类问题。该框架使得能够分析(i)建模期间的过度参数化和非线性;以及(ii)学习期间的损失函数、初始化和正则化器的选择的影响。我们的模型还捕获了训练和测试分布之间的不匹配。作为例子,我们分析了几个特殊的情况,即线性回归和逻辑回归。我们也能够严格地和分析地解释广义线性模型中的\emph{double descent}现象。

英文摘要:

At the heart of machine learning lies the question of generalizability of learned rules over previously unseen data. While over-parameterized models based on neural networks are now ubiquitous in machine learning applications, our understanding of their generalization capabilities is incomplete. This task is made harder by the non-convexity of the underlying learning problems. We provide a general framework to characterize the asymptotic generalization error for single-layer neural networks (i.e., generalized linear models) with arbitrary non-linearities, making it applicable to regression as well as classification problems. This framework enables analyzing the effect of (i) over-parameterization and non-linearity during modeling; and (ii) choices of loss function, initialization, and regularizer during learning. Our model also captures mismatch between training and test distributions. As examples, we analyze a few special cases, namely linear regression and logistic regression. We are also able to rigorously and analytically explain the \emph{double descent} phenomenon in generalized linear models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑