最小化切比雪夫原型风险可神奇地缓解过拟合风险
Minimizing Chebyshev Prototype Risk Magically Mitigates the Perils of Overfitting
AI总结：
该研究提出切比雪夫原型风险（CPR）作为误分类概率的上界，基于此设计对数线性时间的exCPR损失函数，可有效缓解过参数化DNN的过拟合问题，在多种设置下性能优于现有方法。
摘要：
过参数化深度神经网络（DNN）若未经过充分正则化，容易对训练样本过拟合，无法很好地泛化到测试数据。为抑制过拟合，研究人员开发了多分量损失函数，用于在网络的一层或多层中降低类内特征相关性并最大化类间特征距离。通过分析DNN特征提取部分在线性分类器之前输出的倒数第二层特征层激活值，我们发现类内特征协方差与类间原型间距的修正形式是误分类概率的一个基本切比雪夫上界的关键组成部分，我们将该上界命名为切比雪夫原型风险（Chebyshev Prototype Risk，CPR）。以往方法的协方差损失项与网络特征数量呈二次方缩放关系，而我们的CPR界表明，对数线性时间的近似协方差损失足以降低该上界，并且可扩展到大型架构。我们将CPR界的各项实现到我们的显式CPR（explicit CPR，exCPR）损失函数中，通过在多个数据集和网络架构上的实验结果观察到，我们的训练算法可减少过拟合，并且在许多设置下优于以往方法。我们的代码可在https://github.com/Deano1718/Regularization_exCPR 获取。
英文摘要：
Overparameterized deep neural networks (DNNs), if not sufficiently regularized, are susceptible to overfitting their training examples and not generalizing well to test data. To discourage overfitting, researchers have developed multicomponent loss functions that reduce intra-class feature correlation and maximize inter-class feature distance in one or more layers of the network. By analyzing the penultimate feature layer activations output by a DNN's feature extraction section prior to the linear classifier, we find that modified forms of the intra-class feature covariance and inter-class prototype separation are key components of a fundamental Chebyshev upper bound on the probability of misclassification, which we designate the Chebyshev Prototype Risk (CPR). While previous approaches' covariance loss terms scale quadratically with the number of network features, our CPR bound indicates that an approximate covariance loss in log-linear time is sufficient to reduce the bound and is scalable to large architectures. We implement the terms of the CPR bound into our Explicit CPR (exCPR) loss function and observe from empirical results on multiple datasets and network architectures that our training algorithm reduces overfitting and improves upon previous approaches in many settings. Our code is available at https://github.com/Deano1718/Regularization_exCPR .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
