过参数化区域下部分量化神经网络的VC维数
VC dimension of partially quantized neural networks in the overparametrized regime
摘要：
Vapnik-Chervonenkis（VC）理论至今无法解释过参数化神经网络的小泛化误差。事实上，VC理论在大型网络中的现有应用获得了与权重数量成比例的VC维度上界，并且对于一大类网络，这些上界已知是紧的。在这项工作中，我们专注于一类部分量化的网络，我们称之为超平面排列神经网络（HANNs）。使用一个样本压缩分析，我们表明，HANNs可以有VC维显着小于权重的数量，同时具有高度的表达能力。特别是，经验风险最小化超过HANNs在overparametrized制度实现了极大极小率与Lipschitz后验概率分类。我们进一步证明了经验的表达HANNs。在121个UCI数据集的面板上，过度参数化的HANN与最先进的全精度模型的性能相匹配。
英文摘要：
Vapnik-Chervonenkis (VC) theory has so far been unable to explain the small generalization error of overparametrized neural networks. Indeed, existing applications of VC theory to large networks obtain upper bounds on VC dimension that are proportional to the number of weights, and for a large class of networks, these upper bound are known to be tight. In this work, we focus on a class of partially quantized networks that we refer to as hyperplane arrangement neural networks (HANNs). Using a sample compression analysis, we show that HANNs can have VC dimension significantly smaller than the number of weights, while being highly expressive. In particular, empirical risk minimization over HANNs in the overparametrized regime achieves the minimax rate for classification with Lipschitz posterior class probability. We further demonstrate the expressivity of HANNs empirically. On a panel of 121 UCI datasets, overparametrized HANNs match the performance of state-of-the-art full-precision models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
