arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.04181cs.LGcs.NE
收藏

输入数据空间的随机性是推广的先验预测

The Randomness of Input Data Spaces is an A Priori Predictor for Generalization

Johannes Gutenberg-University(约翰内斯·古腾堡大学)

Martin Briesch, Dominik Sobania, Franz Rothlauf

摘要:

过参数化模型可以完美地学习各种类型的数据分布,然而,与人工数据相比,真实数据的泛化误差通常较低。这表明数据分布的性质对泛化能力有影响。这项工作的重点是由输入数据定义的搜索空间,并假设相邻输入值的标签之间的相关性影响泛化。如果相关性低,则输入数据空间的随机性高,导致高的泛化误差。我们建议使用Maurer的泛函数来度量输入数据空间的随机性。合成分类任务和常见图像分类基准(MNIST、CIFAR10和微软的猫与狗数据集)的结果发现,输入数据空间的随机性与深度神经网络的泛化误差之间存在高度相关性。

英文摘要:

Over-parameterized models can perfectly learn various types of data distributions, however, generalization error is usually lower for real data in comparison to artificial data. This suggests that the properties of data distributions have an impact on generalization capability. This work focuses on the search space defined by the input data and assumes that the correlation between labels of neighboring input values influences generalization. If correlation is low, the randomness of the input data space is high leading to high generalization error. We suggest to measure the randomness of an input data space using Maurer's universal. Results for synthetic classification tasks and common image classification benchmarks (MNIST, CIFAR10, and Microsoft's cats vs. dogs data set) find a high correlation between the randomness of input data spaces and the generalization error of deep neural networks for binary classification problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑