arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.09134cs.LGmath.STstat.MLstat.TH
收藏

高斯与非高斯数据增强的普遍性

Gaussian and Non-Gaussian Universality of Data Augmentation

University of Warwick(华威大学);University College London(伦敦大学学院);Harvard University(哈佛大学)

Kevin Han Huang, Peter Orbanz, Morgane Austern

AI总结:

研究揭示数据增强对估计不确定性的影响取决于数据分布、估计量性质及样本量等因素,可能产生高斯或非高斯普遍性结果。

摘要:

我们提供了普遍性结果,量化了数据增强如何通过简单的替代品影响估计量的方差和极限分布,并详细分析了几种特定模型。这些结果证实了一些在机器学习实践中观察到的发现,但也导致了意想不到的发现:数据增强可能会增加而不是减少估计的不确定性,例如经验预测风险。它可以作为正则化器,但在某些高维问题中却无法做到这一点,并且它可能会移动经验风险的双下降峰。总体而言,分析显示,数据增强所赋予的几种属性并非绝对真实或虚假,而是取决于多种因素——特别是数据分布、估计量的性质以及样本量、增强次数和维度之间的相互作用。作为我们主要的理论工具,我们开发了块依赖性的一种Lindeberg技术适应。所得到的普遍性领域可能是高斯或非高斯的。

英文摘要:

We provide universality results that quantify how data augmentation affects the variance and limiting distribution of estimates through simple surrogates, and analyze several specific models in detail. The results confirm some observations made in machine learning practice, but also lead to unexpected findings: Data augmentation may increase rather than decrease the uncertainty of estimates, such as the empirical prediction risk. It can act as a regularizer, but fails to do so in certain high-dimensional problems, and it may shift the double-descent peak of an empirical risk. Overall, the analysis shows that several properties data augmentation has been attributed with are not either true or false, but rather depend on a combination of factors -- notably the data distribution, the properties of the estimator, and the interplay of sample size, number of augmentations, and dimension. As our main theoretical tool, we develop an adaptation of Lindeberg's technique for block dependence. The resulting universality regime may be Gaussian or non-Gaussian.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑