We provide universality results that quantify how data augmentation affects the variance and limiting distribution of estimates through simple surrogates, and analyze several specific models in detail. The results confirm some observations made in machine learning practice, but also lead to unexpected findings: Data augmentation may increase rather than decrease the uncertainty of estimates, such as the empirical prediction risk. It can act as a regularizer, but fails to do so in certain high-dimensional problems, and it may shift the double-descent peak of an empirical risk. Overall, the analysis shows that several properties data augmentation has been attributed with are not either true or false, but rather depend on a combination of factors -- notably the data distribution, the properties of the estimator, and the interplay of sample size, number of augmentations, and dimension. As our main theoretical tool, we develop an adaptation of Lindeberg's technique for block dependence. The resulting universality regime may be Gaussian or non-Gaussian.
Gaussian Universality of Perceptrons with Random Labels
International School of Advanced Studies (SISSA)(国际高等研究院(SISSA));EPFL(洛桑联邦理工学院);École Normale Supérieure (ENS) - PSL(巴黎高等师范学院(PSL大学成员));CNRS(法国国家科学研究中心)
Department of Electrical Engineering, California Institute of Technology(电气工程系,加利福尼亚理工学院);Department of Electrical and Computer Engineering, University of Texas at Austin(电气与计算机工程系,德克萨斯大学奥斯汀分校);Department of Computing and Mathematical Sciences, California Institute of Technology(计算与数学科学系,加利福尼亚理工学院)
Gaussian Universality in Neural Network Dynamics with Generalized Structured Input Distributions
Department of Physics, Korea Advanced Institute of Science and Technology(韩国科学技术院物理系);Center of Complex Systems, Korea Advanced Institute of Science and Technology(韩国科学技术院复杂系统中心)
Universality of Gaussian-Mixture Reverse Kernels in Conditional Diffusion
Center for Mathematics and Interdisciplinary Sciences, Fudan University(复旦大学数学与交叉科学中心);Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS)(上海数学与交叉科学研究院(SIMIS));Northeastern University(东北大学);Infectious Diseases Division, icddr,b(icddr,b传染病部门);Rutgers University(罗格斯大学)
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China;Huawei Noah's Ark Lab, Huawei Technologies, Paris, France;School of Electronic Information;Communications, Huazhong University of Science \& Technology, China
Deterministic equivalent and error universality of deep random features learning
ETH Zurich(苏黎世联邦理工学院);ETH AI Center(苏黎世联邦理工学院人工智能中心);École Normale Supérieure (ENS) - PSL(巴黎高等师范学院(巴黎文理研究大学));CNRS(法国国家科学研究中心);École Polytechnique Fédérale de Lausanne (EPFL)(洛桑联邦理工学院)
On the Multidimensional Augmentation of Fingerprint Data for Indoor Localization in A Large-Scale Building Complex Based on Multi-Output Gaussian Process
University of Liverpool(利物浦大学);Xi’an Jiaotong-Liverpool University(西交利物浦大学)
Data Augmentation for Histopathological Images Based on Gaussian-Laplacian Pyramid Blending
École de Technologie Superiéure, Université du Québec(魁北克大学高等技术学院);State University of Ponta Grossa (UEPG)(蓬塔格罗萨州立大学);Pontifical Catholic University of Paraná(巴拉那天主教大学);Federal University of Paraná(巴拉那联邦大学)
On Universality of Non-Separable Approximate Message Passing Algorithms
Department of Statistics and Data Science, Yale University(统计与数据科学系,耶鲁大学);Halicioglu Data Science Institute, University of California, San Diego(Halicioglu数据科学研究所,加州大学圣地亚哥分校)
Binary Gaussian Copula Synthesis: an LLM-powered data augmentation framework for early dialysis prediction in chronic kidney disease
Department of Industrial & Management Systems Engineering, West Virginia University(威斯康星大学工业与管理系统工程系);Department of Electrical and Computer Engineering, Concordia University(康科迪亚大学电气与计算机工程系);Department of Computer Science, University of California, Davis(加州大学戴维斯分校计算机科学系);School of Mathematical & Data Sciences, West Virginia University(威斯康星大学数学与数据科学学院);School of Systems Science and Industrial Engineering, The State University of New York at Binghamton(纽约州立大学布法罗分校系统科学与工业工程学院);H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology(佐治亚理工学院H.米尔顿·斯图尔特工业与系统工程学院)