We develop a formalism for constructing stochastic upper bounds on the expected full sample risk for supervised classification tasks via the Hilbert coresets approach within a transductive framework. We explicitly compute tight and meaningful bounds for complex datasets and complex hypothesis classes such as state-of-the-art deep neural network architectures. The bounds we develop exhibit nice properties: i) the bounds are non-uniform in the hypothesis space, ii) in many practical examples, the bounds become effectively deterministic by appropriate choice of prior and training data-dependent posterior distributions on the hypothesis space, and iii) the bounds become significantly better with increase in the size of the training set. We also lay out some ideas to explore for future research.
Group-invariant Coresets for Data-efficient Active Learning
Universidade Federal de Santa Catarina(圣卡塔琳娜联邦大学);Universidade Católica de Pelotas(佩洛塔斯天主教大学);Université de Lorraine, CNRS, CRAN(洛林大学,法国国家科学研究中心,自动控制与信号处理研究中心)
Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces
Istituto Italiano di Tecnologia(意大利理工学院);University of Novi Sad(诺维萨德大学);University College London(伦敦大学学院);University of Genova(热那亚大学);Massachusetts Institute of Technology(麻省理工学院)
Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space
Vision and Image Processing Group, Systems Design Engineering, University of Waterloo, ON, Canada.(图像与图像处理组,系统设计工程,滑铁卢大学,加拿大);Electrical and Computer Engineering, University of Waterloo, ON Canada.(电气与计算机工程,滑铁卢大学,加拿大)
Regularized least squares learning with heavy-tailed noise is minimax optimal
Merantix Momentum;Institute for Mathematical Stochastics, Technische Universität Braunschweig(数学概率研究所,布拉unsch维格技术大学);Gatsby Computational Neuroscience Unit, University College London(盖茨计算神经科学单元,伦敦大学学院)
Understanding Robust Machine Learning for Nonparametric Regression with Heavy-Tailed Noise
Department of Mathematics and Statistics, State University of New York at Albany(纽约州立大学阿尔巴尼分校数学与统计学系);Department of Mathematics, University of Tennessee, Knoxville(田纳西大学基洛纳分校数学系)
EPFL(洛桑联邦理工学院);Swiss Finance Institute(瑞士金融研究所);Università di Genova(热那亚大学);CREST(CREST(法国国家统计与经济研究所下属研究中心));ENSAE - Institut Polytechnique de Paris(巴黎综合理工学院下属法国国立统计与经济管理学校);MIT(麻省理工学院);Istituto Italiano di Tecnologia(意大利技术研究院)