We study the learnability of linear separators in $\Re^d$ in the presence of bounded (a.k.a Massart) noise. This is a realistic generalization of the random classification noise model, where the adversary can flip each example $x$ with probability $η(x) \leq η$. We provide the first polynomial time algorithm that can learn linear separators to arbitrarily small excess error in this noise model under the uniform distribution over the unit ball in $\Re^d$, for some constant value of $η$. While widely studied in the statistical learning theory community in the context of getting faster convergence rates, computationally efficient algorithms in this model had remained elusive. Our work provides the first evidence that one can indeed design algorithms achieving arbitrarily small excess error in polynomial time under this realistic noise model and thus opens up a new and exciting line of research.
We additionally provide lower bounds showing that popular algorithms such as hinge loss minimization and averaging cannot lead to arbitrarily small excess error under Massart noise, even under the uniform distribution. Our work instead, makes use of a margin based technique developed in the context of active learning. As a result, our algorithm is also an active learning algorithm with label complexity that is only a logarithmic the desired excess error $ε$.
Achieving $\widetilde{O}(1/ε)$ Sample Complexity for Bilinear Systems Identification under Bounded Noises
Department of Electrical and Computer Engineering, University of Washington（华盛顿大学电气与计算机工程系）；Cornell University AI for Science Institute, Cornell University（康奈尔大学AI for Science研究所）
Optimal uncertainty bounds for multivariate kernel regression under bounded noise: A Gaussian process-based dual function
Institute for Dynamical Systems and Control, ETH Zurich（动态系统与控制研究所，苏黎世联邦理工学院）；Department of Electrical Engineering, Chalmers University of Technology（电气工程系，查尔姆斯理工大学）；Department of Mechanical Engineering, Imperial College London（机械工程系，伦敦帝国理工学院）
