集成学习（Bagging）以线性样本复杂度学习VC类

中文摘要 AI 辅助

我们重新研究了在测试时学习对抗样本鲁棒预测器的问题。我们证明，VC类可通过与VC维$d$线性相关的样本复杂度实现对抗鲁棒学习，相比Montasser、Hanneke和Srebro（2019）给出的先前上界实现了指数级提升。值得注意的是，该结果通过一种简单的非恰当算法实现，该算法结合了Breiman（1996）提出的经典启发式方法Bagging（自助聚合）与鲁棒经验风险最小化（RERM）。我们的算法在$O(d^\text{★})$个独立自助样本上计算RERM，并输出它们的多数投票，其中$d^\text{★}$表示对偶VC维。我们补充了一个下界，表明这是不可避免的：在该oracle模型中，任何学习者通常需要$\boldsymbol{\text{Ω}}(d^\text{★})$次对RERM oracle的调用，即使给定任意多的训练样本。

英文摘要

We revisit the problem of learning predictors robust to adversarial examples at test-time. We prove that VC classes are adversarially robustly learnable with sample complexity linear in the VC dimension $d$, providing an exponential improvement over the previous upper bound of Montasser, Hanneke, and Srebro (2019). Remarkably, this result is achieved with a simple improper algorithm that combines the classic heuristic bagging (bootstrap aggregation) of Breiman (1996) with robust empirical risk minimization (RERM). Our algorithm computes RERMs on $O(d^\star)$ independent bootstrap samples and outputs their majority vote, where $d^\star$ denotes the dual VC dimension. We complement this result with a lower bound showing that this is unavoidable: in general, any learner in this oracle model requires $Ω(d^\star)$ calls to an RERM oracle, even when given arbitrarily many training examples.

发表机构

• Yale University（耶鲁大学）

机构由 AI 辅助整理，请以论文原文为准。