具有最优VC率的对抗鲁棒PAC学习
Adversarially Robust PAC Learning with Optimal VC Rates
浏览论文内容
中文总结 AI 辅助
研究对抗鲁棒PAC学习，提出二项式装袋算法，证明可实现与不可知设置下最优样本复杂度与经典PAC学习匹配，对抗鲁棒性不增加统计成本。
中文摘要 AI 辅助
我们研究对抗鲁棒PAC学习问题。在此框架中，学习者观察来自$\mathcal{X} \times \{0,1\}$上未知分布的独立样本，如同经典PAC学习。然而，给定学习者已知的扰动映射$\mathcal{U}: \mathcal{X} \to 2^{\mathcal{X}}$，目标是高概率地输出一个预测器，使其能正确分类来自同一底层分布的大多数未来样本$(x,y)$的每个扰动$z \in \mathcal{U}(x)$。我们在可实现和不可知两种设置下确定了该问题的最优$\mathcal{U}$无关样本复杂度。具体而言，对于每个VC维为$d$的概念类$\mathcal{H}$，我们证明了可实现设置中$\mathcal{O}(d/\epsilon + \log(1/\delta)/\epsilon)$的上界，以及不可知设置中$\mathcal{O}(d/\epsilon^2 + \log(1/\delta)/\epsilon^2)$的上界，并附带了后者的一阶最优改进。这些界与经典PAC学习的相应下界匹配。因此，或许令人惊讶的是，对抗鲁棒性在所有扰动映射上均匀地不产生额外的分布无关统计成本。我们的界在指数级别上优于[Montasser, Hanneke, and Srebro; COLT '19]的结果。在技术方面，我们基于一种称为二项式装袋的新算法原理，给出了简短而初等的证明。我们相信二项式装袋及其分析可能具有独立的意义。
英文摘要
We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distribution over $\mathcal{X} \times \{0,1\}$, as in classical PAC learning. However, given a perturbation map $\mathcal{U} : \mathcal{X} \to 2^{\mathcal{X}}$ known to the learner, the goal is to output, with high probability, a predictor that correctly classifies \emph{every} perturbation $z \in \mathcal{U}(x)$ of most future examples $(x,y)$ drawn from the same underlying distribution. We determine the \emph{optimal} $\mathcal{U}$-independent sample complexity of this problem in both the realizable and agnostic settings. More specifically, for every concept class $\mathcal{H}$ of $\operatorname{VC}$ dimension $d$, we prove upper bounds of $\mathcal{O} \big( d/ε+ \log(1/δ)/ε\big)$ in the realizable setting and $\mathcal{O} \big( d/ε^2 + \log(1/δ)/ε^2 \big)$ in the agnostic setting, together with an optimal first-order refinement of the latter. These bounds match the corresponding lower bounds for classical PAC learning. Consequently, and perhaps surprisingly, adversarial robustness incurs \emph{no additional} distribution-free statistical cost, uniformly over all perturbation maps. Our bounds improve exponentially on those of [Montasser, Hanneke, and Srebro; COLT '19]. On the technical side, we present short and elementary proofs based on a new algorithmic principle that we call \emph{binomial-bagging}. We believe that binomial-bagging and its analysis may be of independent interest.
发表机构
- Purdue University（普渡大学）
机构由 AI 辅助整理，请以论文原文为准。
