arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03576cs.LGcs.CR

对抗训练在$\ell_0$有界对抗攻击下的泛化性质

Generalization Properties of Adversarial Training for $\ell_0$-Bounded Adversarial Attacks

University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pennsylvania(宾夕法尼亚大学);University of California, Santa Barbara(加州大学圣巴巴拉分校)

Payam Delgosha, Hamed Hassani, Ramtin Pedarsani

AI总结:

本文针对$\ell_0$有界对抗攻击,为截断分类器证明了分布无关的泛化界,通过新编码技术应对非线性和非凸挑战。

摘要:

我们广泛观察到神经网络容易受到输入的小幅加性扰动而导致错误分类。本文聚焦于$\ell_0$有界对抗攻击,旨在从理论上刻画一类重要的截断分类器的对抗训练性能。这类分类器在$\ell_0$对抗设置下,在高斯混合模型中已被证明具有强大的经验性能和理论性能。本文的主要贡献是证明了二分类设置下$\ell_0$有界对抗扰动的分布无关的新型泛化界。在此设置下推导泛化界面临两大挑战:(i) 高度非线性的截断内积;(ii) 对抗训练中对$\ell_0$球的极大化是非凸且高度非光滑的。为应对这些挑战,我们开发了新的编码技术来界定截断假设类的组合维度。

英文摘要:

We have widely observed that neural networks are vulnerable to small additive perturbations to the input causing misclassification. In this paper, we focus on the $\ell_0$-bounded adversarial attacks, and aim to theoretically characterize the performance of adversarial training for an important class of truncated classifiers. Such classifiers are shown to have strong performance empirically, as well as theoretically in the Gaussian mixture model, in the $\ell_0$-adversarial setting. The main contribution of this paper is to prove a novel generalization bound for the binary classification setting with $\ell_0$-bounded adversarial perturbation that is distribution-independent. Deriving a generalization bound in this setting has two main challenges: (i) the truncated inner product which is highly non-linear; and (ii) maximization over the $\ell_0$ ball due to adversarial training is non-convex and highly non-smooth. To tackle these challenges, we develop new coding techniques for bounding the combinatorial dimension of the truncated hypothesis class.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑