概率对抗训练

中文摘要 AI 辅助

基于一种概率视角，其中对抗样本源于基于距离的分布 $p_{\mathrm{dis}}$ 与受害者分类器诱导的分布 $p_{\mathrm{vic}}$ 之间的重叠，我们从一个简单的直觉出发：当这两个分布被推离时，对抗样本变得更难生成，因为它们的重叠变小，从而提高了鲁棒性。这一直觉自然激发了一个基于KL的鲁棒性目标。我们随后证明了 $\mathrm{KL}(p_{\mathrm{dis}}\\|p_{\mathrm{vic}})-\log Z_{\mathrm{vic}}$ 是概率鲁棒性（PR）的下界，其中 $Z_{\mathrm{vic}}$ 表示 $p_{\mathrm{vic}}$ 的归一化常数。由于PR通常难以直接计算，最大化这个基于KL的下界为改进PR提供了一个可处理的替代目标。我们进一步表明，该目标恢复了一种缩放形式的对抗训练，为对抗训练提供了概率解释和改进鲁棒性的原则性途径。我们将由此产生的方法称为概率对抗训练。实验表明，它持续改进PR，消融研究表明，诱导的缩放因子甚至能增强非概率对抗训练方法的PR。

英文摘要

Building on a probabilistic perspective in which adversarial examples arise from the overlap between a distance-based distribution $p_{\mathrm{dis}}$ and a victim-classifier-induced distribution $p_{\mathrm{vic}}$, we start from a simple intuition: adversarial examples become harder to generate when these two distributions are pushed apart, as their overlap becomes smaller, thereby increasing robustness. This intuition naturally motivates a KL-based robustness objective. We then prove that $\mathrm{KL}(p_{\mathrm{dis}}\|p_{\mathrm{vic}})-\log Z_{\mathrm{vic}}$ is a lower bound on probabilistic robustness (PR), where $Z_{\mathrm{vic}}$ denotes the normalizing constant of $p_{\mathrm{vic}}$. Since PR is generally intractable to compute directly, maximizing this KL-based lower bound provides a tractable surrogate objective for improving PR. We further show that this objective recovers a scaled form of adversarial training, offering a probabilistic interpretation of adversarial training and a principled route to robustness improvement. We call the resulting method probabilistic adversarial training. Experiments show that it consistently improves PR, and ablation studies demonstrate that the induced scaling factor can even enhance the PR of non-probabilistic adversarial training methods.

发表机构

• University of Warwick（华威大学）
• Wuhan University（武汉大学）

机构由 AI 辅助整理，请以论文原文为准。