发表机构
City University of Hong Kong; Georgia Institute of Technology（香港城市大学; 佐治亚理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究从RKHS视角结合核积分算子分析对抗训练，推导泛化误差界并发现其统计精度损失源于鲁棒性与噪声的相互作用，提出两阶段去噪方法提升泛化速率。
AI 中文摘要
对抗训练已成为保护模型抵御各类实际应用中对抗攻击的有效方法。本文在再生核希尔伯特空间（RKHS）框架下，通过关联的核积分算子研究对抗训练。我们首先根据鲁棒性水平、样本量、源平滑性及核谱，推导了RKHS对抗训练估计量的源一致泛化误差界。在固定多项式谱模型上，我们进一步建立了匹配的下界，表明最优平衡的泛化速率可能慢于极小极大预测基准，该结果揭示了对抗训练中统计精度的损失。我们的分析显示，这种损失源于对抗鲁棒性与观测噪声的相互作用：混合鲁棒性项中的噪声贡献会减缓近似速率，尽管同一项降低了估计复杂度。为解决该局限，我们提出两阶段去噪过程，用于估计并去除混合项中的噪声贡献。当鲁棒性水平按所述的样本依赖阶选择时，所得估计量可提升泛化速率，达到极小极大多项式速率（仅差一个对数因子）。我们的结果刻画了非参数框架下对抗训练的泛化行为，为对抗鲁棒性与泛化性之间的权衡提供了新解释与原则性解决方案。数值实验验证了理论发现并证明了所提方法的有效性。
英文摘要
Adversarial training can improve the robustness of predictive models to bounded perturbations, often at the cost of statistical efficiency. We study this trade-off in kernel regression over a reproducing kernel Hilbert space (RKHS). It is shown that, under squared loss, adversarial training in RKHS introduces a term involving the product of the function norm with the mean absolute value of the response noise, which we call the \textit{noise premium}. Our analysis shows that the noise premium makes the prediction error of adversarial training converge strictly more slowly than the nonparametric minimax benchmark even after balancing approximation and estimation errors. Moreover, for a fixed perturbation budget, once the budget exceeds a certain threshold, the solution to adversarial training collapses to the zero function. To mitigate these effects of the noise premium, we propose noise-debiased adversarial training. The resulting noise-debiased estimator can attain the minimax optimal rate up to a logarithmic factor for the prediction error, raises the collapse threshold, and admits an explicit bound on the increase in adversarial loss. Numerical experiments on synthetic and real data support the theoretical findings and validate the effectiveness of the proposed noise-debiased method.
