超越鲁棒性普遍定律：随机特征与神经正切核的更精确定律
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels
AI总结：
本文研究随机特征与神经正切核模型的鲁棒性，证明随机特征在任何过参数化程度下都不鲁棒，而偶激活 NTK 在满足过参数化必要条件时即可达到鲁棒性下界，从而超越 Bubeck-Sellke 普遍定律并解决相关猜想。
摘要：
机器学习模型容易受到对抗性扰动的影响，Bubeck 和 Sellke 一篇发人深省的论文通过过参数化的视角分析了这一现象：平滑地插值数据所需的参数数量显著多于简单地记忆数据。然而，这条“普遍”定律仅为鲁棒性提供了一个必要条件，无法区分不同模型。在本文中，我们通过关注两种典型设置下的经验风险最小化来弥补这些不足，即随机特征和神经正切核（NTK）。我们证明，对于随机特征，无论过参数化程度如何，模型都不具备鲁棒性，即使满足来自鲁棒性普遍定律的必要条件也是如此。相比之下，对于偶激活函数，NTK 模型达到了普遍下界，并且一旦过参数化的必要条件得到满足，它就具备鲁棒性。这也解决了 Bubeck、Li 和 Nagaraj 先前工作中提出的一个猜想。我们的分析将模型核的影响与一个“交互矩阵”解耦，该矩阵描述了与测试数据的交互，并刻画了激活函数的影响。我们的理论结果得到了合成数据集和标准数据集（MNIST、CIFAR-10）上数值实验的证实。
英文摘要：
Machine learning models are vulnerable to adversarial perturbations, and a thought-provoking paper by Bubeck and Sellke has analyzed this phenomenon through the lens of over-parameterization: interpolating smoothly the data requires significantly more parameters than simply memorizing it. However, this "universal" law provides only a necessary condition for robustness, and it is unable to discriminate between models. In this paper, we address these gaps by focusing on empirical risk minimization in two prototypical settings, namely, random features and the neural tangent kernel (NTK). We prove that, for random features, the model is not robust for any degree of over-parameterization, even when the necessary condition coming from the universal law of robustness is satisfied. In contrast, for even activations, the NTK model meets the universal lower bound, and it is robust as soon as the necessary condition on over-parameterization is fulfilled. This also addresses a conjecture in prior work by Bubeck, Li and Nagaraj. Our analysis decouples the effect of the kernel of the model from an "interaction matrix", which describes the interaction with the test data and captures the effect of the activation. Our theoretical results are corroborated by numerical evidence on both synthetic and standard datasets (MNIST, CIFAR-10).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
