小波在对抗鲁棒性方面击败猴子
Wavelets Beat Monkeys at Adversarial Robustness
AI总结：
本文用无参数小波散射网络与均匀高斯噪声替代模拟 V1 的 VOneBlock，在 CIFAR-10 自适应攻击下无需对抗训练即获得更强鲁棒性。
摘要：
提高神经网络对对抗噪声（即数据中难以察觉的恶意扰动）鲁棒性的研究已受到广泛关注。目前，获得鲁棒深度神经网络的公认最先进防御方法是对抗训练（AT），但与标准训练相比，它消耗显著更多资源，并以准确率换取鲁棒性。近期一项鼓舞人心的工作 [Dapello 等人] 旨在将神经生物学工具引入这一问题：我们如何开发出能像人类视觉一样鲁棒泛化的神经网络？[Dapello 等人] 设计了一种网络结构，其神经隐藏首层模拟灵长类初级视觉皮层（V1），随后是改编自当前 CNN 视觉模型的后端结构。在标准视觉基准上针对小扰动进行测试时，该结构似乎实现了不可忽视的对抗鲁棒性。本文重新审视这一受生物学启发的工作，并探讨一种受物理学启发、有原则且无参数的表示是否能够实现相同目标。我们发现，小波散射变换可以替代复杂的 V1 皮层，而简单的均匀高斯噪声可以承担神经随机性的作用，从而实现对抗鲁棒性。在 CIFAR-10 基准上使用自适应对抗攻击开展的大量实验表明：1）当对抗攻击半径强度设置为常用基准时，VOneBlock 架构的鲁棒性相对较弱（尽管并非为零）。2）用现成的无参数 Scatternet 替代前端 VOneBlock，并在其后加入简单的均匀高斯噪声，无需对抗训练即可实现显著得多的对抗鲁棒性。我们的工作表明，受物理学启发的结构如何为鲁棒性带来新的见解，而此前这些见解被认为只有通过精细模拟人类皮层才可能获得。
英文摘要：
Research on improving the robustness of neural networks to adversarial noise - imperceptible malicious perturbations of the data - has received significant attention. The currently uncontested state-of-the-art defense to obtain robust deep neural networks is Adversarial Training (AT), but it consumes significantly more resources compared to standard training and trades off accuracy for robustness. An inspiring recent work [Dapello et al.] aims to bring neurobiological tools to the question: How can we develop Neural Nets that robustly generalize like human vision? [Dapello et al.] design a network structure with a neural hidden first layer that mimics the primate primary visual cortex (V1), followed by a back-end structure adapted from current CNN vision models. It seems to achieve non-trivial adversarial robustness on standard vision benchmarks when tested on small perturbations. Here we revisit this biologically inspired work, and ask whether a principled parameter-free representation with inspiration from physics is able to achieve the same goal. We discover that the wavelet scattering transform can replace the complex V1-cortex and simple uniform Gaussian noise can take the role of neural stochasticity, to achieve adversarial robustness. In extensive experiments on the CIFAR-10 benchmark with adaptive adversarial attacks we show that: 1) Robustness of VOneBlock architectures is relatively weak (though non-zero) when the strength of the adversarial attack radius is set to commonly used benchmarks. 2) Replacing the front-end VOneBlock by an off-the-shelf parameter-free Scatternet followed by simple uniform Gaussian noise can achieve much more substantial adversarial robustness without adversarial training. Our work shows how physically inspired structures yield new insights into robustness that were previously only thought possible by meticulously mimicking the human cortex.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
