arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.10802cs.LG
收藏

一种新颖的基于噪声注入的训练方案以提升模型鲁棒性

A Novel Noise Injection-based Training Scheme for Better Model Robustness

University of Rochester(罗切斯特大学);Peking University(北京大学);Huaru Tech(华如科技);HUST(华中科技大学)

Zeliang Zhang, Jinyang Jiang, Minjie Chen, Zhiyuan Wang, Yijie Peng, Zhaofei Yu

AI总结:

本文提出一种基于噪声注入的训练方案,通过似然比梯度估计和近似方法提升脉冲神经网络的对抗鲁棒性与分类准确率,并在MNIST和Fashion-MNIST上验证其优于传统梯度训练方法。

摘要:

在以往的工作中,基于噪声注入的方法已被证明能够提升人工神经网络的鲁棒性。在本工作中,我们提出了一种新颖的基于噪声注入的训练方案,以获得更好的模型鲁棒性。具体而言,我们首先提出了一种似然比方法,用于在随机梯度下降训练中同时估计关于突触权重和噪声水平的梯度。然后,我们为原始的基于噪声注入的训练方法设计了一种近似方案,以减少内存占用并提高计算效率。接着,我们将所提出的方案应用于脉冲神经网络,并在MNIST和Fashion-MNIST数据集上评估分类准确率和鲁棒性。实验结果表明,与传统的基于梯度的训练方法相比,我们提出的方法在对抗鲁棒性上取得了显著更好的性能,同时在原始准确率上也取得了略好的表现。

英文摘要:

Noise injection-based method has been shown to be able to improve the robustness of artificial neural networks in previous work. In this work, we propose a novel noise injection-based training scheme for better model robustness. Specifically, we first develop a likelihood ratio method to estimate the gradient with respect to both synaptic weights and noise levels for stochastic gradient descent training. Then, we design an approximation for the vanilla noise injection-based training method to reduce memory and improve computational efficiency. Next, we apply our proposed scheme to spiking neural networks and evaluate the performance of classification accuracy and robustness on MNIST and Fashion-MNIST datasets. Experiment results show that our proposed method achieves a much better performance on adversarial robustness and slightly better performance on original accuracy, compared with the conventional gradient-based training method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑