arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.05620cs.LGstat.ML
收藏

探索深度神经网络的脆弱性:参数损坏研究

Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption

Peking University(北京大学)

Xu Sun, Zhiyuan Zhang, Xuancheng Ren, Ruixuan Luo, Liangyou Li

摘要:

我们认为,模型参数的脆弱性是至关重要的价值,模型的鲁棒性和推广的研究,但很少有研究一直致力于了解这个问题。在这项工作中,我们提出了一个指标来衡量神经网络参数的鲁棒性,通过利用他们的脆弱性,通过参数腐败。所提出的指标描述了参数损坏情况下非平凡最坏情况下的最大损失变化。为了实用的目的,我们给出了一个基于梯度的估计,这是更有效的随机腐败试验,几乎不会引起最坏的精度下降。在理论支持和经验验证的基础上,我们能够系统地研究不同模型参数的鲁棒性,揭示以前很少关注的深层神经网络的脆弱性。此外,我们可以通过提出的对抗性抗腐蚀训练来相应地增强模型,这不仅提高了参数的鲁棒性,而且还转化为准确性的提升。

英文摘要:

We argue that the vulnerability of model parameters is of crucial value to the study of model robustness and generalization but little research has been devoted to understanding this matter. In this work, we propose an indicator to measure the robustness of neural network parameters by exploiting their vulnerability via parameter corruption. The proposed indicator describes the maximum loss variation in the non-trivial worst-case scenario under parameter corruption. For practical purposes, we give a gradient-based estimation, which is far more effective than random corruption trials that can hardly induce the worst accuracy degradation. Equipped with theoretical support and empirical validation, we are able to systematically investigate the robustness of different model parameters and reveal vulnerability of deep neural networks that has been rarely paid attention to before. Moreover, we can enhance the models accordingly with the proposed adversarial corruption-resistant training, which not only improves the parameter robustness but also translates into accuracy elevation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑