arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2007.03832cs.LGstat.ML
收藏

抗干扰的深度神经网络的快速训练

Fast Training of Deep Neural Networks Robust to Adversarial Perturbations

MIT Lincoln Laboratory(麻省理工学院林肯实验室)

Justin Goodwin, Olivia Brown, Victoria Helus

摘要:

深度神经网络能够在许多领域中快速训练和推广。尽管深度网络的性能很有希望,但它们对输入的扰动(例如,对抗性示例)及其学习的特征表示通常难以解释,从而引起了对其真实能力和可信度的担忧。对抗性训练是一种鲁棒优化的形式,其中模型针对对抗性示例进行了优化,最近的工作证明了提高对扰动的性能敏感性并产生更可解释的特征表示的能力。然而,对抗训练的计算成本比标准训练(即,非鲁棒性)训练,使得其不适用于大规模问题。最近的工作表明,对抗训练的快速近似显示出减少训练时间和在无穷范数约束的扰动存在下保持鲁棒性的希望。在这项工作中,我们证明了这种方法扩展到欧几里德规范,并保留了人类对齐的功能表示,这是常见的强大的模型。此外,我们还表明,使用分布式训练方案可以进一步缩短训练鲁棒深度网络的时间。快速对抗训练是一种很有前途的方法,它将在机器学习应用中提供更高的安全性和可解释性,而鲁棒优化以前被认为是不切实际的。

英文摘要:

Deep neural networks are capable of training fast and generalizing well within many domains. Despite their promising performance, deep networks have shown sensitivities to perturbations of their inputs (e.g., adversarial examples) and their learned feature representations are often difficult to interpret, raising concerns about their true capability and trustworthiness. Recent work in adversarial training, a form of robust optimization in which the model is optimized against adversarial examples, demonstrates the ability to improve performance sensitivities to perturbations and yield feature representations that are more interpretable. Adversarial training, however, comes with an increased computational cost over that of standard (i.e., nonrobust) training, rendering it impractical for use in large-scale problems. Recent work suggests that a fast approximation to adversarial training shows promise for reducing training time and maintaining robustness in the presence of perturbations bounded by the infinity norm. In this work, we demonstrate that this approach extends to the Euclidean norm and preserves the human-aligned feature representations that are common for robust models. Additionally, we show that using a distributed training scheme can further reduce the time to train robust deep networks. Fast adversarial training is a promising approach that will provide increased security and explainability in machine learning applications for which robust optimization was previously thought to be impractical.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑