通过领域不可知防御探索和提高多任务深度神经网络的鲁棒性
Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses
摘要:
在本文中,我们探讨了多任务深度神经网络(MT-DNN)对跨自然语言理解(NLU)任务的非目标对抗攻击的鲁棒性,以及一些可能的防御方法。Liu等人,已经表明,由于交叉任务数据在训练时产生的正则化效应,多任务深度神经网络比仅在一个任务上训练的普通BERT模型更鲁棒(1.1%-1.5%的绝对差异)。我们进一步表明,尽管MT-DNN具有更好的泛化能力,使其可以轻松地跨域和任务转移,但它仍然可以受到损害,因为在仅2次攻击(1个字符和2个字符)之后,SNLI和SciTail任务的准确率分别下降了42.05%和32.24%。最后,我们提出了一个域不可知的防御,恢复模型的准确性(分别为36.75%和25.94%),而不是一个通用的防御或现成的拼写检查。
英文摘要:
In this paper, we explore the robustness of the Multi-Task Deep Neural Networks (MT-DNN) against non-targeted adversarial attacks across Natural Language Understanding (NLU) tasks as well as some possible ways to defend against them. Liu et al., have shown that the Multi-Task Deep Neural Network, due to the regularization effect produced when training as a result of its cross task data, is more robust than a vanilla BERT model trained only on one task (1.1%-1.5% absolute difference). We further show that although the MT-DNN has generalized better, making it easily transferable across domains and tasks, it can still be compromised as after only 2 attacks (1-character and 2-character) the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail tasks. Finally, we propose a domain agnostic defense which restores the model's accuracy (36.75% and 25.94% respectively) as opposed to a general-purpose defense or an off-the-shelf spell checker.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。