arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.16470cs.CL
收藏

揭示自注意力的脆弱性

Unveiling Vulnerability of Self-Attention

Shanghai Jiao Tong University(上海交通大学);Shanghai Data Group Co., Ltd(上海数据集团有限公司)

Khai Jiet Liong, Hongqiu Wu, Hai Zhao

AI总结:

本文揭示自注意力机制的脆弱性,提出HackAttend注意力掩码扰动攻击和S-Attend平滑防御技术,实现1%扰动下98%攻击成功率及与对抗训练相当的鲁棒性。

摘要:

预训练语言模型(PLMs)被证明对微小的词语变化存在脆弱性,这对现实世界系统构成了重大威胁。以往的研究直接聚焦于操纵词语输入,但受限于其生成对抗样本的手段,缺乏对多样化现实世界攻击的泛化能力。本文研究了基于Transformer的预训练语言模型的基本结构,即自注意力(SA)机制。(1)我们提出了一种强大的扰动技术HackAttend,该技术通过精心构造的注意力掩码来扰动SA矩阵中的注意力分数。我们表明,最先进的预训练语言模型存在严重的脆弱性,微小的注意力扰动(1%)就能产生非常高的攻击成功率(98%)。本文将传统的词语扰动文本攻击扩展到更一般的结构扰动。(2)我们引入了S-Attend,一种新颖的平滑技术,通过结构扰动有效地使SA具有鲁棒性。我们通过实验证明,这种简单而有效的技术在面对各种文本攻击者时,能够实现与对抗训练相当的鲁棒性能。代码公开于github.com/liongkj/HackAttend。

英文摘要:

Pre-trained language models (PLMs) are shown to be vulnerable to minor word changes, which poses a big threat to real-world systems. While previous studies directly focus on manipulating word inputs, they are limited by their means of generating adversarial samples, lacking generalization to versatile real-world attack. This paper studies the basic structure of transformer-based PLMs, the self-attention (SA) mechanism. (1) We propose a powerful perturbation technique \textit{HackAttend}, which perturbs the attention scores within the SA matrices via meticulously crafted attention masks. We show that state-of-the-art PLMs fall into heavy vulnerability that minor attention perturbations $(1\%)$ can produce a very high attack success rate $(98\%)$. Our paper expands the conventional text attack of word perturbations to more general structural perturbations. (2) We introduce \textit{S-Attend}, a novel smoothing technique that effectively makes SA robust via structural perturbations. We empirically demonstrate that this simple yet effective technique achieves robust performance on par with adversarial training when facing various text attackers. Code is publicly available at \url{github.com/liongkj/HackAttend}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑