Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descend
Southern University of Science and Technology(南方科技大学);Hong Kong University of Science and Technology(香港科技大学);Agency for Science, Technology and Research(新加坡科技研究局);Tencent AI Lab(腾讯AI实验室);OPPO Mobile Telecommunications Corp.(OPPO移动通信公司)
Ning Lu, Shengcai Liu, Zhirui Zhang, Qi Wang, Haifeng Liu, Ke Tang
备注:
To be published in: 2024 IEEE Conference on Artificial Intelligence (CAI 2024)
词级文本对抗攻击已展现出误导自然语言处理(NLP)模型的显著效力。尽管这类攻击成效显著,但其生效的底层原因以及对抗样本(AEs)的基本特征仍不明确。本研究旨在通过考察词级攻击的n-gram频率模式来解释这类攻击。我们的综合实验显示,在约90%的案例中,词级攻击会生成n-gram频率下降的样本,我们将这一趋势称为n-gram频率下降(n-gram Frequency Descend,n-FD)。这一发现指向一种提升模型鲁棒性的直接策略:使用具有n-FD特征的样本训练模型。为验证该策略的可行性,我们利用n-gram频率信息替代传统的损失梯度,在对抗训练中生成扰动样本。实验结果表明,基于频率的方法在提升模型鲁棒性方面的表现与基于梯度的方法相当。本研究为理解词级文本对抗攻击提供了一种新颖且更直观的视角,并为提升模型鲁棒性提出了新方向。
英文摘要:
Word-level textual adversarial attacks have demonstrated notable efficacy in misleading Natural Language Processing (NLP) models. Despite their success, the underlying reasons for their effectiveness and the fundamental characteristics of adversarial examples (AEs) remain obscure. This work aims to interpret word-level attacks by examining their $n$-gram frequency patterns. Our comprehensive experiments reveal that in approximately 90\% of cases, word-level attacks lead to the generation of examples where the frequency of $n$-grams decreases, a tendency we term as the $n$-gram Frequency Descend ($n$-FD). This finding suggests a straightforward strategy to enhance model robustness: training models using examples with $n$-FD. To examine the feasibility of this strategy, we employed the $n$-gram frequency information, as an alternative to conventional loss gradients, to generate perturbed examples in adversarial training. The experiment results indicate that the frequency-based approach performs comparably with the gradient-based approach in improving model robustness. Our research offers a novel and more intuitive perspective for understanding word-level textual adversarial attacks and proposes a new direction to improve model robustness.
Reversible Jump Attack to Textual Classifiers with Modification Reduction
School of Computer Science, University of Technology Sydney(悉尼科技大学计算机科学学院);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院)
Contrastive Analysis of Linguistic Representations in Large Language Model Outputs through Structured Synthetic Data Generation and Abstracted N-gram Associations
A Case Study of Spanish Text Transformations for Twitter Sentiment Analysis
INFOTEC Centro de Investigación e Innovación en Tecnologías de la Información y Comunicación(INFOTEC 信息与通信技术研究与创新中心);CentroGEO Centro de Investigación en Ciencias de Información Geoespacial(CentroGEO 地理空间信息科学研究中心);CONACyT Consejo Nacional de Ciencia y Tecnología(国家科学技术委员会)
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack
The University of Sydney(悉尼大学);University of Washington(华盛顿大学);University of Liverpool(利物浦大学);University of Maryland(马里兰大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));JD Explore Academy(京东探索研究院)