arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.17196cs.CL
收藏

只需改变一个单词:使用LLMs为文本分类器创建合成训练示例

Single Word Change is All You Need: Using LLMs to Create Synthetic Training Examples for Text Classifiers

MIT LIDS(MIT语言信息科学实验室);IRD(法国国家科研机构);Universidad Rey Juan Carlos(雷伊·胡安·卡洛斯大学)

Lei Xu, Sarah Alnegheimish, Laure Berti-Equille, Alfredo Cuesta-Infante, Kalyan Veeramachaneni

AI总结:

本文揭示了文本分类器的单词扰动漏洞,提出了鲁棒性度量指标$ρ$、攻击方法SP-Attack和防御方法SP-Defense,通过数据增强有效提升了分类器对单词扰动的鲁棒性并降低了攻击成功率。

摘要:

在文本分类中,创建对抗样本意味着在不改变句子含义的情况下对其中的几个词进行微妙的扰动,从而导致其被分类器错误分类。一个令人担忧的观察结果是,现有方法生成的对抗样本中有很大一部分仅改变了一个词。这种单词扰动漏洞代表了分类器的一个重大弱点,恶意用户可以利用它来高效地创建大量对抗样本。本文研究了这个问题并做出了以下关键贡献:(1)我们引入了一个新的度量指标$ρ$,用于定量评估分类器对单词扰动的鲁棒性。(2)我们提出了SP-Attack,旨在利用单词扰动漏洞,与最先进的对抗方法相比,实现了更高的攻击成功率,更好地保留了句子含义,同时降低了计算成本。(3)我们提出了SP-Defense,旨在通过在学习中应用数据增强来提高$ρ$。在4个数据集以及BERT和distilBERT分类器上的实验结果表明,SP-Defense在两个分类器上分别将$ρ$提高了14.6%和13.9%,将SP-Attack的攻击成功率分别降低了30.4%和21.2%,并且降低了涉及多词扰动的现有攻击方法的攻击成功率。

英文摘要:

In text classification, creating an adversarial example means subtly perturbing a few words in a sentence without changing its meaning, causing it to be misclassified by a classifier. A concerning observation is that a significant portion of adversarial examples generated by existing methods change only one word. This single-word perturbation vulnerability represents a significant weakness in classifiers, which malicious users can exploit to efficiently create a multitude of adversarial examples. This paper studies this problem and makes the following key contributions: (1) We introduce a novel metric $ρ$ to quantitatively assess a classifier's robustness against single-word perturbation. (2) We present the SP-Attack, designed to exploit the single-word perturbation vulnerability, achieving a higher attack success rate, better preserving sentence meaning, while reducing computation costs compared to state-of-the-art adversarial methods. (3) We propose SP-Defense, which aims to improve \r{ho} by applying data augmentation in learning. Experimental results on 4 datasets and BERT and distilBERT classifiers show that SP-Defense improves $ρ$ by 14.6% and 13.9% and decreases the attack success rate of SP-Attack by 30.4% and 21.2% on two classifiers respectively, and decreases the attack success rate of existing attack methods that involve multiple-word perturbations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑