arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.06660cs.LGcs.CLcs.CRstat.ML
收藏

对预训练模型的重量中毒攻击

Weight Poisoning Attacks on Pre-trained Models

Carnegie Mellon University(卡内基梅隆大学)

Keita Kurita, Paul Michel, Graham Neubig

摘要:

最近,NLP使用大型预训练模型的情况激增。用户下载在大型数据集上预先训练的模型的权重,然后在他们选择的任务上微调权重。这就提出了一个问题,即下载不受信任的预训练权重是否会构成安全威胁。在本文中,我们证明了可以构建“权重中毒”攻击,其中预先训练的权重被注入漏洞,这些漏洞在微调后暴露“后门”,使攻击者能够简单地通过注入任意关键字来操纵模型预测。我们表明,通过应用正则化方法(我们称之为RIPPLe)和初始化过程(我们称之为嵌入手术),即使对数据集和微调过程的了解有限,这种攻击也是可能的。我们在情感分类、毒性检测和垃圾邮件检测上的实验表明,这种攻击具有广泛的适用性,并构成严重的威胁。最后,我们概述了针对此类攻击的实际防御措施。复制我们实验的代码可在https://github.com/neulab/RIPPLe上获得。

英文摘要:

Recently, NLP has seen a surge in the usage of large pre-trained models. Users download weights of models pre-trained on large datasets, then fine-tune the weights on a task of their choice. This raises the question of whether downloading untrusted pre-trained weights can pose a security threat. In this paper, we show that it is possible to construct ``weight poisoning'' attacks where pre-trained weights are injected with vulnerabilities that expose ``backdoors'' after fine-tuning, enabling the attacker to manipulate the model prediction simply by injecting an arbitrary keyword. We show that by applying a regularization method, which we call RIPPLe, and an initialization procedure, which we call Embedding Surgery, such attacks are possible even with limited knowledge of the dataset and fine-tuning procedure. Our experiments on sentiment classification, toxicity detection, and spam detection show that this attack is widely applicable and poses a serious threat. Finally, we outline practical defenses against such attacks. Code to reproduce our experiments is available at https://github.com/neulab/RIPPLe.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑