PETA:参数高效的特洛伊攻击
PETA: Parameter-Efficient Trojan Attacks
AI总结:
本文提出 PETA,一种基于双层优化的参数高效特洛伊攻击,在模拟 PEFT 下游适配的同时嵌入后门,兼顾攻击成功率与干净准确率。
摘要:
参数高效微调(PEFT)能够高效地将预训练语言模型(PLM)适配到特定任务。通过仅微调极少量的(额外)参数,PEFT 可达到与标准微调相当的性能。然而,尽管 PEFT 使用广泛,其安全影响在很大程度上仍未得到探索。本文迈出初步探索步伐,提出 PETA,一种新型特洛伊攻击,通过双层优化考虑下游适配过程来破坏 PLM 的权重:上层目标将后门嵌入模型,下层目标模拟 PEFT,以同时保持 PLM 的任务特定性能并确保后门在微调后仍然存在。通过在多种下游任务和触发器设计上的广泛评估,我们证明了 PETA 在攻击成功率和干净准确率方面的有效性,即使攻击者并不完全了解受害用户的训练过程。
英文摘要:
Parameter-efficient fine-tuning (PEFT) enables efficient adaptation of pre-trained language models (PLMs) to specific tasks. By tuning only a minimal set of (extra) parameters, PEFT achieves performance that is comparable to standard fine-tuning. However, despite its prevalent use, the security implications of PEFT remain largely unexplored. In this paper, we take the initial steps and present PETA, a novel trojan attack that compromises the weights of PLMs by accounting for downstream adaptation through bilevel optimization: the upper-level objective embeds the backdoor into a model while the lower-level objective simulates PEFT to both retain the PLM's task-specific performance and ensure that the backdoor persists after fine-tuning. With extensive evaluation across a variety of downstream tasks and trigger designs, we demonstrate PETA's effectiveness in terms of both attack success rate and clean accuracy, even when the attacker does not have full knowledge of the victim user's training process.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。