木马语言模型的乐趣和利润
Trojaning Language Models for Fun and Profit
摘要：
近年来，出现了一种构建自然语言处理（NLP）系统的新范式：通用的预训练语言模型（LM）由简单的下游模型组成，并针对各种NLP任务进行微调。这种范式转变大大简化了系统开发周期。然而，由于许多LM是由不受信任的第三方提供的，它们缺乏标准化或监管会带来深刻的安全影响，这在很大程度上是未知的。为了弥合这一差距，这项工作研究了恶意LM对NLP系统构成的安全威胁。具体来说，我们提出了特洛伊-LM，一类新的木马攻击，恶意制作的LM触发主机NLP系统以高度可预测的方式发生故障。通过在一系列安全关键的NLP任务（有毒评论检测、问答、文本完成）中实证研究三种最先进的LM（BERT、GPT-2、XLNet）以及在众包平台上的用户研究，我们证明了TROJAN-LM具有以下特性：（i）灵活性-对手能够灵活地确定逻辑组合（例如，“and”、“or”、“xor”）任意单词作为触发器，（ii）功效-当存在嵌入触发器的输入时，主机系统很有可能按照对手的期望进行不当行为，（iii）特异性-特洛伊木马LM的功能与干净输入上的良性对应物无法区分，以及（iv）流畅性-嵌入语音的输入表现为流畅的自然语言并且与它们周围的上下文高度相关。我们提供了分析理由的实用性TROJAN-LM，并进一步讨论潜在的对策和他们的挑战，这导致几个有前途的研究方向。
英文摘要：
Recent years have witnessed the emergence of a new paradigm of building natural language processing (NLP) systems: general-purpose, pre-trained language models (LMs) are composed with simple downstream models and fine-tuned for a variety of NLP tasks. This paradigm shift significantly simplifies the system development cycles. However, as many LMs are provided by untrusted third parties, their lack of standardization or regulation entails profound security implications, which are largely unexplored. To bridge this gap, this work studies the security threats posed by malicious LMs to NLP systems. Specifically, we present TROJAN-LM, a new class of trojaning attacks in which maliciously crafted LMs trigger host NLP systems to malfunction in a highly predictable manner. By empirically studying three state-of-the-art LMs (BERT, GPT-2, XLNet) in a range of security-critical NLP tasks (toxic comment detection, question answering, text completion) as well as user studies on crowdsourcing platforms, we demonstrate that TROJAN-LM possesses the following properties: (i) flexibility - the adversary is able to flexibly dene logical combinations (e.g., 'and', 'or', 'xor') of arbitrary words as triggers, (ii) efficacy - the host systems misbehave as desired by the adversary with high probability when trigger-embedded inputs are present, (iii) specificity - the trojan LMs function indistinguishably from their benign counterparts on clean inputs, and (iv) fluency - the trigger-embedded inputs appear as fluent natural language and highly relevant to their surrounding contexts. We provide analytical justification for the practicality of TROJAN-LM, and further discuss potential countermeasures and their challenges, which lead to several promising research directions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
