从后门中净化生成式大语言模型而不依赖先验知识或干净参考
Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference
AI总结：
本文提出一种无需先验知识或干净参考的生成式大语言模型净化框架，通过分析后门关联在MLP层中的冗余编码，设计免疫启发的消除方法，通过合成后门变体和对比清洁版本，发现共享的后门签名，从而净化模型以抵御多种后门攻击。
摘要：
后门攻击对大型语言模型（LLMs）构成严重安全威胁，其中模型在良性输入下表现正常，但当隐藏触发器出现时会生成恶意输出。现有后门移除方法通常假设了解触发器的先验知识、能够访问干净参考模型或依赖于激进的微调配置，通常局限于分类任务。然而，在现实中的指令微调LLM设置中，这些假设往往不成立。在本文中，我们提出了一种新的框架，用于净化指令微调的LLM，而无需任何先验触发器知识或干净参考。通过系统化的合理性检查，我们发现后门关联在MLP层中被冗余地编码，而注意力模块主要放大触发信号而不建立行为。基于这一见解，我们将重点从隔离特定后门触发器转移到切断触发器-行为关联，并设计了一种免疫启发的消除方法：通过构建多个合成后门变体的给定可疑模型，每个变体分别用不同的恶意触发器-行为对进行训练，并与它们的清洁版本进行对比。变体中的反复修改揭示了共享的'后门签名'——类似于病毒中的抗原。受此签名的启发，我们中和LLM中高度可疑的组件，并应用轻量级微调以恢复其流畅性，产生能够抵御多种后门攻击和威胁模型且保留生成能力的净化模型。
英文摘要：
Backdoor attacks pose severe security threats to large language models (LLMs), where a model behaves normally under benign inputs but produces malicious outputs when a hidden trigger appears. Existing backdoor removal methods typically assume prior knowledge of triggers, access to a clean reference model, or rely on aggressive finetuning configurations, and are often limited to classification tasks. However, such assumptions fall apart in real-world instruction-tuned LLM settings. In this work, we propose a new framework for purifying instruction-tuned LLM without any prior trigger knowledge or clean references. Through systematic sanity checks, we find that backdoor associations are redundantly encoded across MLP layers, while attention modules primarily amplify trigger signals without establishing the behavior. Leveraging this insight, we shift the focus from isolating specific backdoor triggers to cutting off the trigger-behavior associations, and design an immunization-inspired elimination approach: by constructing multiple synthetic backdoored variants of the given suspicious model, each trained with different malicious trigger-behavior pairs, and contrasting them with their clean counterparts. The recurring modifications across variants reveal a shared "backdoor signature"-analogous to antigens in a virus. Guided by this signature, we neutralize highly suspicious components in LLM and apply lightweight finetuning to restore its fluency, producing purified models that withstand diverse backdoor attacks and threat models while preserving generative capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
