融合触发器,打破后门:面向指令微调语言模型的防御性中毒
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
AI总结:
本文提出MB-Defense框架,通过防御性中毒和后门中和两个阶段,提升指令微调语言模型对后门攻击的鲁棒性,实验表明其有效降低攻击成功率并保持指令遵循能力。
摘要:
大型语言模型(LLMs)通过指令微调显著提升了自然语言处理(NLP)的能力,但其依赖大规模数据集使其易受后门攻击。本文提出MB-Defense(融合与打破防御框架),通过防御性中毒将攻击者和防御者触发器融合为统一后门表示,再通过额外训练打破该表示以恢复干净行为。实验表明,MB-Defense显著降低攻击成功率并保持指令遵循能力,提供了一种通用且数据高效的防御策略。
英文摘要:
Large Language Models (LLMs) have greatly advanced Natural Language Processing (NLP), particularly through instruction tuning, which enables broad task generalization without additional fine-tuning. However, their reliance on large-scale datasets-often collected from human or web sources-makes them vulnerable to backdoor attacks, where adversaries poison a small subset of data to implant hidden behaviors. Despite this growing risk, defenses for instruction-tuned models remain underexplored. We propose MB-Defense (Merging & Breaking Defense Framework), a novel training pipeline that immunizes instruction-tuned LLMs against diverse backdoor threats. MB-Defense comprises two stages: (i) Defensive Poisoning, which merges attacker and defensive triggers into a unified backdoor representation, and (ii) Backdoor Neutralization, which breaks this representation through additional training to restore clean behavior. Extensive experiments across multiple LLMs show that MB-Defense substantially lowers attack success rates while preserving instruction-following ability. Our method offers a generalizable and data-efficient defense strategy, improving the robustness of instruction-tuned LLMs against unseen backdoor attacks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。