遗忘的遗忘：注意力汇作为大语言模型遗忘后门的网关
Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
AI总结：
该研究针对LLM遗忘过程，提出利用注意力汇作为网关的后门攻击方法，可使模型在无触发器时正常遗忘，有触发器时恢复被遗忘知识，经实验验证有效。
摘要：
大语言模型（LLM）遗忘是从预训练模型中移除不期望的数据、知识或行为，同时保留其通用效用的关键方法。然而，随着开源权重LLM的兴起，我们提出：遗忘过程本身是否可以被植入后门，在正常条件下表现为成功遗忘，但在隐藏触发器激活时恢复到遗忘前的行为？借鉴将触发器嵌入训练数据以强制特定行为的经典后门攻击，我们研究遗忘后门，即在干净环境中模型按预期遗忘，但在触发器出现时恢复被遗忘知识的场景。我们表明，设计此类攻击存在独特挑战，取决于触发器放置位置和后门训练的强化方式。我们发现后门功效与注意力汇现象（即浅层输入标记始终吸引不成比例的注意力）之间存在强关联。分析显示，这些注意力汇是遗忘后门的网关：在汇位置放置触发器并对齐其注意力值可显著增强后门持久性。大量实验验证了这些发现，表明注意力汇引导的后门遗忘在存在后门触发器时恢复被遗忘知识，而在触发器不存在时表现与正常遗忘模型无法区分。
英文摘要：
Large language model (LLM) unlearning is a key approach for removing undesired data, knowledge, or behaviors from pretrained models while retaining their general utility. Yet, with the rise of open-weight LLMs, we ask: can the unlearning process itself be backdoored, appearing successful under normal conditions yet reverting to pre-unlearned behavior when a hidden trigger is activated? Drawing inspiration from classical backdoor attacks that embed triggers into training data to enforce specific behaviors, we investigate backdooring unlearning, a setting in which models forget as intended in the clean setting but recover forgotten knowledge when the trigger appears. We show that designing such attacks presents unique challenges, hinging on where triggers are placed and how backdoor training is reinforced. We uncover a strong link between the backdoor efficacy and the attention sink phenomenon (i.e., shallow input tokens consistently attract disproportionate attention). Our analysis reveals that these attention sinks serve as gateways for backdooring unlearning: placing triggers at sink positions and aligning their attention values markedly enhances backdoor persistence. Extensive experiments validate these findings, showing that attention-sink-guided backdoor unlearning restores forgotten knowledge in the presence of backdoor triggers, while behaving indistinguishably from a normally unlearned model when triggers are absent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
