AI 中文总结
该研究针对大型语言模型遗忘学习的挑战，提出基于训练的细粒度框架ADU，通过解耦上下文注意力路径实现精准遗忘，在TOFU和WMDP基准上表现最优，同时保留高模型效用并减少副作用。
AI 中文摘要
大型语言模型（LLMs）需要有效的遗忘学习以满足隐私法规和安全需求，但在不损害通用效用的前提下实现精准遗忘仍具挑战性。现有序列级和标记级方法仅惩罚目标输出，未对其依赖上下文的检索路径进行建模，可能破坏语言结构或抑制良性知识。我们提出ADU，一种基于训练的细粒度框架，将遗忘学习从标记擦除转向上下文注意力路径解耦。ADU利用局部和全局注意力头的功能差异，识别检索持久敏感锚点的预规划位置，并在原始模型下修正其候选路径；随后训练注意力投影适配器以抑制这些路径上的注意力质量，同时保留局部注意力结构和保留集语言建模。训练后激活交换测试用于验证修改后的注意力输出模块是否传递了学习到的遗忘效果。ADU在TOFU和WMDP基准测试中实现了被评估基线中最强的综合性能，其中在TOFU上的遗忘质量为0.93；它保留了87%-98%的模型效用（平均92.9%，而基线为81.9%），同时降低了良性语境中的副作用。
英文摘要
Large language models (LLMs) require effective unlearning to address privacy regulations and safety concerns. However, achieving precise forgetting without compromising general utility remains challenging. Existing sequence- and token-level methods penalize target outputs without modeling their context-dependent retrieval paths, which can disrupt linguistic structure or suppress benign knowledge. We present ADU, a fine-grained, training-based framework that shifts unlearning from token erasure to contextual attention-pathway decoupling. Exploiting the functional distinction between local and global attention heads, ADU identifies preplan positions that retrieve persistent sensitive anchors and fixes their candidate paths under the original model. It then trains attention-projection adapters to suppress attention mass along these paths while preserving local-attention structure and retain-set language modeling. Post-training activation exchange tests whether the modified attention-output module transmits the learned forgetting effect. ADU achieves the strongest aggregate performance among evaluated baselines on the TOFU and WMDP benchmarks, including a Forget Quality of (0.93) on TOFU. It preserves 87--98% of model utility (92.9% on average versus 81.9% for baselines) while reducing side effects in benign contexts.
