arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03379cs.LGcs.AI
收藏

重新思考无害再学习:语法作为消除失败的隐藏驱动因素

Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failures

Department of Artificial Intelligence, Yonsei University(人工智能系,延世大学)

Sangyeon Yoon, Hyesoo Hong, Wonje Jeung, Albert No

AI总结:

本文通过语法相似性驱动良性再学习现象,提出语法多样化方法以抑制再学习并提升无学习效果。

摘要:

机器无学习旨在从训练模型中移除特定内容的同时保持整体性能。然而,良性再学习现象,即即使从良性微调数据中遗忘的信息重新出现,揭示了现有无学习方法仍然根本脆弱。一个常见的解释将这种效果归因于主题相关性,但我们发现这种解释不足。通过系统分析,我们证明语法相似性,而不是主题性,是主要驱动因素:在基准测试中,语法相似的数据在没有主题重叠的情况下一致触发恢复,由于其在表示和梯度上与遗忘内容的一致性。受这一见解的启发,我们引入了语法多样化,它在无学习之前将原始遗忘查询 paraphrase 成异质结构。这种方法有效抑制了良性再学习,加速了遗忘,并显著缓解了无学习效果与模型实用性之间的权衡。

英文摘要:

Machine unlearning aims to remove specific content from trained models while preserving overall performance. However, the phenomenon of benign relearning, in which forgotten information reemerges even from benign fine-tuning data, reveals that existing unlearning methods remain fundamentally fragile. A common explanation attributes this effect to topical relevance, but we find this account insufficient. Through systematic analysis, we demonstrate that syntactic similarity, rather than topicality, is the primary driver: across benchmarks, syntactically similar data consistently trigger recovery even without topical overlap, due to their alignment in representations and gradients with the forgotten content. Motivated by this insight, we introduce syntactic diversification, which paraphrases the original forget queries into heterogeneous structures prior to unlearning. This approach effectively suppresses benign relearning, accelerates forgetting, and substantially alleviates the trade-off between unlearning efficacy and model utility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑