多目标参考对齐机器遗忘
Multi-Objective Reference-Aligned Machine Unlearning
AI总结：
提出多目标框架RAUL，通过将遗忘样本的预测对齐到参考分布（均匀分布或保留集经验分布）来约束遗忘目标，并利用雅可比下降解决多目标优化，实现接近完整重训练的遗忘效果。
摘要：
机器遗忘旨在移除特定训练样本的影响，同时保持模型的效用。现有的单目标方法，如梯度上升或随机重标，常常由于冲突的优化动态和无界的遗忘目标导致灾难性遗忘，使模型偏离其预训练知识。我们提出参考对齐遗忘（RAUL），一个多目标框架，通过将遗忘样本上的无界损失最大化替换为有界的KL对齐，使其预测对齐到代表未见数据的参考分布（可实例化为均匀分布或来自保留参考集的经验分布），从而约束遗忘目标并减少与保留目标的梯度冲突，联合优化遗忘和保留。通过雅可比下降解决由此产生的多目标优化（MOO）问题，该算法将多个梯度聚合到无冲突的方向。我们的结果表明，与完全重训练相比，RAUL实现了最接近的差距。
英文摘要：
Machine unlearning aims to remove the influence of specific training samples while preserving the model's utility. Existing single-objective approaches, such as gradient ascent or random relabeling, often induce catastrophic forgetting due to conflicting optimization dynamics and unbounded forgetting objectives that cause the model to drift from its pre-trained knowledge. We propose Reference-Aligned UnLearning (RAUL), a multi-objective framework that jointly optimizes forgetting and retention by replacing unbounded loss maximization with a bounded KL alignment of predictions on forgotten samples toward a reference distribution representing unseen data, instantiated either as a uniform distribution or an empirical distribution from a held-out reference set, which constrains the forgetting objective and reduces gradient conflict with retention. The resulting multi-objective optimization (MOO) problem is solved via Jacobian descent, which aggregates multiple gradients into a direction that does not conflict. Our results demonstrate that RAUL achieves the closest gap compared to full retraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
