Forgetting-MarI: 通过边际信息正则化实现LLM反训练
Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
AI总结：
Forgetting-MarI通过边际信息正则化实现LLM反训练，有效移除冗余信息并保留关键知识，提升模型可控性和合规性。
摘要：
随着AI模型在不断扩大的数据集上进行训练，能够从已训练模型中移除特定数据影响的能力已成为隐私保护和监管合规的关键。反训练通过选择性地移除训练模型中的参数知识而不进行从头训练，对于资源密集型模型如大型语言模型（LLMs）至关重要。现有反训练方法在尝试“忘记”特定数据时往往通过移除过多信息而降级模型性能。我们引入了Forgetting-MarI，一种LLM反训练框架，可以证明性地仅移除被反训练数据所贡献的额外（边际）信息，同时保留被保留数据所支持的信息。通过惩罚边际信息，我们的方法为反训练数据集在训练模型中的残留影响提供了明确的上界，提供了可证明的不可检测性。广泛的实验证实，我们的方法在当前最先进的反训练方法上表现更优，能够提供可靠的遗忘效果，并在多样化的基准测试中保持更好的通用模型性能。这一进展代表了使AI系统更加可控，并在不损害其有效性的情况下满足隐私和版权法规的重要一步。
英文摘要：
As AI models are trained on ever-expanding datasets, the ability to remove the influence of specific data from trained models has become essential for privacy protection and regulatory compliance. Unlearning addresses this challenge by selectively removing parametric knowledge from the trained models without retraining from scratch, which is critical for resource-intensive models such as Large Language Models (LLMs). Existing unlearning methods often degrade model performance by removing more information than necessary when attempting to ''forget'' specific data. We introduce Forgetting-MarI, an LLM unlearning framework that provably removes only the additional (marginal) information contributed by the data to be unlearned, while preserving the information supported by the data to be retained. By penalizing marginal information, our method yields an explicit upper bound on the unlearn dataset's residual influence in the trained models, providing provable undetectability. Extensive experiments confirm that our approach outperforms current state-of-the-art unlearning methods, delivering reliable forgetting and better preserved general model performance across diverse benchmarks. This advancement represents an important step toward making AI systems more controllable and compliant with privacy and copyright regulations without compromising their effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
