CLIP模型中图像子群的有针对性遗忘
Targeted Forgetting of Image Subgroups in CLIP Models
AI总结：
本文提出一种无需访问预训练数据的三阶段方法，通过遗忘、提醒和模型融合恢复，并引入知识蒸馏，在CLIP模型中实现类内图像子群的有针对性遗忘，同时保持零样本性能。
摘要：
基础模型（FMs）如CLIP通过利用大规模无监督预训练，已在各种任务中展现出令人印象深刻的零样本性能。然而，它们常常从嘈杂的互联网来源数据集中继承有害或不想要的知识，从而损害了它们在现实应用中的可靠性。现有的模型遗忘方法要么依赖于访问预训练数据集，要么专注于粗粒度遗忘（例如，整个类别），从而在细粒度遗忘方面留下了关键空白。在本文中，我们解决了一个具有挑战性的场景：在不访问预训练数据的情况下，选择性地遗忘一个类别内特定部分的知识，同时保持模型的整体性能。我们提出了一种新颖的三阶段方法，该方法逐步遗忘目标知识，同时缓解过度遗忘。它包括：（1）一个遗忘阶段，在待遗忘样本上微调CLIP；（2）一个提醒阶段，在保留样本上恢复性能；以及（3）一个恢复阶段，利用模型融合（model souping）恢复零样本能力。此外，我们引入知识蒸馏来处理遗忘样本、保留样本和未见过的预训练数据之间的分布差异。在CIFAR-10、ImageNet-1K和风格数据集上的大量实验表明，我们的方法能够有效地遗忘特定子群，同时在语义相似的子群和其他类别上保持强大的零样本性能，显著优于在CLIP遗忘设置下失去有效性的基线遗忘方法。
英文摘要：
Foundation models (FMs) such as CLIP have demonstrated impressive zero-shot performance across various tasks by leveraging large-scale, unsupervised pre-training. However, they often inherit harmful or unwanted knowledge from noisy internet-sourced datasets, compromising their reliability in real-world applications. Existing model unlearning methods either rely on access to pre-trained datasets or focus on coarse-grained unlearning (e.g., entire classes), leaving a critical gap for fine-grained unlearning. In this paper, we address the challenging scenario of selectively forgetting specific portions of knowledge within a class, without access to pre-trained data, while preserving the model's overall performance. We propose a novel three-stage approach that progressively unlearns targeted knowledge while mitigating over-forgetting. It consists of (1) a forgetting stage to fine-tune the CLIP on samples to be forgotten, (2) a reminding stage to restore performance on retained samples, and (3) a restoring stage to recover zero-shot capabilities using model souping. Additionally, we introduce knowledge distillation to handle the distribution disparity between forgetting, retaining samples, and unseen pre-trained data. Extensive experiments on CIFAR-10, ImageNet-1K, and style datasets demonstrate that our approach effectively unlearns specific subgroups while maintaining strong zero-shot performance on semantically similar subgroups and other categories, significantly outperforming baseline unlearning methods, which lose effectiveness under the CLIP unlearning setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
