基于合成样本的CLIP零样本类别遗忘
Zero-Shot Class Unlearning in CLIP with Synthetic Samples
AI总结：
针对CLIP模型的类别遗忘需求，提出结合Lipschitz正则化、合成样本生成与选择性层更新的零样本遗忘方法，在多个标准数据集上验证有效并完成消融对比。
摘要：
机器遗忘是一个关键研究领域，其驱动力源于在GDPR等严格法规下，需从模型中移除敏感信息以保障个人的被遗忘权。本研究聚焦于CLIP中的遗忘问题，CLIP是一种采用对比损失、在海量图文对数据集上训练的双视觉-语言编码器模型。为实现遗忘效果，我们将Lipschitz正则化的应用拓展至CLIP的多模态场景中：具体而言，我们确保与待遗忘类别相关的视觉嵌入和文本嵌入，相对于施加于该类别样本的扰动均保持平滑。此外，重要的是，我们通过最大化目标类别的梯度上升方法生成合成样本，从而无需使用真实的遗忘数据。我们的遗忘流程是迭代式的：跟踪合成遗忘集上的准确率，当准确率降至选定阈值以下时停止。我们还采用基于各层平均绝对梯度值的选择性层更新策略，以缓解过度遗忘问题。我们在多个标准数据集上验证了所提方法，并开展了详尽的消融实验分析，同时与现有研究进行了对比。
英文摘要：
Machine unlearning is a crucial area of research. It is driven by the need to remove sensitive information from models to safeguard individuals' right to be forgotten under rigorous regulations such as GDPR. In this work, we focus on unlearning within CLIP, a dual vision-language encoder model trained on a massive dataset of image-text pairs using contrastive loss. To achieve forgetting we expand the application of Lipschitz regularization to the multimodal context of CLIP. Specifically, we ensure the smoothing of both visual and textual embeddings associated with the class intended to be forgotten relative to the perturbation introduced to the samples from that class. Additionally, importantly, we remove the necessity for real forgetting data by generating synthetic samples through gradient ascent maximizing the target class. Our forgetting procedure is iterative, where we track accuracy on a synthetic forget set and stop when accuracy falls below a chosen threshold. We employ a selective layers update strategy based on their average absolute gradient value to mitigate over-forgetting. We validate our approach on several standard datasets and provide thorough ablation analysis and comparisons with previous work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
