arXiv 2501.097052026-01-10cs.CVcs.AIcs.LG

实用的预训练视觉模型持续遗忘

Practical Continual Forgetting for Pre-trained Vision Models

State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（人工智能与机器人中心，香港科学创新研究院，中国科学院）；SenseTime Research（商汤科技研究院）

Hongbo Zhao, Fei Zhu, Bolin Ni, Feng Zhu, Gaofeng Meng, Zhaoxiang Zhang

备注：: Accepted by TPAMI. arXiv admin note: substantial text overlap with arXiv:2403.11530

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究提出GS-LoRA方法，通过组稀疏正则化和原型信息监督，实现预训练视觉模型的持续遗忘，有效删除特定类别信息同时最小化对其他类别的影响。

摘要：

出于隐私和安全考虑，现在显而易见需要从预训练视觉模型中删除不需要的信息。在现实世界中，擦除请求可能随时由用户和模型所有者产生，并且这些请求通常形成一个序列。因此，在这种设置下，期望在保持其余信息的情况下，持续地从预训练模型中删除特定信息。我们将此问题定义为持续遗忘，并识别了三个关键挑战。 (i) 对于不需要的知识，高效且有效的删除至关重要。 (ii) 对于剩余的知识，遗忘过程带来的影响应尽可能小。 (iii) 在现实世界场景中，遗忘过程中训练样本可能稀缺或部分缺失。为了解决这些问题，我们首先提出了组稀疏LoRA (GS-LoRA)。具体而言，针对 (i)，我们引入低秩适应 (LoRA) 模块，用于独立地对 Transformer 块中的 Feed-Forward 网络 (FFN) 层进行微调。针对 (ii)，采用简单的组稀疏正则化，使能够自动选择特定的 LoRA 组并将其设为零。为了进一步扩展 GS-LoRA 到更实际的场景，我们将其原型信息作为额外的监督，并引入了更实际的方法，GS-LoRA++。对于每个被遗忘的类别，我们将 logit 远离其原始原型。对于剩余的类别，我们将 logit 吸引到其各自的原型附近。我们在人脸识别、目标检测和图像分类上进行了广泛的实验，并证明我们的方法能够以最小的影响遗忘特定类别。代码已发布在 https://github.com/bjzhb666/GS-LoRA。

英文摘要：

For privacy and security concerns, the need to erase unwanted information from pre-trained vision models is becoming evident nowadays. In real-world scenarios, erasure requests originate at any time from both users and model owners, and these requests usually form a sequence. Therefore, under such a setting, selective information is expected to be continuously removed from a pre-trained model while maintaining the rest. We define this problem as continual forgetting and identify three key challenges. (i) For unwanted knowledge, efficient and effective deleting is crucial. (ii) For remaining knowledge, the impact brought by the forgetting procedure should be minimal. (iii) In real-world scenarios, the training samples may be scarce or partially missing during the process of forgetting. To address them, we first propose Group Sparse LoRA (GS-LoRA). Specifically, towards (i), we introduce Low-Rank Adaptation (LoRA) modules to fine-tune the Feed-Forward Network (FFN) layers in Transformer blocks for each forgetting task independently, and towards (ii), a simple group sparse regularization is adopted, enabling automatic selection of specific LoRA groups and zeroing out the others. To further extend GS-LoRA to more practical scenarios, we incorporate prototype information as additional supervision and introduce a more practical approach, GS-LoRA++. For each forgotten class, we move the logits away from its original prototype. For the remaining classes, we pull the logits closer to their respective prototypes. We conduct extensive experiments on face recognition, object detection, and image classification and demonstrate that our method manages to forget specific classes with minimal impact on other classes. Codes have been released on https://github.com/bjzhb666/GS-LoRA.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。