视觉语言模型的近似域遗忘
Approximate Domain Unlearning for Vision-Language Models
AI总结：
本文提出近似域遗忘（ADU）问题，通过显式解缠域分布并自适应捕获实例特定域信息，降低指定域识别精度而保持其他域性能，实验优于VLM微调基线。
摘要：
预训练的视觉语言模型（VLM）展现出强大的泛化能力，使其无需额外训练即可识别跨多个领域的各种物体。然而，它们常常保留超出特定下游任务需求的不相关信息，引发了对计算效率和潜在信息泄露的担忧。这激发了人们对近似遗忘日益增长的兴趣，其目标是在保持整体模型性能的同时，选择性地移除不必要的知识。现有的近似遗忘方法主要集中于类别遗忘，即重新训练VLM使其无法识别指定物体类别，同时保持对其他类别的识别准确性。然而，在实际应用中，仅仅遗忘物体类别往往是不够的。例如，自动驾驶系统应准确识别真实汽车，同时避免将路边广告中描绘的插图汽车误识别为真实汽车，否则可能造成危险。在本文中，我们引入了近似域遗忘（ADU），这是一个新颖的问题设定，要求降低对指定域（例如插图）图像的识别准确性，同时保持对其他域（例如真实）的准确性。ADU带来了新的技术挑战：由于预训练VLM强大的域泛化能力，域分布在特征空间中高度纠缠，使得基于惩罚目标域的朴素方法效果不佳。为解决这一局限，我们提出了一种新颖的方法，显式地解缠域分布并自适应地捕获实例特定的域信息。大量实验表明，我们的方法优于基于VLM微调技术构建的基线方法，为VLM中实用且细粒度的遗忘铺平了道路。代码：https://kodaikawamura.github.io/Domain_Unlearning/。
英文摘要：
Pre-trained Vision-Language Models (VLMs) exhibit strong generalization capabilities, enabling them to recognize a wide range of objects across diverse domains without additional training. However, they often retain irrelevant information beyond the requirements of specific downstream tasks, raising concerns about computational efficiency and potential information leakage. This has motivated growing interest in approximate unlearning, which aims to selectively remove unnecessary knowledge while preserving overall model performance. Existing approaches to approximate unlearning have primarily focused on class unlearning, where a VLM is retrained to fail to recognize specified object classes while maintaining accuracy for others. However, merely forgetting object classes is often insufficient in practical applications. For instance, an autonomous driving system should accurately recognize real cars while avoiding misrecognition of illustrated cars depicted in roadside advertisements as real cars, which could be hazardous. In this paper, we introduce Approximate Domain Unlearning (ADU), a novel problem setting that requires reducing recognition accuracy for images from specified domains (e.g., illustration) while preserving accuracy for other domains (e.g., real). ADU presents new technical challenges: due to the strong domain generalization capability of pre-trained VLMs, domain distributions are highly entangled in the feature space, making naive approaches based on penalizing target domains ineffective. To tackle this limitation, we propose a novel approach that explicitly disentangles domain distributions and adaptively captures instance-specific domain information. Extensive experiments show that our approach outperforms baselines built upon VLM tuning techniques, paving the way for practical and fine-grained unlearning in VLMs. Code: https://kodaikawamura.github.io/Domain_Unlearning/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
