SAUCE：基于稀疏自编码器的视觉-语言模型选择性概念遗忘
SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoders
AI总结：
针对视觉-语言模型现有遗忘方法粒度粗、易过度遗忘且需大量标注的问题，提出基于稀疏自编码器的SAUCE方法，实现细粒度选择性概念遗忘，性能优于SOTA且具鲁棒性与可扩展性。
摘要：
视觉-语言模型（VLM）的遗忘方法主要改编自大型语言模型（LLM）的技术，依赖于需要大量带标注遗忘集的权重更新。此外，这些方法以较粗的粒度执行遗忘，往往导致过度遗忘并降低模型效用。为解决这一问题，我们提出SAUCE，这是一种利用稀疏自编码器（SAE）在VLM中实现细粒度选择性概念遗忘的新方法。简而言之，SAUCE首先训练SAE以捕获高维、语义丰富的稀疏特征；随后识别与待遗忘目标概念最相关的特征；在推理阶段，它选择性地修改这些特征以抑制特定概念，同时保留无关信息。我们在两种不同的VLM（LLaVA-v1.5-7B和LLaMA-3.2-11B-Vision-Instruct）上评估SAUCE，涵盖两类任务：具体概念遗忘（物体与运动场景）和抽象概念遗忘（情绪、颜色与材质），共涉及60个概念。大量实验表明，SAUCE的遗忘质量较当前最优方法提升18.04%，同时保持相当的模型效用。此外，我们还研究了SAUCE针对常用对抗攻击的鲁棒性、跨模型的可迁移性，以及处理多个同时遗忘请求的可扩展性。我们的研究结果证实，SAUCE是VLM中选择性概念遗忘的一种有效且可扩展的解决方案。
英文摘要：
Unlearning methods for vision-language models (VLMs) have primarily adapted techniques from large language models (LLMs), relying on weight updates that demand extensive annotated forget sets. Moreover, these methods perform unlearning at a coarse granularity, often leading to excessive forgetting and reduced model utility. To address this issue, we introduce SAUCE, a novel method that leverages sparse autoencoders (SAEs) for fine-grained and selective concept unlearning in VLMs. Briefly, SAUCE first trains SAEs to capture high-dimensional, semantically rich sparse features. It then identifies the features most relevant to the target concept for unlearning. During inference, it selectively modifies these features to suppress specific concepts while preserving unrelated information. We evaluate SAUCE on two distinct VLMs, LLaVA-v1.5-7B and LLaMA-3.2-11B-Vision-Instruct, across two types of tasks: concrete concept unlearning (objects and sports scenes) and abstract concept unlearning (emotions, colors, and materials), encompassing a total of 60 concepts. Extensive experiments demonstrate that SAUCE outperforms state-of-the-art methods by 18.04% in unlearning quality while maintaining comparable model utility. Furthermore, we investigate SAUCE's robustness against widely used adversarial attacks, its transferability across models, and its scalability in handling multiple simultaneous unlearning requests. Our findings establish SAUCE as an effective and scalable solution for selective concept unlearning in VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
