arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.20196cs.AI
收藏

面向选择性多模态大语言模型遗忘的良性记忆遗忘

Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

Hefei University of Technology(合肥工业大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);University of Amsterdam(阿姆斯特丹大学)

Zhen Zeng, Leijiang Gu, Zhangling Duan, Feng Li, Cees G. M. Snoek, Meng Wang, Zenglin Shi

AI总结:

针对多模态大语言模型隐私泄露问题,提出S-MLLMUn Bench基准和SMFA框架,实现精确删除敏感知识同时保持模型基础能力。

摘要:

多模态大语言模型(MLLMs)在训练过程中可能会无意中记忆隐私敏感信息。虽然现有的遗忘方法可以移除这些内容,但它们通常会严重降低模型的基础能力,例如通用图像理解。这一关键缺陷促使我们研究良性记忆遗忘,即精确移除目标隐私敏感知识,同时严格保留无关能力。为了开创并评估这一目标的进展,我们引入了S-MLLMUn Bench,这是第一个旨在联合定量评估遗忘方法在知识擦除和图像理解保持方面有效性的基准。此外,我们提出了雕塑记忆遗忘适配器(SMFA),这是一个实现良性记忆遗忘的新框架。SMFA将遗忘限制在指定的记忆区域,保持整体模型性能。通过初始微调模型将敏感输出替换为拒绝回答,SMFA生成一个记忆遗忘适配器,随后通过保留锚引导的掩码机制保护无关知识。在S-MLLMUn Bench上的大量实验表明,现有方法无法实现良性遗忘,而我们提出的SMFA作为一个有效基线,成功实现了目标知识擦除而不损害模型的基础视觉能力。代码和数据可在该 https URL 获取。

英文摘要:

Multimodal large language models (MLLMs) can inadvertently memorize privacy-sensitive information during training. While existing unlearning methods can remove such content, they often severely degrade the model's foundational capabilities, such as general image understanding. This critical shortfall motivates our investigation into benign memory forgetting, the precise removal of targeted, privacy-sensitive knowledge while rigorously preserving unrelated capabilities. To pioneer and evaluate progress toward this objective, we introduce S-MLLMUn Bench, the first benchmark designed to jointly and quantitatively assess an unlearning method's efficacy in knowledge erasure and the preservation of image understanding. Furthermore, we propose the Sculpted Memory Forgetting Adapter (SMFA), a new framework that enables benign memory forgetting. SMFA confines forgetting to designated memory regions, maintaining overall model performance. By initially fine-tuning the model to replace sensitive outputs with refusals, SMFA generates a memory forgetting adapter, followed by a retaining anchor-guided masking mechanism that safeguards unrelated knowledge. Extensive experiments on S-MLLMUn Bench demonstrate that existing methods fail to achieve benign forgetting, whereas our proposed SMFA serves as an effective baseline, successfully achieving targeted knowledge erasure without compromising the model's foundational visual capabilities. Code and data are available at https://github.com/zeng-zhen/S-MLLMUn.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑