发表机构
KAIST（韩国科学技术院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对多模态大语言模型身份遗忘需依赖难获取保留集的问题，提出PAVA方法，定位解码器早期到中期MLPs并结合遗忘损失与视觉属性锚定，在基准上取得良好遗忘-保留权衡。
AI 中文摘要
多模态大语言模型（MLLMs）部署后常需移除特定个人的信息，但现有方法依赖保留集，而此时保留集最难获取，重建保留集会造成遗忘旨在消除的隐私暴露。仅从遗忘集进行遗忘会破坏共享的视觉-语言计算，损害感知能力。我们将无保留集遗忘视为一个定位问题：因果追踪、权重移植和Fisher重叠均指向解码器早期到中期的多层感知机（MLPs），这些层是身份信息存储的位置，且与其他模块族不同，修改这些层不会显著干扰视觉功能。我们将此转化为路径感知视觉属性锚定（PAVA），该方法将更新限制在这些层内，将遗忘损失与视觉属性锚定配对，通过仅从遗忘图像中蒸馏模型遗忘前的自身答案，从而保留基于图像的行为。在MLLMU-Bench和ReMem基准上，PAVA在仅使用遗忘集的方法中实现了最强的遗忘-保留权衡，且与基于保留集的基线相比仍具竞争力。
英文摘要
Removing a specific individual's information from multimodal large language models (MLLMs) is often needed after deployment, but existing methods rely on a retain set, which is hardest to obtain at that point, and rebuilding it recreates the privacy exposure that unlearning aims to remove. Forgetting from the forget set alone instead damages the shared visual-language computation, harming perception. We cast retain-free unlearning as a localization problem: causal tracing, weight transplant, and Fisher overlap all point to early-to-mid decoder MLPs as the layers where identity information is stored and, unlike other module families, can be modified without substantially disrupting vision. We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which confines updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone. On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.
CommentsAccepted to Findings of the Association for Computational Linguistics: EMNLP 2026
