多模态大语言模型中基于影响神经元路径编辑的跨模态遗忘
Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models
AI总结：
针对多模态大语言模型遗忘不一致和通用能力下降问题，提出MIP-Editor，通过模态特定归因识别影响神经元路径并进行表征误导编辑，实现高效跨模态遗忘并保持通用性能。
摘要：
多模态大语言模型（MLLMs）通过整合文本和视觉等输入，将基础模型扩展到现实世界应用中。然而，其广泛的知识容量引发了人们对隐私泄露、毒性缓解和知识产权侵犯日益增长的担忧。机器遗忘（MU）通过选择性遗忘目标知识，同时保持整体模型效用，提供了一种实用的解决方案。当应用于MLLMs时，现有基于神经元编辑的MU方法面临两个根本性挑战：（1）跨模态的遗忘变得不一致，因为现有的逐点归因方法无法捕捉连接不同模态的结构化、逐层信息流；（2）当同时支持重要推理路径的敏感神经元被剪除时，通用知识性能会下降，因为这会破坏模型的泛化能力。为了缓解这些局限性，我们提出了一种用于MU的多模态影响神经元路径编辑器（MIP-Editor）。我们的方法引入模态特定的归因分数来识别负责编码遗忘集知识的影响神经元路径，并通过表征误导应用影响路径感知的神经元编辑。该策略还能够在保持模型通用能力的同时，实现跨模态的有效且协调的遗忘。实验结果表明，MIP-Editor在多模态任务上取得了优越的遗忘性能，最大遗忘率为87.75%，通用知识保留率提升高达54.26%。在文本任务上，MIP-Editor实现了高达80.65%的遗忘率，并保留了77.9%的通用性能。代码可在https://github.com/PreckLi/MIP-Editor获取。
英文摘要：
Multimodal Large Language Models (MLLMs) extend foundation models to real-world applications by integrating inputs such as text and vision. However, their broad knowledge capacity raises growing concerns about privacy leakage, toxicity mitigation, and intellectual property violations. Machine Unlearning (MU) offers a practical solution by selectively forgetting targeted knowledge while preserving overall model utility. When applied to MLLMs, existing neuron-editing-based MU approaches face two fundamental challenges: (1) forgetting becomes inconsistent across modalities because existing point-wise attribution methods fail to capture the structured, layer-by-layer information flow that connects different modalities; and (2) general knowledge performance declines when sensitive neurons that also support important reasoning paths are pruned, as this disrupts the model's ability to generalize. To alleviate these limitations, we propose a multimodal influential neuron path editor (MIP-Editor) for MU. Our approach introduces modality-specific attribution scores to identify influential neuron paths responsible for encoding forget-set knowledge and applies influential-path-aware neuron-editing via representation misdirection. This strategy also enables effective and coordinated forgetting across modalities while preserving the model's general capabilities. Experimental results demonstrate that MIP-Editor achieves a superior unlearning performance on multimodal tasks, with a maximum forgetting rate of 87.75% and up to 54.26% improvement in general knowledge retention. On textual tasks, MIP-Editor achieves up to 80.65% forgetting and preserves 77.9% of general performance. Codes are available at https://github.com/PreckLi/MIP-Editor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
