面向多模态大语言模型的局部化与解耦知识编辑
Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models
AI总结：
针对多模态知识编辑中因果错位和特征纠缠问题，提出LDKE框架，通过快速定位关键层和解耦分类器实现精准泛化编辑并保持高局部性。
摘要：
现有的多模态知识编辑（MKE）方法在纠正多模态大语言模型（MLLMs）中过时或不准确的知识方面取得了进展。然而，它们存在一个关键局限性：虽然能有效修改目标事实对，但无法将编辑泛化到逻辑相关的查询，并且常常对无关但视觉或语义上关联的信息造成意外改变。我们识别并形式化了导致该问题的两种潜在失败模式：因果错位（将编辑限制在特定样本）和特征纠缠（对耦合但无关的信息造成意外改变）。为解决这些问题，我们提出局部化与解耦知识编辑（LDKE），一种通过定位事实特定模型层并将目标相关输入与无关输入解耦来实现精确和泛化编辑的新框架。我们的方法引入快速定位模块以高效识别和更新关键层，以及解耦分类器以适当路由输入从而保留无关知识。在各种基准和MLLMs上的大量实验表明，LDKE在将编辑传播到相关上下文方面实现了优越性能，同时保持了高局部性。
英文摘要：
Existing methods in Multimodal Knowledge Editing (MKE) have advanced the ability to correct outdated or inaccurate knowledge in Multimodal Large Language Models (MLLMs). However, they exhibit a critical limitation: while effectively modifying target factual pairs, they fail to generalize edits to logically related queries and often cause unintended alterations to unrelated but visually or semantically linked information. We identify and formalize two underlying failure modes causing this issue: Causal Misalignment, which confines edits to the specific sample, and Feature Entanglement, which causes unintended alterations to coupled but irrelevant information. To address these issues, we propose Localized and Disentangled Knowledge Editing (LDKE), a new framework that achieves precise and generalized editing by localizing fact-specific model layers and disentangling target-relevant inputs from irrelevant ones. Our approach introduces a Fast Localization module to identify and update critical layers efficiently, along with a Disentanglement Classifier that routes inputs appropriately to preserve unrelated knowledge. Extensive experiments across various benchmarks and MLLMs demonstrate that LDKE achieves superior performance in propagating edits to related contexts while maintaining high locality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
