MemEIC：迈向持续与组合式知识编辑的一步
MemEIC: A Step Toward Continual and Compositional Knowledge Editing
AI总结：
针对大型视觉语言模型多模态知识连续更新问题，提出MemEIC方法，通过混合内外编辑器、双重外部记忆、双重LoRA适配器及受大脑启发的知识连接器，实现持续与组合式知识编辑，显著提升多模态问答性能并有效保留历史编辑。
摘要：
信息的动态特性要求持续更新大型视觉语言模型（LVLMs）。尽管近期的知识编辑技术指明了有前景的方向，但它们通常孤立地专注于编辑单一模态（视觉或语言）。这种普遍做法忽视了LVLMs固有的多模态特性以及知识更新的连续性，在考虑模态间的相互作用和持续知识精炼的需求时，可能导致次优的编辑结果。为了解决这些局限性，我们提出了MemEIC，一种用于LVLMs中持续与组合式知识编辑（CCKE）的新方法。MemEIC能够按顺序对视觉和文本知识进行组合式编辑。我们的方法采用混合的外部-内部编辑器，具有用于跨模态证据检索的双重外部记忆，以及促进为每个模态进行解耦参数更新的双重LoRA适配器。一个关键组件是受大脑启发的知识连接器，它被选择性地激活以进行组合推理，从而整合不同模态间的信息。实验表明，MemEIC显著提升了在复杂多模态问题上的性能，并有效保留了先前的编辑，为LVLMs中的CCKE设定了新基准。
英文摘要：
The dynamic nature of information necessitates continuously updating large vision-language models (LVLMs). While recent knowledge editing techniques hint at promising directions, they often focus on editing a single modality (vision or language) in isolation. This prevalent practice neglects the inherent multimodality of LVLMs and the continuous nature of knowledge updates, potentially leading to suboptimal editing outcomes when considering the interplay between modalities and the need for ongoing knowledge refinement. To address these limitations, we propose MemEIC, a novel method for Continual and Compositional Knowledge Editing (CCKE) in LVLMs. MemEIC enables compositional editing of both visual and textual knowledge sequentially. Our approach employs a hybrid external-internal editor featuring a dual external memory for cross-modal evidence retrieval and dual LoRA adapters that facilitate disentangled parameter updates for each modality. A key component is a brain-inspired knowledge connector, activated selectively for compositional reasoning, that integrates information across different modalities. Experiments demonstrate that MemEIC significantly improves performance on complex multimodal questions and effectively preserves prior edits, setting a new benchmark for CCKE in LVLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
