arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.23780cs.AI
收藏

超越二元编辑:基于对抗子空间对齐的鲁棒多模态知识编辑

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment

Zhejiang University(浙江大学);National University of Singapore(新加坡国立大学);Jiangxi Normal University(江西师范大学)

Haoyuan Wang, Xiaohao Liu, Jiajie Su, Jianmao Xiao, Chaochao Chen

AI总结:

针对多模态大语言模型知识编辑泛化性不足的问题,提出对抗子空间对齐方法(ASAM),通过潜在对抗鲁棒化(LAR)和秩约束子空间学习(RCSL)实现鲁棒的多模态知识编辑。

摘要:

多模态大语言模型(MLLMs)需要高效的机制来更新知识,同时不降低现有能力。虽然内在多模态知识编辑实现了强可靠性和局部性,但它通常表现出有限的泛化性,无法在语义等价的视觉和语言变体之间传播编辑。这个问题源于在高维多模态空间中缺乏显式的语义监督、僵化的编辑范围以及对单个样本的有偏锚定。我们通过显式地针对泛化性来解决鲁棒的内在多模态知识编辑。我们通过知识单元(将语义等价的多模态输入分组)形式化鲁棒性,并将泛化性定义为每个单元内一致的预测。为了暴露脆弱的语义区域,我们引入了潜在对抗鲁棒化(LAR),它在联合潜在空间中生成对抗但语义连贯的变体。我们进一步提出了秩约束子空间学习(RCSL),通过基于奇异值的目标在编辑层强制对抗表示的低秩对齐。大量实验证明了ASAM的有效性。

英文摘要:

Multimodal large language models (MLLMs) need efficient mechanisms to update knowledge without degrading existing capabilities. While intrinsic multimodal knowledge editing achieves strong reliability and locality, it often exhibits limited generality, failing to propagate edits across semantically equivalent visual and linguistic variations. This issue arises from the lack of explicit semantic supervision, rigid editing scopes, and biased anchoring to individual samples in high-dimensional multimodal spaces. We address robust intrinsic multimodal knowledge editing by explicitly targeting generalization. We formalize robustness through knowledge units that group semantically equivalent multimodal inputs and define generality as consistent predictions within each unit. To expose fragile semantic regions, we introduce Latent Adversarial Robustification (LAR), which generates adversarial yet semantically coherent variants in the joint latent space. We further propose Rank-Constrained Subspace Learning (RCSL), enforcing low-rank alignment of adversarial representations at the edit layer via a singular value-based objective. Extensive analysis demonstrates the effectiveness of ASAM empirically.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑