AIM：面向视觉问答持续学习的非对称信息掩码
AIM: Asymmetric Information Masking for Visual Question Answering Continual Learning
AI总结：
针对视觉问答持续学习中因模型结构不对称导致的灾难性遗忘问题，提出AIM方法，通过模态特定敏感性指导的针对性掩码平衡稳定性与可塑性，提升在VQA v2和GQA任务中的平均性能和平均遗忘度。
摘要：
在持续视觉问答（VQA）中，现有持续学习（CL）方法大多针对对称、单模态架构设计。然而，现代视觉语言模型（VLMs）打破了这一假设，因其可训练组件本质上是不对称的。这种结构不匹配导致VLMs在持续数据流中学习时极易出现灾难性遗忘。具体而言，不对称性导致标准全局正则化在优化过程中倾向于大规模语言解码器，使较小但关键的视觉投影层高度易受干扰。因此，这种局部退化导致组合推理能力严重丧失。为解决此问题，我们提出非对称信息掩码（AIM），通过基于模态特定敏感性的针对性掩码平衡稳定性与可塑性。在VQA v2和GQA的持续VQA设置下，实验表明AIM在平均性能（AP）和平均遗忘（AF）方面均达到最先进的性能，同时更好地保持对新技能-概念组合的泛化能力。
英文摘要：
In continual visual question answering (VQA), existing Continual Learning (CL) methods are mostly built for symmetric, unimodal architectures. However, modern Vision-Language Models (VLMs) violate this assumption, as their trainable components are inherently asymmetric. This structural mismatch renders VLMs highly prone to catastrophic forgetting when learning from continuous data streams. Specifically, the asymmetry causes standard global regularization to favor the massive language decoder during optimization, leaving the smaller but critical visual projection layers highly vulnerable to interference. Consequently, this localized degradation leads to a severe loss of compositional reasoning capabilities. To address this, we propose Asymmetric Information Masking (AIM), which balances stability and plasticity by applying targeted masks based on modality-specific sensitivity. Experiments on VQA v2 and GQA under continual VQA settings show that AIM achieves state-of-the-art performance in both Average Performance (AP) and Average Forgetting (AF), while better preserving generalization to novel skill-concept compositions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
