arXiv 2411.174712025-08-04cs.LGcs.CRcs.CV

学习新概念，记住旧知识：多模态概念瓶颈模型的持续学习

Learning New Concepts, Remembering the Old: Continual Learning for Multimodal Concept Bottleneck Models

HKUST(GZ) Deep Interdisciplinary Intelligence Lab（香港科技大学（广州）深度跨学科智能实验室）；Wuhan University（武汉大学）；Shandong University（山东大学）；Tongji University（同济大学）；Peking University（北京大学）；University of Macau（澳门大学）；HKUST(GZ) Institute of Deep Perception Technology, JITRI Deep Interdisciplinary Intelligence Lab（香港科技大学（广州）感知技术研究所，JITRI深度跨学科智能实验室）

Songning Lai, Mingqian Liao, Zhangyi Hu, Jiayu Yang, Wenshuo Chen, Hongru Xiao, Jianheng Tang, Haicheng Liao, Yutao Yue

发表信息：: ACM MM 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多模态概念瓶颈模型在动态数据流下的持续学习难题，提出CONCIL框架，将概念与决策层更新重构为线性回归，避免灾难性遗忘，实现绝对知识记忆并显著超越传统方法。

摘要：

概念瓶颈模型（CBMs）通过将视觉输入与人类可理解的概念相连接，增强了人工智能系统的可解释性，有效地充当了一种多模态可解释性模型。然而，现有的CBMs通常假设数据集是静态的，这从根本上限制了它们对现实世界中不断演变的多模态数据流的适应性。为解决这一问题，我们为CBMs定义了一项新颖的持续学习任务：同时处理概念增量学习和类别增量学习。该任务要求模型在持续获取新概念（通常代表跨模态属性）和新类别的同时，稳健地保留先前学到的知识。为应对这一挑战，我们提出了概念持续增量学习（CONCIL）框架，该框架从根本上将概念层和决策层的更新重新构想为线性回归问题。这种重新表述消除了对基于梯度的优化的需求，从而有效防止了灾难性遗忘。关键在于，CONCIL仅依赖递归矩阵运算，使其具有高度计算效率，非常适合实时和大规模多模态数据应用。实验结果令人信服地表明，CONCIL实现了“绝对知识记忆”，并在概念增量和类别增量设置中均显著超越了传统CBM方法的性能，从而为CBMs中的持续学习确立了新范式，对动态多模态理解尤为有价值。

英文摘要：

Concept Bottleneck Models (CBMs) enhance the interpretability of AI systems, particularly by bridging visual input with human-understandable concepts, effectively acting as a form of multimodal interpretability model. However, existing CBMs typically assume static datasets, which fundamentally limits their adaptability to real-world, continuously evolving multimodal data streams. To address this, we define a novel continual learning task for CBMs: simultaneously handling concept-incremental and class-incremental learning. This task requires models to continuously acquire new concepts (often representing cross-modal attributes) and classes while robustly preserving previously learned knowledge. To tackle this challenging problem, we propose CONceptual Continual Incremental Learning (CONCIL), a novel framework that fundamentally re-imagines concept and decision layer updates as linear regression problems. This reformulation eliminates the need for gradient-based optimization, thereby effectively preventing catastrophic forgetting. Crucially, CONCIL relies solely on recursive matrix operations, rendering it highly computationally efficient and well-suited for real-time and large-scale multimodal data applications. Experimental results compellingly demonstrate that CONCIL achieves "absolute knowledge memory" and significantly surpasses the performance of traditional CBM methods in both concept- and class-incremental settings, thus establishing a new paradigm for continual learning in CBMs, particularly valuable for dynamic multimodal understanding.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。