arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.06136cs.CV
收藏

Mamba-FSCIL: 基于选择性状态空间模型的少样本类增量学习动态适应

Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning

School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)计算机科学与技术学院);Pengcheng Laboratory(鹏城实验室);King Abdullah University of Science and Technology(国王阿卜杜勒-阿齐兹大学);Department of Computer Science and Engineering, University of California, Merced(加州大学默塞德分校计算机科学与工程系)

Xiaojie Li, Yibo Yang, Jianlong Wu, Yue Yu, Ming-Hsuan Yang, Liqiang Nie, Min Zhang

AI总结:

提出Mamba-FSCIL,利用选择性状态空间模型(SSM)的输入依赖参数实现动态适应,通过双选择性SSM投影器和类敏感选择性扫描机制,在不扩展参数空间的情况下解耦基类和新类处理,在少样本类增量学习中达到最优性能。

摘要:

少样本类增量学习(FSCIL)旨在从有限样本中增量学习新类别,同时保留先前学习类别的知识。现有方法面临关键困境:静态架构依赖恒定参数空间来学习顺序到达的数据,容易过拟合当前会话;而动态架构不断扩展参数空间,导致复杂度增加。在本研究中,我们探索了选择性状态空间模型(SSM)在FSCIL中的潜力。Mamba利用其输入依赖参数动态调整处理模式,生成内容感知的扫描模式,无需会话级投影器扩展。这使其能够为基础类和新类配置不同的处理,有助于在适应新类的同时保留现有知识。为发挥Mamba在FSCIL中的潜力,我们设计了两个关键模块:首先,提出双选择性SSM投影器,从中间特征生成输入条件的状态空间参数以实现动态适应。双设计在结构上解耦了基础类和新类处理,采用冻结的基础分支以保持稳定的基础类特征,以及动态增量分支自适应学习新类的独特特征偏移。其次,开发了类敏感选择性扫描机制,引导增量分支的动态适应。它减少了新类数据训练对基础类表示的干扰,同时鼓励选择性扫描在基础类和新类之间以不同模式进行。在miniImageNet、CIFAR-100和CUB-200上的大量实验表明,Mamba-FSCIL达到了最先进的性能。

英文摘要:

Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes from limited examples while preserving knowledge of previously learned classes. Existing methods face a critical dilemma: static architectures rely on a constant parameter space to learn from data that arrive sequentially, making them prone to overfitting to the current session, while dynamic architectures continually expand the parameter space, leading to increased complexity. In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL. Mamba leverages its input-dependent parameters to dynamically adjust its processing patterns and generate content-aware scan patterns without session-wise projector expansion. This enables it to configure distinct processing for base and novel classes, helping preserve existing knowledge while adapting to new ones. To leverage Mamba's potential for FSCIL, we design two key modules: First, we propose a dual selective SSM projector that generates input-conditioned state-space parameters from intermediate features for dynamic adaptation. The dual design structurally decouples base and novel-class processing, employing a frozen base branch to maintain stable base-class features and a dynamic incremental branch that adaptively learns distinctive feature shifts for novel classes. Second, we develop a class-sensitive selective scan mechanism to guide dynamic adaptation of the incremental branch. It reduces the disruption to base-class representations caused by training on novel data, and meanwhile, encourages the selective scan to perform in distinct patterns between base and novel classes. Extensive experiments on miniImageNet, CIFAR-100, and CUB-200 demonstrate that Mamba-FSCIL achieves state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑