利用即插即用状态空间模型与类条件离散化混合增强在线持续学习
Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization
AI总结：
提出即插即用模块S6MOD，通过离散化混合和类条件路由算法动态调整状态空间模型参数，显著提升在线持续学习的模型适应性与性能。
摘要：
在线持续学习（OCL）旨在从仅出现一次的数据流中学习新任务，同时保留对先前所学任务的知识。大多数现有方法依赖于重放，侧重于通过正则化或蒸馏来增强记忆保留。然而，它们往往忽略了模型的适应性，限制了从在线训练数据中增量学习可泛化和判别性特征的能力。为了解决这一问题，我们引入了一个即插即用模块 S6MOD，它可以集成到大多数现有方法中并直接提升适应性。具体而言，S6MOD 在骨干网络之后引入了一个额外分支，其中离散化混合在选择状态空间模型中选择性地调整参数，丰富了选择性扫描模式，使得模型能够自适应地为当前动态选择最敏感的离散化方法。我们进一步设计了一种类条件路由算法，用于基于不确定性的动态调整，并实现了对比离散化损失来优化它。将我们的模块与各种模型结合的大量实验表明，S6MOD 显著增强了模型的适应性，带来了实质性的性能提升，并取得了最先进的结果。
英文摘要：
Online continual learning (OCL) seeks to learn new tasks from data streams that appear only once, while retaining knowledge of previously learned tasks. Most existing methods rely on replay, focusing on enhancing memory retention through regularization or distillation. However, they often overlook the adaptability of the model, limiting the ability to learn generalizable and discriminative features incrementally from online training data. To address this, we introduce a plug-and-play module, S6MOD, which can be integrated into most existing methods and directly improve adaptability. Specifically, S6MOD introduces an extra branch after the backbone, where a mixture of discretization selectively adjusts parameters in a selective state space model, enriching selective scan patterns such that the model can adaptively select the most sensitive discretization method for current dynamics. We further design a class-conditional routing algorithm for dynamic, uncertainty-based adjustment and implement a contrastive discretization loss to optimize it. Extensive experiments combining our module with various models demonstrate that S6MOD significantly enhances model adaptability, leading to substantial performance gains and achieving the state-of-the-art results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
