arXiv 2605.175712026-05-17cs.CVcs.LG

混合专家在类增量学习中的稳定路由

Stable Routing for Mixture-of-Experts in Class-Incremental Learning

State Key Laboratory of Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）

Zirui Guo, Quan Cheng, Da-Wei Zhou, Lijun Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究了在类增量学习中混合专家模型的稳定路由问题，提出了一种稳定路由框架StaR-MoE，通过敏感性感知路由对齐和不对称容量正则化，提高了模型对新类别的适应能力和旧类别的知识保留能力。

摘要：

类增量学习（CIL）要求模型在学习新类别时保持先前知识。最近，结合预训练模型与混合专家（MoE）的方法在CIL中受到越来越多关注：它们通常在学习过程中扩展专家，并使用路由器分配权重。然而，现有MoE方法往往忽视了专家扩展引起的路由漂移。一旦引入新的专家，路由器可能会将样本从早期类别重新分配给新加入的专家，从而扰动已建立的专家组合，即使旧专家保持冻结。我们主张，可扩展的MoE在CIL中需要两个互补的性质：稳定的旧类路由用于知识保留和足够的容量利用用于新类适应。为此，我们提出了Stable Routing for MoE（StaR-MoE），一种用于可扩展MoE的路由级别框架。通过结合敏感性感知的路由对齐，StaR-MoE通过敏感性引导的约束将当前旧类路由行为与历史路由分布对齐。同时，StaR-MoE引入了不对称容量正则化，以鼓励有效利用扩展的专家池，而不影响类特定的路由专业化。在四个标准CIL基准上的广泛实验表明，StaR-MoE在平均准确率和最后准确率上均优于现有最先进方法，突显了稳定路由的重要性。

英文摘要：

Class-incremental learning (CIL) requires models to learn new classes sequentially while preserving prior knowledge. Recently, approaches that combine pre-trained models with mixture-of-experts (MoE) have received increasing attention in CIL: they typically expand experts during learning and employ a router to assign weights across experts. However, existing MoE methods often overlook routing drift induced by expert expansion. Once new experts are introduced, the router may reassign samples from earlier classes to newly added experts, thereby perturbing previously established expert compositions and causing interference even when old experts remain frozen. We argue that expandable MoE in CIL requires two complementary properties: stable old-class routing for knowledge preservation and sufficient capacity utilization for new-class adaptation. To this end, we propose Stable Routing for MoE (StaR-MoE), a routing-level framework for expandable MoE in CIL. By incorporating sensitivity-aware routing alignment, StaR-MoE aligns current old-class routing behavior with historical routing distributions through sensitivity-guided constraints. Complementarily, StaR-MoE introduces asymmetric capacity regularization to encourage effective utilization of the expanded expert pool without compromising class-specific routing specialization. Extensive experiments across four standard CIL benchmarks demonstrate that StaR-MoE consistently improves both average and last accuracy over state-of-the-art methods, highlighting the importance of stable routing.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。