Continual learning in environments with shifting data distributions is a challenging problem with several real-world applications. In this paper we consider settings in which the data distribution(task) shifts abruptly and the timing of these shifts are not known. Furthermore, we consider a semi-supervised task-agnostic setting in which the learning algorithm has access to both task-segmented and unsegmented data for offline training. We propose a novel approach called mixture of Basismodels (MoB) for addressing this problem setting. The core idea is to learn a small set of basis models and to construct a dynamic, task-dependent mixture of the models to predict for the current task. We also propose a new methodology to detect observations that are out-of-distribution with respect to the existing basis models and to instantiate new models as needed. We test our approach in multiple domains and show that it attains better prediction error than existing methods in most cases while using fewer models than other multiple model approaches. Moreover, we analyze the latent task representations learned by MoB and show that similar tasks tend to cluster in the latent space and that the latent representation shifts at the task boundaries when tasks are dissimilar.
Tree of Concepts: Interpretable Continual Learners in Non-Stationary Clinical Domains
Computer Science Department(计算机科学系);New York University(纽约大学);Department of Population Health(人口健康部);NYU Grossman School of Medicine(NYU Grossman医学院);School of Global Public Health(全球公共卫生学院)
本文提出Tree of Concepts框架,通过浅层决策树定义固定规则概念接口,并训练概念瓶颈模型预测概念,实现持续学习中的可解释性和稳定性平衡。
Mixture of Decision Trees for Interpretable Machine Learning
University of Münster(明斯特大学);Fraunhofer Institute for Manufacturing Engineering and Automation IPA(弗劳恩霍夫制造工程与自动化研究所IPA);Dresden University of Technology(德累斯顿工业大学);University of Stuttgart(斯图加特大学)
ICICLE: Interpretable Class Incremental Continual Learning
Jagiellonian University(雅盖隆大学);Ardigen SA(阿迪根公司);Autonomous University of Barcelona(巴塞罗那自治大学);Computer Vision Center(计算机视觉中心);IDEAS NCBR(IDEAS国家计算智能研究中心)
CI-CBM: Class-Incremental Concept Bottleneck Model for Interpretable Continual Learning
Department of Electrical and Computer Engineering, University of California San Diego(加州大学圣地亚哥分校电气与计算机工程系);Department of Computer Science and Engineering, University of California San Diego(加州大学圣地亚哥分校计算机科学与工程系);Halıcıoğlu Data Science Institute, University of California San Diego(加州大学圣地亚哥分校Halıcıoğlu数据科学研究所)
Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization
Harbin Institute Of Technology(哈尔滨工业大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Harbin Institute Of Technology (Shenzhen)(哈尔滨工业大学(深圳));University of Oxford(牛津大学)
Learn to Memorize: Scalable Continual Learning in Semiparametric Models with Mixture-of-Neighbors Induction Memory
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机科学学院,北京大学);Microsoft(微软公司);Microsoft Research Asia(微软亚洲研究院)
Scene-Adaptive Continual Learning for CSI-based Human Activity Recognition with Mixture of Experts
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(香港理工大学电子与电气工程系);School of Computer Science and Engineering, The Macau University of Science and Technology(澳门科学理工学院计算机科学与工程学院)
HARMONIA: Interpretable Graph Learning through Mixtures of Neural Bases
Center for AI Research, VinUniversity(文大人工智能研究中心);University of Florida(佛罗里达大学);Posts and Telecommunications Institute of Technology(邮电技术学院);University of Illinois Chicago(伊利诺伊大学芝加哥分校)
Dynamic Mixture of Latent Memories for Self-Evolving Agents
The Chinese University of Hong Kong(香港中文大学);University of Oxford(牛津大学);Nanyang Technological University(南洋理工大学);University of Edinburgh(爱丁堡大学);Shanghai Jiao Tong University(上海交通大学)
CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering
School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);Shanghai Open University(上海开放大学);ZhuQingTing Data Technology (Zhejiang) Co., Ltd(朱清庭数据科技(浙江)有限公司)
Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts
School of Computing and Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);Center for Embodied Artificial Intelligence and Computer Vision, Shenzhen Loop Area Institute(具身人工智能与计算机视觉中心,深圳环宇区研究所)
Faculty of Information Technology, University of Jyväskylä(约赫斯库利亚大学信息科技学院);National University of Singapore(新加坡国立大学);School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院)
One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
Trivita AI;The University of Texas at Austin(德克萨斯大学奥斯汀分校);Northeastern University(东北大学);Hanoi University of Science and Technology(河内科学技术大学);Rutgers University(罗格斯大学)
Interpretability Is Not Just a Tax: Sparsity and Diversity Yield Legible and Composable Reward Experts for Personalization
Saarland University(萨尔兰大学);Independent Researcher(独立研究者);Bielefeld University(比勒菲尔德大学);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所);Max Planck Institute for Informatics(马克斯·普朗克信息研究所)
SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(人工智能国家重点实验室,智能科学与技术学院,北京大学);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)