SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(人工智能国家重点实验室,智能科学与技术学院,北京大学);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)
Lirui Luo, Guoxi Zhang, Hongming Xu, Cong Fang, Qing Li
In deep reinforcement learning (DRL), an agent is trained from a stream of experience. In a continual learning setting, such agents can suffer from plasticity loss: their ability to learn new skills from new experiences diminishes over training. Recently, Mixture-of-Experts (MoE) networks have been reported to enable scaling laws and facilitate the learning of diverse skills. However, in continual reinforcement learning settings, their performance can degenerate as learning proceeds, indicating a loss of plasticity. To address this, building on Neural Tangent Kernel (NTK) theory, we formalize the plasticity loss in MoE policies as a loss of spectral plasticity. We then derive a tractable proxy for spectral plasticity, one expressible in terms of individual expert feature matrices. Leveraging this proxy, we introduce SPHERE, a practical Parseval penalty tailored for MoE-based policies that alleviates the loss of spectral plasticity. On MetaWorld and HumanoidBench, SPHERE improves average success under continual RL by 133% and 50% over an unregularized MoE baseline, while maintaining higher spectral plasticity throughout training.
The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning
School of Computer Software, Tianjin University(天津大学计算机软件学院);International Joint Institute of Tianjin University, Fuzhou(天津大学福州国际联合学院);The Hong Kong University of Science and Technology(香港科技大学)
Normative Networks for Source Separation via Local Plasticity and Dendritic Computation
Gatsby Computational Neuroscience Unit, University College London, UK(Gatsby计算神经科学单元,伦敦大学学院,英国);Department of Computer Science, University of Oxford, UK(计算机科学系,牛津大学,英国);Brain Network Dynamics Unit, University of Oxford, UK(脑网络动力学单元,牛津大学,英国);MRC Centre of Research Excellence in Restorative Neural Dynamics, UK(英国医学研究委员会修复神经动力学研究卓越中心)
Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
Zhejiang University(浙江大学);Shanghai Innovation Institute(上海创新研究院);ByteDance Inc.(字节跳动公司);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)
Revisiting Clustering of Neural Bandits: Selective Reinitialization for Mitigating Loss of Plasticity
Gaoling School of Artificial Intelligence Renmin University of China Beijing China;Gaoling School of Artificial Intelligence Renmin University of China
FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
Department of Automation, Tsinghua University(自动化系,清华大学);Academy of Medical Engineering and Translational Medicine, Tianjin University(医学工程与转化医学学院,天津大学)
One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
Trivita AI;The University of Texas at Austin(德克萨斯大学奥斯汀分校);Northeastern University(东北大学);Hanoi University of Science and Technology(河内科学技术大学);Rutgers University(罗格斯大学)
Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts
School of Computing and Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);Center for Embodied Artificial Intelligence and Computer Vision, Shenzhen Loop Area Institute(具身人工智能与计算机视觉中心,深圳环宇区研究所)
Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning
Department of Electrical and Computer Engineering, The University of Hong Kong(香港大学电子与计算机工程系);School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院)
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts
National Engineering Research Center for Multimedia Software(国家多媒体软件工程技术研究中心);School of Computer Science, Wuhan University(武汉大学计算机学院);Hubei Luojia Laboratory(湖北珞珈实验室);Sun Yat-sen University(中山大学);JD Explore Academy(京东探索研究院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Nanyang Technological University(南洋理工大学)
Mammo-Mamba: A Hybrid State-Space and Transformer Architecture with Sequential Mixture of Experts for Multi-View Mammography
Concordia Institute for Information Systems Engineering (CIISE)(康科德信息系统工程研究所);Concordia University(康科德大学);Thomas Jefferson University Hospital(托马斯·杰斐逊大学医院)
Understanding and Mitigating the Bias in Sample Selection for Learning with Noisy Labels
College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Singapore University of Technology and Design(新加坡科技设计大学);School of Software, Zhejiang University(浙江大学软件学院)