A continual learning agent builds on previous experiences to develop increasingly complex behaviors by adapting to non-stationary and dynamic environments while preserving previously acquired knowledge. However, scaling these systems presents significant challenges, particularly in balancing the preservation of previous policies with the adaptation of new ones to current environments. This balance, known as the stability-plasticity dilemma, is especially pronounced in complex multi-agent domains such as the train scheduling problem, where environmental and agent behaviors are constantly changing, and the search space is vast. In this work, we propose addressing these challenges in the train scheduling problem using curriculum learning. We design a curriculum with adjacent skills that build on each other to improve generalization performance. Introducing a curriculum with distinct tasks introduces non-stationarity, which we address by proposing a new algorithm: Continual Deep Q-Network (DQN) Expansion (CDE). Our approach dynamically generates and adjusts Q-function subspaces to handle environmental changes and task requirements. CDE mitigates catastrophic forgetting through EWC while ensuring high plasticity using adaptive rational activation functions. Experimental results demonstrate significant improvements in learning efficiency and adaptability compared to RL baselines and other adapted methods for continual learning, highlighting the potential of our method in managing the stability-plasticity dilemma in the adaptive train scheduling setting.
Escaping Stability-Plasticity Dilemma in Online Continual Learning for Motion Forecasting via Synergetic Memory Rehearsal
School of Mechanical Engineering, Beijing Institute of Technology(北京理工大学机械工程学院);Beijing Institute of Technology(北京理工大学);Zhengzhou Research Institute, Beijing Institute of Technology(北京理工大学郑州研究所);School of Mechanical and Aerospace Engineering, Nanyang Technological University(南洋理工大学机械与航空航天工程学院);Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University(同济大学交通工程重点实验室);Zhengzhou Nissan Automobile Co., Ltd.(郑州日产汽车有限公司)
The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning
School of Computer Software, Tianjin University(天津大学计算机软件学院);International Joint Institute of Tianjin University, Fuzhou(天津大学福州国际联合学院);The Hong Kong University of Science and Technology(香港科技大学)
Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning
School of Software, Beihang University(北京航空航天大学软件学院);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区网络空间科学与技术学院);Nanyang Technological University(南洋理工大学)
Balancing Plasticity and Stability with Fast and Slow Successor Features
School of Computer Science, McGill University, Montr\'eal, Canada(谷歌DeepMind);Mila - Quebec Artificial Intelligence Institute, Montr\'eal, QC, Canada;CIFAR Learning in Machines;Google Deepmind;Montreal Neurological Institute of McGill University, Montr\'eal, Canada
Reviewing continual learning from the perspective of human-level intelligence
University of Science and Technology of China(中国科学技术大学);Institute of Intelligent Machines, Chinese Academy of Sciences(中国科学院合肥智能机械研究所);Mississippi State University(密西西比州立大学);Sichuan University(四川大学);Southwest Jiaotong University(西南交通大学);Central South University(中南大学)
AIGS: Adaptive Incremental Gating System for Online Representation Learning in Non-Stationary Data Streams
Xiamen University Malaysia(厦门大学马来西亚分校);Multimedia University(多媒体大学);School of Economics and Management, Xiamen University Malaysia(厦门大学马来西亚分校经济与管理学院);School of Computing and Data Science, Xiamen University Malaysia(厦门大学马来西亚分校计算与数据科学学院)