Artificial neural networks have shown remarkable success in supervised learning when trained on a single task using a fixed dataset. However, when neural networks are trained on a reinforcement learning task, their ability to continue learning from new experiences declines over time. This decline in learning ability is known as plasticity loss. To restore plasticity, prior work has explored periodically resetting the parameters of the learning network, a strategy that often improves performance. However, such resets come at the cost of a temporary drop in performance, which can be dangerous in real-world settings. To overcome this instability, we introduce AltNet, a reset-based approach that restores plasticity without performance degradation by leveraging a pair of twin networks. The use of twin networks anchors performance during resets through a mechanism that allows networks to periodically alternate roles: one network learns as it acts in the environment, while the other learns off-policy from the active network's interactions through a replay buffer. At fixed intervals, the active network is reset and the passive network, having learned from prior experience, becomes the new active network. AltNet restores plasticity, improving sample efficiency and achieving higher performance, while avoiding performance drops that pose risks in safety-critical settings. We demonstrate these advantages in several high-dimensional control tasks from the DeepMind Control Suite, where AltNet outperforms various relevant baseline methods, as well as state-of-the-art reset-based techniques.
Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
Tsinghua University（清华大学）；The University of Sydney（悉尼大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；JD Explore Academy（京东探索研究院）；Nanyang Technological University（南洋理工大学）
Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning
School of Software, Beihang University（北京航空航天大学软件学院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间科学与技术学院）；Nanyang Technological University（南洋理工大学）
Balancing Plasticity and Stability with Fast and Slow Successor Features
School of Computer Science, McGill University, Montr\'eal, Canada（谷歌DeepMind）；Mila - Quebec Artificial Intelligence Institute, Montr\'eal, QC, Canada；CIFAR Learning in Machines；Google Deepmind；Montreal Neurological Institute of McGill University, Montr\'eal, Canada
Reviewing continual learning from the perspective of human-level intelligence
University of Science and Technology of China（中国科学技术大学）；Institute of Intelligent Machines, Chinese Academy of Sciences（中国科学院合肥智能机械研究所）；Mississippi State University（密西西比州立大学）；Sichuan University（四川大学）；Southwest Jiaotong University（西南交通大学）；Central South University（中南大学）
Escaping Stability-Plasticity Dilemma in Online Continual Learning for Motion Forecasting via Synergetic Memory Rehearsal
School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械工程学院）；Beijing Institute of Technology（北京理工大学）；Zhengzhou Research Institute, Beijing Institute of Technology（北京理工大学郑州研究所）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航空航天工程学院）；Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University（同济大学交通工程重点实验室）；Zhengzhou Nissan Automobile Co., Ltd.（郑州日产汽车有限公司）
