Continual reinforcement learning (CRL) requires agents to learn from a sequence of tasks without forgetting previously acquired policies. In this work, we introduce a novel benchmark suite for CRL based on realistically simulated robots in the Gazebo simulator. Our Continual Robotic Simulation Suite (CRoSS) benchmarks rely on two robotic platforms: a two-wheeled differential-drive robot with lidar, camera and bumper sensor, and a robotic arm with seven joints. The former represent an agent in line-following and object-pushing scenarios, where variation of visual and structural parameters yields a large number of distinct tasks, whereas the latter is used in two goal-reaching scenarios with high-level cartesian hand position control (modeled after the Continual World benchmark), and low-level control based on joint angles. For the robotic arm benchmarks, we provide additional kinematics-only variants that bypass the need for physical simulation (as long as no sensor readings are required), and which can be run two orders of magnitude faster. CRoSS is designed to be easily extensible and enables controlled studies of continual reinforcement learning in robotic settings with high physical realism, and in particular allow the use of almost arbitrary simulated sensors. To ensure reproducibility and ease of use, we provide a containerized setup (Apptainer) that runs out-of-the-box, and report performances of standard RL algorithms, including Deep Q-Networks (DQN) and policy gradient methods. This highlights the suitability as a scalable and reproducible benchmark for CRL research.
Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning
Johannes Kepler University Linz（林茨约翰内斯·开普勒大学）；Dynatrace Research（Dynatrace研究院）；Institute of Advanced Research in Artificial Intelligence (IARAI)（高级人工智能研究所（IARAI））
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
ElementAI（ElementAI公司）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Facebook AI Research（脸书人工智能研究院）；McGill University（麦吉尔大学）；HEC Montréal（蒙特利尔高等商学院）
Towards Lifelong Aerial Autonomy: Geometric Memory Management for Continual Visual Place Recognition in Dynamic Environments
Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；College of Instrument Science and Opto-electronics Engineering, Beijing Information Science and Technology University（北京信息科技大学仪器科学与光电工程学院）；Key Laboratory of Complex System Intelligent Control and Decision Making, Beijing Institute of Technology（北京理工大学复杂系统智能控制与决策重点实验室）；School of Aerospace Engineering, Beijing Institute of Technology（北京理工大学宇航学院）
Defining Benchmarks for Continual Few-Shot Learning
University of Edinburgh（爱丁堡大学）
由于采用了适当的基准，Few-Shot和持续学习在过去几年中都取得了重大进展。话虽如此，该领域仍然需要为持续Few-Shot学习的高度理想设置构建一套基准，其中学习者被一个接一个地呈现许多Few-Shot任务，然后被要求在源于所有先前看到的任务的验证集上表现良好。连续少样本学习具有较小的计算占用空间，因此是高效调查和实验的绝佳设置。在本文中，我们首先定义了一个连续少样本学习的理论框架，考虑到最近的文献，然后我们提出了一系列灵活的基准，统一的评估标准，并允许从多个角度探索问题。作为基准测试的一部分，我们引入了ImageNet的一个紧凑型变体，称为SlimageNet64，它保留了所有原始的1000个类，但只包含每个类的200个实例（总共20万个数据点），缩小到64 x 64像素。我们使用一些流行的少样本学习算法为建议的基准提供基线，因此，在连续和数据有限的设置中暴露了这些算法以前未知的优点和缺点。
Department of Computer Science and Technology, Tsinghua University, China（计算机科学与技术系，清华大学，中国）；College of Computer Science, Zhejiang University, China（浙江大学计算机科学学院，中国）；College of Computer Science, Beijing University of Posts and Telecommunications, China（北京邮电大学计算机科学学院，中国）
本文提出连续GUI代理任务，通过引入GUI-Anchoring in Flux框架，解决GUI分布变化时持续学习稳定性问题，实验显示其优于现有方法。
Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
Wuhan University（武汉大学）；Northeastern University（东北大学）；Sun Yat-sen University（中山大学）；Beijing Institute of Technology（北京理工大学）；Nanyang Technological University（南洋理工大学）
