Online Adaptive Reinforcement Learning with Echo State Networks for Non-Stationary Dynamics
Department of Computer Science, Nagoya Institute of Technology（名古屋技术大学计算机科学系）；Nagoya Institute of Technology（名古屋技术大学）；International Research Center for Neurointelligence, The University of Tokyo（神经智能国际研究中心，东京大学）
Reinforcement learning (RL) policies trained in simulation often suffer from severe performance degradation when deployed in real-world environments due to non-stationary dynamics. While Domain Randomization (DR) and meta-RL have been proposed to address this issue, they typically rely on extensive pretraining, privileged information, or high computational cost, limiting their applicability to real-time and edge systems. In this paper, we propose a lightweight online adaptation framework for RL based on Reservoir Computing. Specifically, we integrate an Echo State Networks (ESNs) as an adaptation module that encodes recent observation histories into a latent context representation, and update its readout weights online using Recursive Least Squares (RLS). This design enables rapid adaptation without backpropagation, pretraining, or access to privileged information. We evaluate the proposed method on CartPole and HalfCheetah tasks with severe and abrupt environment changes, including periodic external disturbances and extreme friction variations. Experimental results demonstrate that the proposed approach significantly outperforms DR and representative adaptive baselines under out-of-distribution dynamics, achieving stable adaptation within a few control steps. Notably, the method successfully handles intra-episode environment changes without resetting the policy. Due to its computational efficiency and stability, the proposed framework provides a practical solution for online adaptation in non-stationary environments and is well suited for real-world robotic control and edge deployment.
TempEE: Temporal-Spatial Parallel Transformer for Radar Echo Extrapolation Beyond Auto-Regression
Guangdong-Hongkong-Macao Greater Bay Area Weather Research Center for Monitoring Warning and Forecasting (Shenzhen Institute of Meteorological Innovation)（粤港澳大湾区气象监测预警预报中心（深圳气象创新研究院））；Australian Artificial Intelligence Institute, School of Computer Science, FEIT, University of Technology Sydney（悉尼科技大学计算机科学学院澳大利亚人工智能研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外语外贸大学信息科学与技术学院）；Shenzhen Meteorological Bureau / Shenzhen Key laboratory of severe weather in south China（深圳市气象局 / 深圳市南方强天气重点实验室）
Bilevel Online Deep Learning in Non-stationary Environment
China University of Petroleum, Beijing Campus (CUP)（中国石油大学（北京）校区）；Department of Automation, College of Information Science and Engineering, China University of Petroleum, Beijing Campus (CUP)（中国石油大学（北京）校区信息科学与工程学院自动化系）
Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Univ. Grenoble Alpes（格勒诺布尔大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；Carnegie Mellon University（卡内基梅隆大学）
在线预测在不断变化的环境下已经成为一个越来越重要的问题，在许多现实世界的应用。在本文中，我们考虑的元算法中提出的\citet{zhang2017dynamic}结合不同的子程序。我们证明了对于参数序列的总变差由$C_n$约束的非平稳在线线性回归，可以得到一个期望的$\tilde{O}(n^{1/3} C_n^{2/3})$阶累积误差。本文将\cite{baby2019online}中提出的一维时间序列在线预测的结果推广到一般的$d$维非平稳线性回归。我们提高了Zhang等人2017和Besbes等人2015获得的$O(\sqrt{n C_n})$率。我们进一步扩展我们的分析，非平稳在线核回归。与非平稳在线回归情况类似，我们使用Zhang et al. 2017的元程序结合Kernel-AWV（Jezequel et al. 2020）来实现由RKHS的有效维度和序列的总变异控制的预期累积。据我们所知，这项工作是非平稳在线回归的非平稳核回归的第一次扩展。最后，我们评估我们的方法与现有的几个基准经验，并将其与本文得到的理论界。
Non-Stationary Online Structured Prediction with Surrogate Losses
CyberAgent, Tokyo, Japan（日本东京CyberAgent公司）；National Institute of Informatics, Tokyo, Japan（日本东京信息机构）；Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan（日本东京RIKEN高级智能项目）；The Institute of Statistical Mathematics, Tokyo, Japan（日本东京统计数学研究所）；Tohoku University, Miyagi, Japan（日本宫城县立东大学）；Nanyang Technological University, Singapore（新加坡南洋理工大学）
FDNet: A Deep Learning Approach with Two Parallel Cross Encoding Pathways for Precipitation Nowcasting
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Guiyang Academy of Information Technology（贵阳信息技术研究院）；NTT DATA Corporation（NTT DATA株式会社）；NTT DATA Institute of Management Consulting Inc.（NTT DATA经营研究所）
Online Learning and Matching for Resource Allocation Problems
University of California, Irvine（加州大学欧文分校）；DAMO Academy, Alibaba US（阿里巴巴美国达摩院）；Amherst College（阿默斯特学院）；University of California, Los Angeles（加州大学洛杉矶分校）；University College London（伦敦大学学院）；Purdue University（普渡大学）
Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamics
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；Polixir Technologies（Polixir科技公司）
