Several real-world scenarios, such as remote control and sensing, are comprised of action and observation delays. The presence of delays degrades the performance of reinforcement learning (RL) algorithms, often to such an extent that algorithms fail to learn anything substantial. This paper formally describes the notion of Markov Decision Processes (MDPs) with stochastic delays and shows that delayed MDPs can be transformed into equivalent standard MDPs (without delays) with significantly simplified cost structure. We employ this equivalence to derive a model-free Delay-Resolved RL framework and show that even a simple RL algorithm built upon this framework achieves near-optimal rewards in environments with stochastic delays in actions and observations. The delay-resolved deep Q-network (DRDQN) algorithm is bench-marked on a variety of environments comprising of multi-step and stochastic delays and results in better performance, both in terms of achieving near-optimal rewards and minimizing the computational overhead thereof, with respect to the currently established algorithms.
Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays
The University of Liverpool（利物浦大学）；Northwestern University（西北大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；National Taiwan University（国立台湾大学）；Nanyang Technological University（南洋理工大学）；The Swiss AI Lab IDSIA/USI/SUPSI（瑞士IDSIA人工智能实验室（隶属于瑞士意大利语区大学与瑞士应用科学大学））；The University of Southampton（南安普顿大学）
Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
Tsinghua University（清华大学）；The University of Sydney（悉尼大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；JD Explore Academy（京东探索研究院）；Nanyang Technological University（南洋理工大学）
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；Intelligent Game and Decision Lab (IGDL)（智能游戏与决策实验室）；Institute of Military Transportation（军事交通运输研究院）；School of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）
University of Southampton（南安普顿大学）；Northwestern University（西北大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；National Taiwan University（台湾大学）；Nanyang Technological University（南洋理工大学）
Revisiting Multi-Agent Asynchronous Online Optimization with Delays: the Strongly Convex Case
Southeast University-Monash University Joint Graduate School（东南大学-墨尔本大学联合研究生院）；School of Computer Science and Engineering（计算机科学与工程学院）；School of Software Technology（软件技术学院）
Revisiting Edge Perturbation for Graph Neural Network in Graph Data Augmentation and Attack
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Renmin University of China（中国人民大学）；Zhongguancun Laboratory（中关村实验室）；Griffith University（格里菲斯大学）
DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control
Department of Computer Engineering, School of Computation, Information and Technology, Technical University of Munich（计算机工程系，计算、信息与技术学院，慕尼黑技术大学）；Rosenheim University of Applied Sciences（罗森海姆应用技术大学）
