This paper introduces single-partition adaptive Q-learning (SPAQL), an algorithm for model-free episodic reinforcement learning (RL), which adaptively partitions the state-action space of a Markov decision process (MDP), while simultaneously learning a time-invariant policy (i. e., the mapping from states to actions does not depend explicitly on the episode time step) for maximizing the cumulative reward. The trade-off between exploration and exploitation is handled by using a mixture of upper confidence bounds (UCB) and Boltzmann exploration during training, with a temperature parameter that is automatically tuned as training progresses. The algorithm is an improvement over adaptive Q-learning (AQL). It converges faster to the optimal solution, while also using fewer arms. Tests on episodes with a large number of time steps show that SPAQL has no problems scaling, unlike AQL. Based on this empirical evidence, we claim that SPAQL may have a higher sample efficiency than AQL, thus being a relevant contribution to the field of efficient model-free RL methods.
DRL-AdaPart: DRL-Driven Adaptive STAR-RIS Partitioning for Fair and Efficient Resource Utilization
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Imperial College London（伦敦帝国学院）；University of Peradeniya（佩拉德尼亚大学）；University of Southampton（南安普顿大学）
Learning Data-Driven Uncertainty Set Partitions for Robust and Adaptive Energy Forecasting with Missing Data
Department of Electrical and Electronic Engineering, Imperial College London（帝国理工学院电子与电气工程系）；Center for Processes, Renewable Energy and Energy Systems (PERSEE), Mines Paris, PSL University（进程、可再生能源与能源系统中心（PERSEE），巴黎 Mines，PSL 大学）
RSD-Poker: Structure-Adaptive and Shift-Robust Risk-Utility Certification for Residual Policies in Imperfect-Information Games
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
