Deep Reinforcement Learning (DRL) has shown outstanding performance on inducing effective action policies that maximize expected long-term return on many complex tasks. Much of DRL work has been focused on sequences of events with discrete time steps and ignores the irregular time intervals between consecutive events. Given that in many real-world domains, data often consists of temporal sequences with irregular time intervals, and it is important to consider the time intervals between temporal events to capture latent progressive patterns of states. In this work, we present a general Time-Aware RL framework: Time-aware Q-Networks (TQN), which takes into account physical time intervals within a deep RL framework. TQN deals with time irregularity from two aspects: 1) elapsed time in the past and an expected next observation time for time-aware state approximation, and 2) action time window for the future for time-aware discounting of rewards. Experimental results show that by capturing the underlying structures in the sequences with time irregularities from both aspects, TQNs significantly outperform DQN in four types of contexts with irregular time intervals. More specifically, our results show that in classic RL tasks such as CartPole and MountainCar and Atari benchmark with randomly segmented time intervals, time-aware discounting alone is more important while in the real-world tasks such as nuclear reactor operation and septic patient treatment with intrinsic time intervals, both time-aware state and time-aware discounting are crucial. Moreover, to improve the agent's learning capacity, we explored three boosting methods: Double networks, Dueling networks, and Prioritized Experience Replay, and our results show that for the two real-world tasks, combining all three boosting methods with TQN is especially effective.
Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energy
University of Illinois Chicago(伊利诺伊大学芝加哥分校);Potsdam Institute for Climate Impact Research(波茨坦气候影响研究所);Technical University of Berlin(柏林技术大学);University of Southern California(南加州大学);University of Hong Kong(香港大学);Broad Institute of MIT and Harvard(MIT和哈佛大学Broad研究所)
Deep reinforcement learning for near-deterministic preparation of cubic- and quartic-phase gates in photonic quantum computing
Department of Physics, University of Virginia(弗吉尼亚大学物理系);Charles L. Brown Department of Electrical and Computer Engineering, University of Virginia(弗吉尼亚大学查尔斯·L·布朗电气与计算机工程系)
CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries
Beijing Key Laboratory of Embodied Intelligence Systems, Department of Automation, Tsinghua University, Beijing, China(北京智能体智能系统重点实验室,自动化系,清华大学,北京,中国);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China(复杂系统认知与决策智能重点实验室,自动化研究所,中国科学院,北京,中国)
False Correlation Reduction for Offline Reinforcement Learning
University of Technology Sydney(悉尼科技大学);Northwestern University(西北大学);Yale University(耶鲁大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Maryland(马里兰大学)
Fraunhofer Institute for Ceramic Technologies and Systems IKTS(弗劳恩霍夫陶瓷技术与系统研究所IKTS);Brandenburgische Technische Universität (BTU) Cottbus–Senftenberg(科特布斯-森夫滕伯格勃兰登堡工业大学(BTU))
Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning
Eindhoven University of Technology(埃因霍温理工大学);University of Alberta(阿尔伯塔大学);University of Twente(特文特大学);Alberta Machine Intelligence Institute (Amii)(阿尔伯塔机器智能研究所(Amii));University of Luxembourg(卢森堡大学)
DARTS: Distribution-Aware Active Rollout Trajectory Shaping for Accelerating LLM Reinforcement Learning
School of Computer Science \& Beijing Key Laboratory of Software;Hardware Cooperative Artificial Intelligence Systems, Peking University, Beijing, China;School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China;Institute of Computational Social Science, Peking University (Qingdao), Qingdao, China
Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);The State Key Lab of Brain-Machine Intelligence, Zhejiang University(浙江大学脑机智能国家重点实验室);Center for AI Safety and Governance, Peking University(北京大学人工智能安全与治理中心)
PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedback
University of Maryland, College Park(马里兰大学学院公园分校);University of Central Florida(中佛罗里达大学);J.P. Morgan AI Research(摩根大通人工智能研究院);Oregon State University(俄勒冈州立大学);Princeton University(普林斯顿大学)
ShanghaiTech University(上海科技大学);University of Electronic Science and Technology of China(电子科技大学);Shanghai Jiao Tong University(上海交通大学);University College London(伦敦大学学院);MoE Key Laboratory of Intelligent Perception and Human Machine Collaboration(智能感知与人机协同联合实验室)