On the Equilibrium between Feasible Zone and Uncertain Model in Safe Exploration
School of Vehicle and Mobility and State Key Lab of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, 100084, China（车辆与移动系统学院和智能绿色车辆与移动系统国家重点实验室，清华大学，北京，100084，中国）
Ensuring the safety of environmental exploration is a critical problem in reinforcement learning (RL). While limiting exploration to a feasible zone has become widely accepted as a way to ensure safety, key questions remain unresolved: what is the maximum feasible zone achievable through exploration, and how can it be identified? This paper, for the first time, answers these questions by revealing that the goal of safe exploration is to find the equilibrium between the feasible zone and the environment model. This conclusion is based on the understanding that these two components are interdependent: a larger feasible zone leads to a more accurate environment model, and a more accurate model, in turn, enables exploring a larger zone. We propose the first equilibrium-oriented safe exploration framework called safe equilibrium exploration (SEE), which alternates between finding the maximum feasible zone and the least uncertain model. Using a graph formulation of the uncertain model, we prove that the uncertain model obtained by SEE is monotonically refined, the feasible zones monotonically expand, and both converge to the equilibrium of safe exploration. Experiments on classic control tasks show that our algorithm successfully expands the feasible zones with zero constraint violation, and achieves the equilibrium of safe exploration within a few iterations.
Feasible Policy Iteration for Safe Reinforcement Learning
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与出行学院）；College of Artificial Intelligence, Tsinghua University（清华大学人工智能学院）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）
Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics
Institute for Data Science in Mechanical Engineering (DSME), RWTH Aachen University（机械工程数据科学研究所（DSME），亚琛工业大学）；Institute of Climate and Energy Systems (ICE), Energy Systems Engineering (ICE-1), Forschungszentrum Jülich GmbH（气候与能源系统研究所（ICE），能源系统工程（ICE-1），焦耳研究中心有限公司）
Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planning
IT University of Copenhagen, Denmark（丹麦哥本哈根IT大学）；HEC Montréal, Qc, Canada（加拿大魁北克省蒙特利尔高等商学院）；GERAD, Qc, Canada（加拿大魁北克省GERAD）；Mila - Quebec AI Institute, Qc, Canada（加拿大魁北克省Mila-魁北克人工智能研究所）
Distributed Risk-Sensitive Safety Filters for Uncertain Discrete-Time Systems
Department of Electrical and Computer Engineering, College of Design and Engineering, National University of Singapore（新加坡国立大学电气与计算机工程系，设计与工程学院）；Learning & Adaptive Systems Group, Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系学习与自适应系统小组）
From Uncertain to Safe: Conformal Adaptation of Diffusion Models for Safe PDE Control
intern（实习生）；Westlake University（西湖大学）；School of Engineering, Westlake University（西湖大学工程学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Fudan University（复旦大学）；Zhongguancun Academy（中关村学院）
