Designing controllers that accomplish tasks while guaranteeing safety constraints remains a significant challenge. We often want an agent to perform well in a nominal task, such as environment exploration, while ensuring it can avoid unsafe states and return to a desired target by a specific time. In particular we are motivated by the setting of safe, efficient, hands-off training for reinforcement learning in the real world. By enabling a robot to safely and autonomously reset to a desired region (e.g., charging stations) without human intervention, we can enhance efficiency and facilitate training. Safety filters, such as those based on control barrier functions, decouple safety from nominal control objectives and rigorously guarantee safety. Despite their success, constructing these functions for general nonlinear systems with control constraints and system uncertainties remains an open problem. This paper introduces a safety filter obtained from the value function associated with the reach-avoid problem. The proposed safety filter minimally modifies the nominal controller while avoiding unsafe regions and guiding the system back to the desired target set. By preserving policy performance while allowing safe resetting, we enable efficient hands-off reinforcement learning and advance the feasibility of safe training for real world robots. We demonstrate our approach using a modified version of soft actor-critic to safely train a swing-up task on a modified cartpole stabilization problem.
Exact Imposition of Safety Boundary Conditions in Neural Reachable Tubes
ECE Department at University of Southern California(南加州大学电子工程系);EE Department at Indian Institute of Technology Patna(印度理工学院帕纳布电子工程系);Department of Aeronautics and Astronautics at Stanford University(斯坦福大学航空与航天工程系)
Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control
University of Michigan(密歇根大学);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Dalle Molle Institute for Artificial Intelligence Research (IDSIA)(达勒莫尔人工智能研究所);Università della Svizzera italiana (USI)(瑞士意大利语区大学);Scuola universitaria professionale della Svizzera italiana (SUPSI)(瑞士意大利语区应用科学与艺术大学)
Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs
Institute for Robotics and Intelligent Machines (IRIM)(机器人与智能机器研究所(IRIM));Georgia Institute of Technology(佐治亚理工学院);School of Aerospace Engineering(航空航天工程学院);Stanford University(斯坦福大学)
Input-to-State Safe Backstepping: Robust Safety-Critical Control with Unmatched Uncertainties
Department of Electrical and Computer Engineering, North Carolina State University(电气与计算机工程系,北卡罗来纳州立大学);Department of Mechanical and Civil Engineering, California Institute of Technology(机械与土木工程系,加州理工学院)
Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics
GRASP Lab, University of Pennsylvania, USA(宾夕法尼亚大学GRASP实验室);Department of Computer and Information Science, University of Pennsylvania, USA(宾夕法尼亚大学计算机与信息科学系(注:根据机构粒度要求,这里提取到大学级别,即“宾夕法尼亚大学”,但原文明确写了具体部门,所以保留原文部门名+大学的组合))
Uncertainty Estimators for Robust Backup Control Barrier Functions
California Institute of Technology(加州理工学院);Illinois Institute of Technology(伊利诺伊理工大学);Delft University of Technology(代尔夫特理工大学);Georgia Institute of Technology(佐治亚理工学院);Cleveland State University(克利夫兰州立大学);Texas A&M University(德克萨斯农工大学);The Aerospace Corporation(美国航空航天公司)