Deep reinforcement learning has been successfully applied to many control tasks, but the application of such agents in safety-critical scenarios has been limited due to safety concerns. Rigorous testing of these controllers is challenging, particularly when they operate in probabilistic environments due to, for example, hardware faults or noisy sensors. We propose MOSAIC, an algorithm for measuring the safety of deep reinforcement learning agents in stochastic settings. Our approach is based on the iterative construction of a formal abstraction of a controller's execution in an environment, and leverages probabilistic model checking of Markov decision processes to produce probabilistic guarantees on safe behaviour over a finite time horizon. It produces bounds on the probability of safe operation of the controller for different initial configurations and identifies regions where correct behaviour can be guaranteed. We implement and evaluate our approach on agents trained for several benchmark control problems.
Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation
Department of Computer Science, University of Verona（威尼斯大学计算机科学系）；Department of Engineering for Innovation Medicine, University of Verona（威尼斯大学创新医学工程系）；Massachusetts Institute of Technology（麻省理工学院）
Robust Probabilistic Shielding for Safe Offline Reinforcement Learning
Radboud University Nijmegen The Netherlands；Ruhr University\,\&\,Radboud University Bochum Germany；Radboud University；Ruhr University\,\&\,Radboud University
Sample-Efficient Learning of Probabilistic Causes for Reachability in Markov Decision Processes with Probabilistic Guarantees
Toyota Research Institute of North America（丰田美国研究院）；Toyota Motor North America Frontier Research Center（丰田北美前沿研究中心）；Toyota Motor Corporation（丰田公司）；Toyota Motor North America（丰田北美）
Safe Learning Control with Optimality and Stability Guarantees
Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments, School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, Guangdong 518055, China（深圳先进 motion 控制与现代自动化装备重点实验室，智能科学与工程学院，哈尔滨工业大学，深圳，广东 518055，中国）；School of Data Science, Lingnan University, Hong Kong（数据科学学院，岭南大学，香港）；School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（电气与电子工程学院，南洋理工大学，新加坡）；MIT CSAIL
Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
CSX-AI；University of Liverpool（利物浦大学）；Carl von Ossietzky Universität Oldenburg（奥尔登堡卡尔·冯·奥西特齐克大学）；Austrian Institute of Technology（奥地利技术研究院）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）
Connecting Surrogate Safety Measures to Crash Probablity via Causal Probabilistic Time Series Prediction
University of California, Berkeley（加州大学伯克利分校）；Safe Transportation Research and Education Center（安全交通研究与教育中心）；Institute of Transportation Studies（交通研究所）
Safe Probabilistic Planning for Human-Robot Interaction using Conformal Risk Control
University of Washington, Department of Electrical and Computer Engineering（华盛顿大学电气与计算机工程系）；University of Washington, Department of Aeronautics and Astronautics（华盛顿大学航空与航天系）
Towards provable probabilistic safety for scalable embodied AI systems
Department of Automation, Tsinghua University（清华大学自动化系）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；School of Computer, Liaocheng University（聊城大学计算机学院）；Department of Automation, Southeast University（东南大学自动化学院）；Department of Mechanical and Aerospace Engineering, George Washington University（乔治华盛顿大学机械与航空航天工程系）；University of Michigan Transportation Research Institute（密歇根大学交通研究所）；Department of Civil and Environmental Engineering, University of Michigan（密歇根大学土木与环境工程系）
