We address multi-robot safe mission planning in uncertain dynamic environments. This problem arises in several applications including safety-critical exploration, surveillance, and emergency rescue missions. Computation of a multi-robot optimal control policy is challenging not only because of the complexity of incorporating dynamic uncertainties while planning, but also because of the exponential growth in problem size as a function of number of robots. Leveraging recent works obtaining a tractable safety maximizing plan for a single robot, we propose a scalable two-stage framework to solve the problem at hand. Specifically, the problem is split into a low-level single-agent control problem and a high-level task allocation problem. The low-level problem uses an efficient approximation of stochastic reachability for a Markov decision process to derive the optimal control policy under dynamic uncertainty. The task allocation is solved using polynomial-time forward and reverse greedy heuristics and in a distributed auction-based manner. By leveraging the properties of our safety objective function, we provide provable performance bounds on the safety of the approximate solutions proposed by these two heuristics. We evaluate the theory with extensive numerical case studies.
Hierarchical LLMs In-the-Loop Optimization for Real-Time Multi-Robot Target Tracking under Unknown Hazards
GRASP Lab, University of Pennsylvania（宾夕法尼亚大学GRASP实验室）；Department of Electrical and Computer Engineering, Drexel University（德雷塞尔大学电气与计算机工程系）；Department of Computer Science, University of Southern California（南加州大学计算机科学系）
Despite the successful demonstration of autonomous vehicles (AVs), such as self-driving cars, ensuring AV safety remains a challenging task. Although some actors influence an AV's driving decisions more than others, current approaches pay equal attention to each actor on the road. An actor's influence on the AV's decision can be characterized in terms of its ability to decrease the number of safe navigational choices for the AV. In this work, we propose a safety threat indicator (STI) using counterfactual reasoning to estimate the importance of each actor on the road with respect to its influe…
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
Industrial human-robot collaborative systems must be validated thoroughly with regard to safety. The sooner potential hazards for workers can be exposed, the less costly is the implementation of necessary changes. Due to the complexity of robot systems, safety flaws often stay hidden, especially at early design stages, when a physical implementation is not yet available for testing. Simulation-based testing is a possible way to identify hazards in an early stage. However, creating simulation conditions in which hazards become observable can be difficult. Brute-force or Monte-Carlo-approaches a…
A collision hazard measure that has the essential characteristics to provide a measurement of safety that will be useful to AV developers, traffic infrastructure developers and managers, regulators and the public is introduced here. The Streetscope Collision Hazard Measure (SHM) overcomes the limitations of existing measures, and provides an independent leading indication of safety. * Trailing indicators, such as collision statistics, incur pain and loss on society, and are not an ethically acceptable approach. * Near-misses have been shown to be effective predictors of incidents. * Time-to-Co…
Hazard detection and avoidance is a key technology for future robotic small body sample return and lander missions. Current state-of-the-practice methods rely on high-fidelity, a priori terrain maps, which require extensive human-in-the-loop verification and expensive reconnaissance campaigns to resolve mapping uncertainties. We propose a novel safety mapping paradigm that leverages deep semantic segmentation techniques to predict landing safety directly from a single monocular image, thus reducing reliance on high-fidelity, a priori data products. We demonstrate precise and accurate safety ma…
Safety controllers are system or software components responsible for handling risk in many machine applications. This tool paper describes a use case and a workflow for YAP, a research tool for risk modelling and discrete-event safety controller design. The goal of this use case is to derive a safety controller from hazard analysis and risk assessment, to define a design space for this controller, and to select a verified optimal controller instance from this design space. We represent this design space as a stochastic model and use YAP for risk modelling and generation of parts of this stocha…
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
We introduce a novel simulation-based approach to identify hazards that result from unexpected worker behavior in human-robot collaboration. Simulation-based safety testing must take into account the fact that human behavior is variable and that human error can occur. When only the expected worker behavior is simulated, critical hazards can remain undiscovered. On the other hand, simulating all possible worker behaviors is computationally infeasible. This raises the problem of how to find interesting (i.e., potentially hazardous) worker behaviors given a limited number of simulation runs. We f…
Runtime Safety Filtering for Two-Terminal Hazards in Robotic Battery Recycling
National University of Singapore（新加坡国立大学）；Griffith University（格里菲斯大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；Southern Cross University（南十字星大学）；Macquarie University（麦考瑞大学）
Robotics landscape is experiencing big changes. Robots are spreading and will soon be everywhere. Systems traditionally employed in industry are being replaced by collaborative robots, while more and more professional and consumer robots are introduced in people's daily activities. Robots are increasingly intertwined with other facets of IT and envisioned to get much more autonomy, interacting physically with humans. We claim that, following Personal Computers (PCs) and smartphones, robots are the next technological revolution and yet, robot security is being ignored by manufacturers. The pres…
University of Southern California（南加州大学）；Norwegian Defence Research Establishment（挪威国防研究院）
Autonomous underwater vehicles (AUVs) are robotic platforms that are commonly used to map the sea floor, for example for benthic surveys or for naval mine countermeasures (MCM) operations. AUVs create an acoustic image of the survey area, such that objects on the seabed can be identified and, in the case of MCM, mines can be found and disposed of. The common method for creating such seabed maps is to run a lawnmower survey, which is a standard method in coverage path planning. We are interested in exploring alternate techniques for surveying areas of interest, in order to reduce mission time o…
Systems-theoretic Safety Assessment of Robotic Telesurgical Systems
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Applied Dexterity（Applied Dexterity公司）；Rush University Medical Center（拉什大学医学中心）；Massachusetts Institute of Technology（麻省理工学院）
California Institute of Technology（加州理工学院）；NVIDIA（英伟达）；University of Washington（华盛顿大学）；Stanford University（斯坦福大学）
Drivers have a responsibility to exercise reasonable care to avoid collision with other road users. This assumed responsibility allows interacting agents to maintain safety without explicit coordination. Thus to enable safe autonomous vehicle (AV) interactions, AVs must understand what their responsibilities are to maintain safety and how they affect the safety of nearby agents. In this work we seek to understand how responsibility is shared in multi-agent settings where an autonomous agent is interacting with human counterparts. We introduce Responsibility-Aware Control Barrier Functions (RA-…
SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
Department of Computer Science and Engineering（计算机科学与工程系）；Shanghai Jiao Tong University（上海交通大学）；Department of Computer Science（计算机科学系）；University of Georgia（佐治亚大学）；Shanghai AI Laboratory（上海人工智能实验室）
UniSTPA: A Safety Analysis Framework for End-to-End Autonomous Driving
National Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University, Changchun, China（吉林大学汽车底盘集成与仿生国家级重点实验室）；School of Engineering and Physical Sciences, Heriot‐Watt University, Edinburgh, United Kingdom（赫瑞-沃森大学工程与物理科学学院）
During the planning phase of industrial robot workplaces, hazard analyses are required so that potential hazards for human workers can be identified and appropriate safety measures can be implemented. Existing hazard analysis methods use human reasoning, checklists and/or abstract system models, which limit the level of detail. We propose a new approach that frames hazard analysis as a search problem in a dynamic simulation environment. Our goal is to identify workplace hazards by searching for simulation sequences that result in hazardous situations. We solve this search problem by placing vi…
ARGOS: Automated Functional Safety Requirement Synthesis for Embodied AI via Attribute-Guided Combinatorial Reasoning
Southern University of Science and Technology, Shenzhen, China（南方科技大学, 深圳, 中国）；City University of Hong Kong, Hong Kong, China（香港城市大学, 香港, 中国）；The University of Tokyo, Tokyo, Japan（东京大学, 东京, 日本）；Lingnan University, Hong Kong, China（岭南大学, 香港, 中国）
Robotic and human lunar landings are a focus of future NASA missions. Precision landing capabilities are vital to guarantee the success of the mission, and the safety of the lander and crew. During the approach to the surface there are multiple challenges associated with Hazard Relative Navigation to ensure safe landings. This paper will focus on a passive autonomous hazard detection and avoidance sub-system to generate an initial assessment of possible landing regions for the guidance system. The system uses a single camera and the MobileNetV2 neural network architecture to detect and discern…
There is a desire to move towards more flexible and automated factories. To enable this, we need to assure the safety of these dynamic factories. This safety assurance must be achieved in a manner that does not unnecessarily constrain the systems and thus negate the benefits of flexibility and automation. We previously developed a modular safety assurance approach, using safety contracts, as a way to achieve this. In this case study we show how this approach can be applied to Autonomous Guided Vehicles (AGV) operating as part of a dynamic factory and why it is necessary. We empirically evaluat…
Communication technologies enable coordination among connected and autonomous vehicles (CAVs). However, it remains unclear how to utilize shared information to improve the safety and efficiency of the CAV system in dynamic and complicated driving scenarios. In this work, we propose a framework of constrained multi-agent reinforcement learning (MARL) with a parallel Safety Shield for CAVs in challenging driving scenarios that includes unconnected hazard vehicles. The coordination mechanisms of the proposed MARL include information sharing and cooperative policy learning, with Graph Convolutiona…
