Hamilton-Jacobi（HJ）可达性是一种严谨的数学框架，使机器人能够同时检测不安全状态并生成防止未来故障的动作。虽然在理论上，HJ可达性可以为非线性系统和非凸约束合成安全控制器，但实践中它已被限制在由低维状态空间表示和第一原理动力学建模的手工工程碰撞回避约束。在本工作中，我们的目标是将安全机器人控制器泛化，以防止那些难以手工编写但可通过高维观测直观识别的故障：例如，倒翻袋中的内容物。我们提出了潜在安全过滤器，这是HJ可达性的潜在空间泛化，能够直接在原始观测数据（如RGB图像）上有效运作，自动计算安全保持动作，而无需显式的恢复演示，通过在生成世界模型的潜在嵌入空间中进行安全性分析。我们的方法利用了不同质量和多样化的机器人观测-动作数据（包括成功案例、随机探索和不安全演示）来学习世界模型。约束规范然后被转换为在学习的世界模型的潜在空间中的分类问题。在模拟和硬件实验中，我们计算了Latent Safety Filters的近似值，以保护任意策略（从模仿学习策略到直接遥控）免受复杂安全危害的影响，例如防止Franka Research 3机械臂倒翻袋中的内容物或推倒杂乱物品。
英文摘要：
Hamilton-Jacobi (HJ) reachability is a rigorous mathematical framework that enables robots to simultaneously detect unsafe states and generate actions that prevent future failures. While in theory, HJ reachability can synthesize safe controllers for nonlinear systems and nonconvex constraints, in practice, it has been limited to hand-engineered collision-avoidance constraints modeled via low-dimensional state-space representations and first-principles dynamics. In this work, our goal is to generalize safe robot controllers to prevent failures that are hard--if not impossible--to write down by hand, but can be intuitively identified from high-dimensional observations: for example, spilling the contents of a bag. We propose Latent Safety Filters, a latent-space generalization of HJ reachability that tractably operates directly on raw observation data (e.g., RGB images) to automatically compute safety-preserving actions without explicit recovery demonstrations by performing safety analysis in the latent embedding space of a generative world model. Our method leverages diverse robot observation-action data of varying quality (including successes, random exploration, and unsafe demonstrations) to learn a world model. Constraint specification is then transformed into a classification problem in the latent space of the learned world model. In simulation and hardware experiments, we compute an approximation of Latent Safety Filters to safeguard arbitrary policies (from imitation- learned policies to direct teleoperation) from complex safety hazards, like preventing a Franka Research 3 manipulator from spilling the contents of a bag or toppling cluttered objects.
Hamilton--Jacobi Reachability for Spacecraft Collision Avoidance
Department of Mechanical Engineering, University of California, Berkeley（加州大学伯克利分校机械工程系）；Aerospace Engineering Program, University of California, Berkeley（加州大学伯克利分校航空航天工程项目）；Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）
The increasing prevalence of neural networks (NNs) in safety-critical applications calls for methods to certify their behavior and guarantee safety. This paper presents a backward reachability approach for safety verification of neural feedback loops (NFLs), i.e., closed-loop systems with NN control policies. While recent works have focused on forward reachability as a strategy for safety certification of NFLs, backward reachability offers advantages over the forward strategy, particularly in obstacle avoidance scenarios. Prior works have developed techniques for backward reachability analysis…
Recently there have been a lot of interests in introducing UAVs for a wide range of applications, making ensuring safety of multi-vehicle systems a highly crucial problem. Hamilton-Jacobi (HJ) reachability is a promising tool for analyzing safety of vehicles for low-dimensional systems. However, reachability suffers from the curse of dimensionality, making its direct application to more than two vehicles intractable. Recent works have made it tractable to guarantee safety for 3 and 4 vehicles with reachability. However, the number of vehicles safety can be guaranteed for remains small. In this…
This paper provides a general framework for efficiently obtaining the appropriate intervention time for collision avoidance systems to just avoid a rear-end crash. The proposed framework incorporates a driver comfort model and a vehicle model. We show that there is a relationship between driver steering manoeuvres based on acceleration and jerk, and steering angle and steering angle rate profiles. We investigate how four different vehicle models influence the time when steering needs to be initiated to avoid a rear-end collision. The models assessed were: a dynamic bicycle model (DM), a steady…
Generating safe motion plans in real-time is a key requirement for deploying robot manipulators to assist humans in collaborative settings. In particular, robots must satisfy strict safety requirements to avoid self-damage or harming nearby humans. Satisfying these requirements is particularly challenging if the robot must also operate in real-time to adjust to changes in its environment.This paper addresses these challenges by proposing Reachability-based Signed Distance Functions (RDFs) as a neural implicit representation for robot safety. RDF, which can be constructed using supervised learn…
Simon Fraser University（西蒙弗雷泽大学）；University of California, Berkeley（加州大学伯克利分校）
Safety is an important topic in autonomous driving since any collision may cause serious injury to people and damage to property. Hamilton-Jacobi (HJ) Reachability is a formal method that verifies safety in multi-agent interaction and provides a safety controller for collision avoidance. However, due to the worst-case assumption on the cars future behaviours, reachability might result in too much conservatism such that the normal operation of the vehicle is badly hindered. In this paper, we leverage the power of trajectory prediction and propose a prediction-based reachability framework to com…
NeuroHJR: Hamilton-Jacobi Reachability-based Obstacle Avoidance in Complex Environments with Physics-Informed Neural Networks
Department of Physical Science, Indian Institute of Science Education and Research（物理科学系，印度科学教育与研究学院）；Department of Aerospace Engineering, Indian Institute of Science（航空航天工程系，印度科学研究院）；Department of Computer Science and Engineering, Shiv Nadar University（计算机科学与工程系，施瓦斯纳大学）
Exact Imposition of Safety Boundary Conditions in Neural Reachable Tubes
ECE Department at University of Southern California（南加州大学电子工程系）；EE Department at Indian Institute of Technology Patna（印度理工学院帕纳布电子工程系）；Department of Aeronautics and Astronautics at Stanford University（斯坦福大学航空与航天工程系）
University of California, Berkeley（加州大学伯克利分校）；Facebook AI Research（脸书人工智能研究院）
Multi-vehicle collision avoidance is a highly crucial problem due to the soaring interests of introducing autonomous vehicles into the real world in recent years. The safety of these vehicles while they complete their objectives is of paramount importance. Hamilton-Jacobi (HJ) reachability is a promising tool for guaranteeing safety for low-dimensional systems. However, due to its exponential complexity in computation time, no reachability-based methods have been able to guarantee safety for more than three vehicles successfully in unstructured scenarios. For systems with four or more vehicles…
Mitigating Cross-Modal Distraction and Ensuring Geometric Feasibility via Affordance-Guided and Self-Consistent MLLMs for Task Planning in Instruction-Following Manipulation
Department of Computer Science, National Yang Ming Chiao Tung University（计算机科学系，阳明交通大学）；Department of Computer Science and Information Engineering, National Taiwan University（计算机科学与信息工程系，台湾大学）
Obstacle avoidance is a fundamental requirement for autonomous robots which operate in, and interact with, the real world. When perception is limited to monocular vision avoiding collision becomes significantly more challenging due to the lack of 3D information. Conventional path planners for obstacle avoidance require tuning a number of parameters and do not have the ability to directly benefit from large datasets and continuous use. In this paper, a dueling architecture based deep double-Q network (D3QN) is proposed for obstacle avoidance, using only monocular RGB vision. Based on the duelin…
Time-To-Reach Separation and Safety Filtering for Safe, Fair, and Efficient Multi-Agent Coordination
Department of Electrical Engineering and Computer Sciences, University of California, Berkeley（加州大学伯克利分校电子工程与计算机科学系）；Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（麻省理工学院航空与航天系）；GRASP Laboratory, University of Pennsylvania（宾夕法尼亚大学GRASP实验室）；Department of Electrical and Computer Engineering, University of California, Los Angeles（加州大学洛杉矶分校电子与计算机工程系）
Although extensive research in emergency collision avoidance has been carried out for straight or curved roads in a highway scenario, a general method that could be implemented for all road environments has not been thoroughly explored. Moreover, most current algorithms don't consider collision mitigation in an emergency. This functionality is essential since the problem may have no feasible solution. We propose a safe controller using model predictive control and artificial potential function to address these problems. A new artificial potential function inspired by line charge is proposed as…
Hamilton-Jacobi reachability analysis is a powerful technique used to verify the safety of autonomous systems. This method is very good at handling non-linear system dynamics with disturbances and flexible set representations. A drawback to this approach is that it suffers from the curse of dimensionality, which prevents real-time deployment on safety-critical systems. In this paper, we show that a customized hardware design on a Field Programmable Gate Array (FPGA) could accelerate 4D grid-based Hamilton-Jacobi (HJ) reachability analysis up to 16 times compared to an optimized implementation …
VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
Department of Automation, Tsinghua University（清华大学自动化系）；Institute for Embodied Intelligence and Robotics, Tsinghua University（清华大学智能与机器人研究院）；TetraBOT；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
