Providing non-trivial certificates of safety for non-linear stochastic systems is an important open problem that limits the wider adoption of autonomous systems in safety-critical applications. One promising solution to address this problem is barrier functions. The composition of a barrier function with a stochastic system forms a supermartingale, thus enabling the computation of the probability that the system stays in a safe set over a finite time horizon via martingale inequalities. However, existing approaches to find barrier functions for stochastic systems generally rely on convex optimization programs that restrict the search of a barrier to a small class of functions such as low degree SoS polynomials and can be computationally expensive. In this paper, we parameterize a barrier function as a neural network and show that techniques for robust training of neural networks can be successfully employed to find neural barrier functions. Specifically, we leverage bound propagation techniques to certify that a neural network satisfies the conditions to be a barrier function via linear programming and then employ the resulting bounds at training time to enforce the satisfaction of these conditions. We also present a branch-and-bound scheme that makes the certification framework scalable. We show that our approach outperforms existing methods in several case studies and often returns certificates of safety that are orders of magnitude larger.
This paper introduces differentiable higher-order control barrier functions (CBF) that are end-to-end trainable together with learning systems. CBFs are usually overly conservative, while guaranteeing safety. Here, we address their conservativeness by softening their definitions using environmental dependencies without loosing safety guarantees, and embed them into differentiable quadratic programs. These novel safety layers, termed a BarrierNet, can be used in conjunction with any neural network-based controller, and can be trained by gradient descent. BarrierNet allows the safety constraints…
University of California, Berkeley（加州大学伯克利分校）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）
In this paper, we consider the problem of certifying the robustness of neural networks to perturbed and adversarial input data. Such certification is imperative for the application of neural networks in safety-critical decision-making and control systems. Certification techniques using convex optimization have been proposed, but they often suffer from relaxation errors that void the certificate. Our work exploits the structure of ReLU networks to improve relaxation errors through a novel partition-based certification procedure. The proposed method is proven to tighten existing linear programmi…
Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Design
Department of Computer Science Iowa State University（计算机科学系爱荷华州立大学）；Department of Civil, Construction and Environmental Engineering Iowa State University（土木、建设与环境工程系爱荷华州立大学）
Methods to certify the robustness of neural networks in the presence of input uncertainty are vital in safety-critical settings. Most certification methods in the literature are designed for adversarial or worst-case inputs, but researchers have recently shown a need for methods that consider random input noise. In this paper, we examine the setting where inputs are subject to random noise coming from an arbitrary probability distribution. We propose a robustness certification method that lower-bounds the probability that network outputs are safe. This bound is cast as a chance-constrained opt…
We propose an adaptive Model Predictive Safety Certification (MPSC) scheme for learning-based control of linear systems with bounded disturbances and uncertain parameters where the true parameters are contained within an a priori known set of parameters. An MPSC is a modular framework which can be used in combination with any learning-based controller to ensure state and input constraint satisfaction of a dynamical system by solving an online optimisation problem. By continuously connecting the current system state with a safe terminal set using a robust tube, safety can be ensured. Thereby, t…
Reinforcement learning (RL) methods have demonstrated their efficiency in simulation environments. However, many applications for which RL offers great potential, such as autonomous driving, are also safety critical and require a certified closed-loop behavior in order to meet safety specifications in the presence of physical constraints. This paper introduces a concept, called probabilistic model predictive safety certification (PMPSC), which can be combined with any RL algorithm and provides provable safety certificates in terms of state and input chance constraints for potentially large-sca…
Challenges related to automated driving are no longer focused on just the construction of such automated vehicles (AVs), but in assuring the safety of their operation. Recent advances in Level 3 and Level 4 autonomous driving have motivated more extensive study in safety guarantees of complicated AV maneuvers, which aligns with the goal of ISO 21448 (Safety of the Intended Functions, or SOTIF), i.e. minimizing unsafe scenarios both known and unknown, as well as Vision Zero -- eliminating highway fatalities by 2050. A majority of approaches used in providing safety guarantees for AV motion cont…
While distributed algorithms provide advantages for the control of complex large-scale systems by requiring a lower local computational load and less local memory, it is a challenging task to design high-performance distributed control policies. Learning-based control algorithms offer promising opportunities to address this challenge, but generally cannot guarantee safety in terms of state and input constraint satisfaction. A recently proposed safety framework for centralized linear systems ensures safety by matching the learning-based input online with the initial input of a model predictive …
Northwestern University（西北大学）；University of Liverpool（利物浦大学）
The robustness of deep neural networks has received significant interest recently, especially when being deployed in safety-critical systems, as it is important to analyze how sensitive the model output is under input perturbations. While most previous works focused on the local robustness property around an input sample, the studies of the global robustness property, which bounds the maximum output change under perturbations over the entire input space, are still lacking. In this work, we formulate the global robustness certification for neural networks with ReLU activation functions as a mix…
One of the major impediments in deployment of Autonomous Driving Systems (ADS) is their safety and reliability. The primary reason for the complexity of testing ADS is that it operates in an open world characterized by its non-deterministic, high-dimensional and non-stationary nature where the actions of other actors in the environment are uncontrollable from the ADS's perspective. This leads to a state space explosion problem and one way of mitigating this problem is by concretizing the scope for the system under test (SUT) by testing for a set of behavioral competencies which an ADS must dem…
The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）
Safety guarantee is essential in many engineering implementations. Reinforcement learning provides a useful way to strengthen safety. However, reinforcement learning algorithms cannot completely guarantee safety over realistic operations. To address this issue, this work adopts control barrier functions over reinforcement learning, and proposes a compensated algorithm to completely maintain safety. Specifically, a sum-of-squares programming has been exploited to search for the optimal controller, and tune the learning hyperparameters simultaneously. Thus, the control actions are pledged to be …
Pacific Northwest National Laboratory（太平洋西北国家实验室）；Battelle Memorial Institute（巴特尔纪念研究所）
We develop a novel form of differentiable predictive control (DPC) with safety and robustness guarantees based on control barrier functions. DPC is an unsupervised learning-based method for obtaining approximate solutions to explicit model predictive control (MPC) problems. In DPC, the predictive control policy parametrized by a neural network is optimized offline via direct policy gradients obtained by automatic differentiation of the MPC problem. The proposed approach exploits a new form of sampled-data barrier function to enforce offline and online safety requirements in DPC settings while …
Safety-critical applications require controllers/policies that can guarantee safety with high confidence. The control barrier function is a useful tool to guarantee safety if we have access to the ground-truth system dynamics. In practice, we have inaccurate knowledge of the system dynamics, which can lead to unsafe behaviors due to unmodeled residual dynamics. Learning the residual dynamics with deterministic machine learning models can prevent the unsafe behavior but can fail when the predictions are imperfect. In this situation, a probabilistic learning method that reasons about the uncerta…
Modern nonlinear control theory seeks to endow systems with properties of stability and safety, and have been deployed successfully in multiple domains. Despite this success, model uncertainty remains a significant challenge in synthesizing safe controllers, leading to degradation in the properties provided by the controllers. This paper develops a machine learning framework utilizing Control Barrier Functions (CBFs) to reduce model uncertainty as it impact the safe behavior of a system. This approach iteratively collects data and updates a controller, ultimately achieving safe behavior. We va…
Recent advances in artificial intelligence and machine learning may soon yield paradigm-shifting benefits for aerospace systems. However, complexity and possible continued on-line learning makes neural network control systems (NNCS) difficult or impossible to certify under the United States Military Airworthiness Certification Criteria defined in MIL-HDBK-516C. Run time assurance (RTA) is a control system architecture designed to maintain safety properties regardless of whether a primary control system is fully verifiable. This work examines how to satisfy compliance with MIL-HDBK-516C while u…
While learning-based control techniques often outperform classical controller designs, safety requirements limit the acceptance of such methods in many applications. Recent developments address this issue through so-called predictive safety filters, which assess if a proposed learning-based control input can lead to constraint violations and modifies it if necessary to ensure safety for all future time steps. The theoretical guarantees of such predictive safety filters rely on the model assumptions and minor deviations can lead to failure of the filter putting the system at risk. This paper in…
Deploying safety-critical controllers in practice necessitates the ability to modulate uncertainties in control systems. In this context, robust control barrier functions -- in a variety of forms -- have been used to obtain safety guarantees for uncertain systems. Yet the differing types of uncertainty experienced in practice have resulted in a fractured landscape of robustification -- with a variety of instantiations depending on the structure of the uncertainty. This paper proposes a framework for generalizing these variations into a single form: parameterized barrier functions (PBFs), which…
Control barrier functions have been widely used for synthesizing safety-critical controls, often via solving quadratic programs. However, the existence of Gaussian-type noise may lead to unsafe actions and result in severe consequences. In this paper, we study systems modeled by stochastic differential equations (SDEs) driven by Brownian motions. We propose a notion of stochastic control barrier functions (SCBFs)and show that SCBFs can significantly reduce the control efforts, especially in the presence of noise, compared to stochastic reciprocal control barrier functions (SRCBFs), and offer a…
This work provides formal safety guarantees for control systems with disturbance. A disturbance observer-based robust safety-critical controller is proposed, that estimates the effect of the disturbance on safety and utilizes this estimate with control barrier functions to attain provably safe dynamic behavior. The observer error bound - which consists of transient and steady-state parts - is quantified, and the system is endowed with robustness against this error via the proposed controller. An adaptive cruise control problem is used as illustrative example through simulations including real …
Control barrier functions (CBFs) have become a popular tool to enforce safety of a control system. CBFs are commonly utilized in a quadratic program formulation (CBF-QP) as safety-critical constraints. A class $\mathcal{K}$ function in CBFs usually needs to be tuned manually in order to balance the trade-off between performance and safety for each environment. However, this process is often heuristic and can become intractable for high relative-degree systems. Moreover, it prevents the CBF-QP from generalizing to different environments in the real world. By embedding the optimization procedure…
In this paper we seek to quantify the ability of learning to improve safety guarantees endowed by Control Barrier Functions (CBFs). In particular, we investigate how model uncertainty in the time derivative of a CBF can be reduced via learning, and how this leads to stronger statements on the safe behavior of a system. To this end, we build upon the idea of Input-to-State Safety (ISSf) to define Projection-to-State Safety (PSSf), which characterizes degradation in safety in terms of a projected disturbance. This enables the direct quantification of both how learning can improve safety guarante…
For a broad class of nonlinear systems, we formulate the problem of guaranteeing safety with optimality under constraints. Specifically, we define controlled safety for differential inclusions with constraints on the states and the inputs. Through the use of nonsmooth analysis tools, we show that a continuous optimal control law can be selected from a set-valued constraint capturing the system constraints and conditions guaranteeing safety using control barrier functions. Our results guarantee optimality and safety via a continuous state-feedback law designed using nonsmooth control barrier fu…
In this paper, we propose the safety barrier certificates for uncertain multi-robot systems playing red light, green light game. According to the rule of the game, the robots are allowed to move forward after a doll shouts `green light' and must stop when it shouts `red light'. Following this rule, a two-mode nominal controller is designed where one mode is for moving forward and the other one is for slowing down and being motionless. Then, multiple exponential control barrier functions(ECBFs) are developed to handle safety constraints for limited playground, collision avoidance, and saturatio…
Adaptive Control Lyapunov Functions (aCLFs) were introduced 20 years ago, and provided a Lyapunov-based methodology for stabilizing systems with parameter uncertainty. The goal of this paper is to revisit this classic formulation in the context of safety-critical control. This will motivate a variant of aCLFs in the context of safety: adaptive Control Barrier Functions (aCBFs). Our proposed approach adaptively achieves safety by keeping the systems state within a safe set even in the presence of parametric model uncertainty. We unify aCLFs and aCBFs into a single control methodology for system…
In many safety-critical control systems, possibly opposing safety restrictions and control performance objectives arise. To confront such a conflict, this letter proposes a novel methodology that embeds safety into stability of control systems. The development enforces safety by means of barrier functions used in optimization through the construction of barrier states (BaS) which are embedded in the control system's model. As a result, as long as the equilibrium point of interest of the closed loop system is asymptotically stable, the generated trajectories are guaranteed to be safe. Consequen…
This report aims to compare two safety methods: control barrier function and Hamilton-Jacobi reachability analysis. We will consider the difference with a focus on the following aspects: generality of system dynamics, difficulty of construction and computation cost. A standard Dubins car model will be evaluated numerically to make the comparison more concrete.
Safety is one of the fundamental problems in robotics. Recently, a quadratic program-based control barrier function (CBF) method has emerged as a way to enforce safety-critical constraints. Together with control Lyapunov function (CLF), it forms a safety-critical control strategy, named CLF-CBF-QP, which can mediate between achieving the control objective and ensuring safety, while being executable in real-time. However, once additional constraints such as input constraints are introduced, the CLF-CBF-QP may encounter infeasibility. In order to address the challenge that arises due to the infe…
The safety-critical nature of adaptive cruise control (ACC) systems calls for systematic design procedures, e.g., based on formal methods or control barrier functions (CBFs), to provide strong guarantees of safety and performance under all driving conditions. However, existing approaches have mostly focused on fully verified solutions under smooth traffic conditions, with the exception of stop-and-go scenarios. Systematic methods for high-performance ACC design under safety and regulatory constraints like traffic signals are still elusive. A challenge for correct-by-construction approaches bas…
Dynamic Constraint Reconstruction Based Control Barrier Functions for Safety-Critical Control of High-Dimensional Manipulators
Yaowu (Shenzhen) Technology Co., Ltd.（耀武（深圳）科技有限公司）；National Engineering Research Center of Mobile Network Technologies, Beijing University of Posts and Telecommunications（北京邮电大学移动通信国家重点实验室）
Control barrier functions are widely used to synthesize safety-critical controls. The existence of Gaussian-type noise may lead to unsafe actions and result in severe consequences. While studies are widely done in safety-critical control for stochastic systems, in many real-world applications, we do not have the knowledge of the stochastic component of the dynamics. In this paper, we study safety-critical control of stochastic systems with an unknown diffusion part and propose a data-driven method to handle these scenarios. More specifically, we propose a data-driven stochastic control barrier…
This work proposes a sliding mode control barrier function to robustly deal with high relative-degree safety constraints in safety-critical control systems. Stability/tracking objectives, expressed as a nominal control law, and safety constraints, expressed as control barrier functions are unified through quadratic programming. The proposed control framework is numerically validated considering a Furuta pendulum and a magnetic levitation system. For the first system, a linear quadratic regulator is considered as a nominal control law, and a safety constraint is considered to guarantee that the…
