Reinforcement learning has been established over the past decade as an effective tool to find optimal control policies for dynamical systems, with recent focus on approaches that guarantee safety during the learning and/or execution phases. In general, safety guarantees are critical in reinforcement learning when the system is safety-critical and/or task restarts are not practically feasible. In optimal control theory, safety requirements are often expressed in terms of state and/or control constraints. In recent years, reinforcement learning approaches that rely on persistent excitation have been combined with a barrier transformation to learn the optimal control policies under state constraints. To soften the excitation requirements, model-based reinforcement learning methods that rely on exact model knowledge have also been integrated with the barrier transformation framework. The objective of this paper is to develop safe reinforcement learning method for deterministic nonlinear systems, with parametric uncertainties in the model, to learn approximate constrained optimal policies without relying on stringent excitation conditions. To that end, a model-based reinforcement learning technique that utilizes a novel filtered concurrent learning method, along with a barrier transformation, is developed in this paper to realize simultaneous learning of unknown model parameters and approximate optimal state-constrained control policies for safety-critical systems.
We propose a tube-based guaranteed cost model predictive controller considering a homothetic formulation for constrained linear systems subject to multiplicative structured norm-bounded uncertainties. It provides an upper bound to the general min-max model predictive control. The invariance property of the proposed tube holds for any arbitrary scaling. It yields a second-order cone programming problem which is less computationally expensive than standard semi-definite programming problems. We also present a numerical example with a comparative study among the proposed approach, an open-loop gu…
This paper presents a framework for designing provably safe feedback controllers for sampled-data control affine systems with measurement and actuation uncertainties. Based on the interval Taylor model of nonlinear functions, a sampled-data control barrier function (CBF) condition is proposed which ensures the forward invariance of a safe set for sampled-data systems. Reachable set overapproximation and Lasserre's hierarchy of polynomial optimization are used for finding the margin term in the sampled-data CBF condition. Sufficient conditions for a safe controller in the presence of measuremen…
We address the issue of safe optimal path planning under parametric uncertainties using a novel regularizer that allows trading off optimality with safety. The proposed regularizer leverages the notion that collisions may be modeled as constraint violations in an optimal control setting in order to produce open-loop trajectories with reduced risk of collisions. The risk of constraint violation is evaluated using a state-dependent relevance function and first-order variations in the constraint function with respect to parametric variations. The approach is generic and can be adapted to any opti…
Input-to-State Safe Backstepping: Robust Safety-Critical Control with Unmatched Uncertainties
Department of Electrical and Computer Engineering, North Carolina State University（电气与计算机工程系，北卡罗来纳州立大学）；Department of Mechanical and Civil Engineering, California Institute of Technology（机械与土木工程系，加州理工学院）
We present a novel technique for solving the problem of safe control for a general class of nonlinear, control-affine systems subject to parametric model uncertainty. Invoking Lyapunov analysis and the notion of fixed-time stability (FxTS), we introduce a parameter adaptation law which guarantees convergence of the estimates of unknown parameters in the system dynamics to their true values within a fixed-time independent of the initial parameter estimation error. We then synthesize the adaptation law with a robust, adaptive control barrier function (RaCBF) based quadratic program to compute sa…
This paper focuses on safety critical control with sector-bounded uncertainties at the plant input. The uncertainties can represent nonlinear and/or time-varying components. We propose a new robust control barrier function (RCBF) approach to enforce safety requirements in the presence of these uncertainties. The primary objective is to minimally alter the given baseline control command to guarantee safety in the presence of modeled uncertainty. The resulting min-norm optimization problem can be recast as a Second-Order Cone Program (SOCP) to enable online implementation. Properties of this con…
Supply chain security threats pose new challenges to security risk modeling techniques for complex ICT systems such as the IoT. With established techniques drawn from attack trees and reliability analysis providing needed points of reference, graph-based analysis can provide a framework for considering the role of suppliers in such systems. We present such a framework here while highlighting the need for a component-centered model. Given resource limitations when applying this model to existing systems, we study various classes of uncertainties in model development, including structural uncert…
Georgia Institute of Technology（佐治亚理工学院）；Meiji University（明治大学）
Inspired by the success of control barrier functions (CBFs) in addressing safety, and the rise of data-driven techniques for modeling functions, we propose a non-parametric approach for online synthesis of CBFs using Gaussian Processes (GPs). Mathematical constructs such as CBFs have achieved safety by designing a candidate function a priori. However, designing such a candidate function can be challenging. A practical example of such a setting would be to design a CBF in a disaster recovery scenario where safe and navigable regions need to be determined. The decision boundary for safety in suc…
Computer Science and Artificial Intelligence Laboratory (CSAIL)（计算机科学与人工智能实验室（CSAIL））；Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT））
In this paper, we address the real-time risk-bounded safety verification problem of continuous-time state trajectories of autonomous systems in the presence of uncertain time-varying nonlinear safety constraints. Risk is defined as the probability of not satisfying the uncertain safety constraints. Existing approaches to address the safety verification problems under uncertainties either are limited to particular classes of uncertainties and safety constraints, e.g., Gaussian uncertainties and linear constraints, or rely on sampling based methods. In this paper, we provide a fast convex algori…
Safe control of constrained linear systems under both epistemic and aleatory uncertainties is considered. The aleatory uncertainty characterizes random noises and is modeled by a probability distribution function (PDF) and the epistemic uncertainty characterizes the lack of knowledge on the system dynamics. Data-based probabilistic safe controllers are designed for the cases where the noise PDF is 1) zero-mean Gaussian with a known covariance, 2) zero-mean Gaussian with an uncertain covariance, and 3) zero-mean non-Gaussian with an unknown distribution. Easy-to-check model-based conditions for…
Interval Prediction for Continuous-Time Systems with Parametric Uncertainties
Renault（雷诺）；Inria, Univ. Lille, CNRS, UMR 9189 - CRIStAL（法国国家信息与自动化研究所，里尔大学，法国国家科学研究中心，联合研究实验室9189）；ITMO University（信息技术、机械与光学大学）；Conservatoire National des Arts et Métiers (CNAM)（国立工艺学院）；Centrale Lille, CNRS, UMR 9189 - CRIStAL（里尔中央理工学院，法国国家科学研究中心，联合研究实验室9189）
We consider the task of motion control for non-prehensile manipulation using parallel kinematics mechatronic setup, in particular, stabilization of a ball on a plate under unmeasured external harmonic disturbances. System parameters are assumed to be unknown, and only a ball position is measurable with a resistive touch sensor. To solve the task we propose a novel passivity-based output control algorithm that can be implemented for unstable linearized systems of an arbitrary relative degree. In contrast to previous works, we describe a new way to parametrize harmonic signal generators and an e…
Uncertainties in Deep Neural Network (DNN)-based perception and vehicle's motion pose challenges to the development of safe autonomous driving vehicles. In this paper, we propose a safe motion planning framework featuring the quantification and propagation of DNN-based perception uncertainties and motion uncertainties. Contributions of this work are twofold: (1) A Bayesian Deep Neural network model which detects 3D objects and quantitatively captures the associated aleatoric and epistemic uncertainties of DNNs; (2) An uncertainty-aware motion planning algorithm (PU-RRT) that accounts for uncer…
The trade-off between optimality and complexity has been one of the most important challenges in the field of robust Model Predictive Control (MPC). To address the challenge, we propose a flexible robust MPC scheme by synergizing the multi-stage and tube-based MPC approaches. The key idea is to exploit the non-conservatism of the multi-stage MPC and the simplicity of the tube-based MPC. The proposed scheme provides two options for the user to determine the trade-off depending on the application: the choice of the robust horizon and the classification of the uncertainties. Beyond the robust hor…
Robots operating in real world settings must navigate and maintain safety while interacting with many heterogeneous agents and obstacles. Multi-Agent Control Barrier Functions (CBF) have emerged as a computationally efficient tool to guarantee safety in multi-agent environments, but they assume perfect knowledge of both the robot dynamics and other agents' dynamics. While knowledge of the robot's dynamics might be reasonably well known, the heterogeneity of agents in real-world environments means there will always be considerable uncertainty in our prediction of other agents' dynamics. This wo…
In this paper, we study the leaderless consensus problem for multiple Lagrangian systems in the presence of parametric uncertainties and external disturbances under directed graphs. For achieving asymptotic behavior, a robust continuous term with adaptive varying gains is added to alleviate the effects of the external disturbances with unknown bounds. In the case of a fixed directed graph, by introducing an integrate term in the auxiliary variable design, the final consensus equilibrium can be explicitly derived. We show that the agents achieve weighted average consensus, where the final equil…
Modern and future high precision pointing space missions face increasingly high challenges related to the widespread use of large flexible structures. The development of new modeling tools which are able to account for the multidisciplinary nature of this problem becomes extremely relevant in order to meet both structure and control performance criteria. This paper proposes a novel methodology to analytically model large truss structures in a sub-structuring framework. A three dimensional unit cube element has been designed and validated with a Finite Element commercial software. This model is…
Robust Safety-Critical Control of Networked SIR Dynamics
Elmore Family School of Electrical and Computer Engineering at Purdue University（普渡大学埃尔莫电气与计算机工程学院）；Department of Electrical Engineering and Computer Science at the University of California, Irvine（加州大学尔湾分校电气与计算机工程系）
This work investigates the finite-horizon optimal covariance steering problem for discrete-time linear systems subject to both additive and multiplicative uncertainties as well as state and input chance constraints. In particular, a tractable convex approximation of the optimal covariance steering problem is developed by tightening the chance constraints and by introducing a suitable change of variables. The solution of the convex approximation is shown to be a valid (albeit potentially suboptimal) solution to the original chance-constrained covariance steering problem.
To bring complex systems into real world environments in a safe manner, they will have to be robust to uncertainties - both in the environment and the system. This paper investigates the safety of control systems under input disturbances, wherein the disturbances can capture uncertainties in the system. Safety, framed as forward invariance of sets in the state space, is ensured with the framework of control barrier functions (CBFs). Concretely, the definition of input to state safety (ISSf) is generalized to allow the synthesis of non-conservative, tunable controllers that are provably safe un…
This paper addresses the problem of computing controllers that are correct by design for safety-critical systems and can provably satisfy (complex) functional requirements. We develop new methods for models of systems subject to both stochastic and parametric uncertainties. We provide for the first time novel simulation relations for enabling correct-by-design control refinement, that are founded on coupling uncertainties of stochastic systems via sub-probability measures. Such new relations are essential for constructing abstract models that are related to not only one model but to a set of p…
Considering uncertainties and disturbances is an important, yet challenging, step in successful decision making. The problem becomes more challenging in safety-constrained environments. In this paper, we propose a robust and safe trajectory optimization algorithm through solving a constrained min-max optimal control problem. The proposed method leverages a game theoretic differential dynamic programming approach with barrier states to handle parametric and non-parametric uncertainties in safety-critical control systems. Barrier states are embedded into the differential game's dynamics and cost…
Input constraints as well as parametric uncertainties must be accounted for in the design of safe control systems. This paper presents an adaptive controller for multiple-input-multiple-output (MIMO) plants with input magnitude and rate saturation in the presence of parametric uncertainties. A filter is introduced in the control path to accommodate the presence of rate limits. An output feedback adaptive controller is designed to stabilize the closed loop system even in the presence of this filter. The overall control architecture includes adaptive laws that are modified to account for the mag…
Stability certification and identifying a safe and stabilizing initial set are two important concerns in ensuring operational safety, stability, and robustness of dynamical systems. With the advent of machine-learning tools, these issues need to be addressed for the systems with machine-learned components in the feedback loop. To develop a general theory for stability and stabilizability of a neural network (NN)-controlled nonlinear system subject to bounded parametric variation, a Lyapunov-based stability certificate is proposed and is further used to devise a maximal Lipschitz bound for the …
Katholieke Universiteit Leuven（鲁汶天主教大学）；University of Melbourne（墨尔本大学）
We propose a safety-guaranteed planning and control framework for unmanned surface vessels (USVs), using Gaussian processes (GPs) to learn uncertainties. The uncertainties encountered by USVs, including external disturbances and model mismatches, are potentially state-dependent, time-varying, and hard to capture with constant models. GP is a powerful learning-based tool that can be integrated with a model-based planning and control framework, which employs a Hamilton-Jacobi differential game formulation. Such a combination yields less conservative trajectories and safety-guaranteeing control s…
This paper considers the safety-critical control design problem with output measurements. An observer-based safety control framework that integrates the estimation error quantified observer and the control barrier function (CBF) approach is proposed. The function approximation technique is employed to approximate the uncertainties introduced by the state estimation error, and an adaptive CBF approach is proposed to design the safe controller which is obtained by solving a convex quadratic program (QP). Theoretical results for CBFs with a relative degree 1 and a higher relative degree are given…
Massachusetts Institute of Technology（麻省理工学院）；Ford Motor Company（福特汽车公司）
Density of the reachable states can help understand the risk of safety-critical systems, especially in situations when worst-case reachability is too conservative. Recent work provides a data-driven approach to compute the density distribution of autonomous systems' forward reachable states online. In this paper, we study the use of such approach in combination with model predictive control for verifiable safe path planning under uncertainties. We first use the learned density distribution to compute the risk of collision online. If such risk exceeds the acceptable threshold, our method will p…
This paper involves the innovative development of a symmetric Gauss-Seidel ADMM algorithm to solve the H-infinity guaranteed cost control problem. In the presence of parametric uncertainties, the H-infinity guaranteed cost control problem generally leads to the large-scale optimization. This is due to the exponential growth of the number of the extreme systems involved with respect to the number of parametric uncertainties. In this work, through a variant of the Youla-Kucera parameterization, the stabilizing controllers are parameterized in a convex set; yielding the outcome that the H-infinit…
