The objective of this research is to enable safety-critical systems to simultaneously learn and execute optimal control policies in a safe manner to achieve complex autonomy. Learning optimal policies via trial and error, i.e., traditional reinforcement learning, is difficult to implement in safety-critical systems, particularly when task restarts are unavailable. Safe model-based reinforcement learning techniques based on a barrier transformation have recently been developed to address this problem. However, these methods rely on full state feedback, limiting their usability in a real-world environment. In this work, an output-feedback safe model-based reinforcement learning technique based on a novel barrier-aware dynamic state estimator has been designed to address this issue. The developed approach facilitates simultaneous learning and execution of safe control policies for safety-critical linear systems. Simulation results indicate that barrier transformation is an effective approach to achieve online reinforcement learning in safety-critical systems using output feedback.
Safety and performance are often two competing objectives in sequential decision-making problems. Existing performant controllers, such as controllers derived from reinforcement learning algorithms, often fall short of safety guarantees. On the contrary, controllers that guarantee safety, such as those derived from classical control theory, require restrictive assumptions and are often conservative in performance. Our goal is to blend a performant and a safe controller to generate a single controller that is safer than the performant and accumulates higher rewards than the safe controller. To …
In this work, we present a compositional safety controller synthesis approach for the class of discrete-time linear control systems. Here, we leverage a state-of-the-art result on the computation of robust controlled invariant sets. To tackle the complexity of controller synthesis over complex interconnected systems, this paper introduces a decentralized controller synthesis scheme. Rather than treating the interconnected system as a whole, we first design local safety controllers for each subsystem separately to enforce local safety properties, with polytopic state and input constraints as we…
In this paper, we propose a construction scheme for a Safe-visor architecture for sandboxing unverified controllers, e.g., artificial intelligence-based (a.k.a. AI-based) controllers, in two-players non-cooperative stochastic games. Concretely, we leverage abstraction-based approaches to construct a supervisor that checks and decides whether or not to accept the inputs provided by the unverified controller, and a safety advisor that provides fallback control inputs to ensure safety whenever the unverified controller is rejected. Moreover, by leveraging an ($ε,δ$)-approximate probabilistic rela…
High performance but unverified controllers, e.g., artificial intelligence-based (a.k.a. AI-based) controllers, are widely employed in cyber-physical systems (CPSs) to accomplish complex control missions. However, guaranteeing the safety and reliability of CPSs with this kind of controllers is currently very challenging, which is of vital importance in many real-life safety-critical applications. To cope with this difficulty, we propose in this work a Safe-visor architecture for sandboxing unverified controllers in CPSs operating in noisy environments (a.k.a. stochastic CPSs). The proposed arc…
Saturations in control signal can challenge the stability proof of a feedback-linearization-based controller, even leading the system unstable [1]. Thus, several approaches are established to avoid reaching the saturation bound [2,3]. Meanwhile, to help design the controller for a quad-tilt-rotor, [1] modeled a tilt vehicle with implementing the feedback-linearization-based controllers. In this article, we provide a gait plan for this tilt vehicle and control it utilizing the feedback linearization. Since saturations exist in the control signals, we study the stability based on Lyapunov theory…
We introduce a novel learning-based approach to synthesize safe and robust controllers for autonomous Cyber-Physical Systems and, at the same time, to generate challenging tests. This procedure combines formal methods for model verification with Generative Adversarial Networks. The method learns two Neural Networks: the first one aims at generating troubling scenarios for the controller, while the second one aims at enforcing the safety constraints. We test the proposed method on a variety of case studies.
To bring complex systems into real world environments in a safe manner, they will have to be robust to uncertainties - both in the environment and the system. This paper investigates the safety of control systems under input disturbances, wherein the disturbances can capture uncertainties in the system. Safety, framed as forward invariance of sets in the state space, is ensured with the framework of control barrier functions (CBFs). Concretely, the definition of input to state safety (ISSf) is generalized to allow the synthesis of non-conservative, tunable controllers that are provably safe un…
We consider the optimal distributed controller design problem subject to two structural requirements: locality, i.e. available measurements and sub-controllers' interactions are governed by a graph structure, and relative feedback, i.e. only differences of measurements are available to the controller. We formalize controller locality in terms of the controller's transfer function, state-space realization, or resulting closed-loop mapping. We demonstrate that the relative feedback requirement can be written as a convex constraint on the controller and (in special cases) on the resulting closed-…
Neural networks have been increasingly applied for control in learning-enabled cyber-physical systems (LE-CPSs) and demonstrated great promises in improving system performance and efficiency, as well as reducing the need for complex physical models. However, the lack of safety guarantees for such neural network based controllers has significantly impeded their adoption in safety-critical CPSs. In this work, we propose a controller adaptation approach that automatically switches among multiple controllers, including neural network controllers, to guarantee system safety and improve energy effic…
This paper investigates the car-following problem and proposes a nonlinear controller that considers driving comfort, safety concerns, steady-state response and transient response. This controller is designed based on the demands of lower cost, faster response, increased comfort, enhanced safety and elevated extendability from the automotive industry. Design insights and intuitions are provided in detail. Also, theoretical analysis are performed on plant stability, string stability and tracking performance of the closed-loop system. Conditions and guidelines are provided on the selection of co…
Switching control strategies that unite a potentially high-performance but uncertified controller and a stabilizing albeit conservative controller are shown to be able to balance safety with efficiency, but have been less studied under partial observation of state. To address this gap, we propose a switching control strategy for partially observed linear-Gaussian systems with provable performance guarantees. We show that the proposed switching strategy is both safe and efficient, in the sense that: (1) the linear-quadratic cost of the system is always bounded even if the original uncertified c…
The design of an observer-based state feedback (OBSF) controller with guaranteed passivity properties for port-Hamiltonian systems (PHS) is addressed using linear matrix inequalities (LMIs). The observer gain is freely chosen and the LMIs conditions such that the state feedback is equivalent to control by interconnection with an input strictly passive (ISP) and/or an output strictly passive (OSP) and zero state detectable (ZSD) port-Hamiltonian controller are established. It is shown that the proposed controller exponentially stabilizes a class of infinite-dimensional PHS and asymptotically st…
This paper proposes an on-policy reinforcement learning (RL) control algorithm that solves the optimal regulation problem for a class of uncertain continuous-time nonlinear systems under user-defined state constraints. We formulate the safe RL problem as the minimization of the Hamiltonian subject to a constraint on the time-derivative of a barrier Lyapunov function (BLF). We subsequently use the analytical solution of the optimization problem to modify the Actor-Critic-Identifier architecture to learn the optimal control policy safely. The proposed method does not require the presence of exte…
We consider abstraction-based design of output-feedback controllers for dynamical systems with a finite set of inputs and outputs against specifications in linear-time temporal logic. The usual procedure for abstraction-based controller design (ABCD) first constructs a finite-state abstraction of the underlying dynamical system, and second, uses reactive synthesis techniques to compute an abstract state-feedback controller on the abstraction. In this context, our contribution is two-fold: (I) we define a suitable relation between the original system and its abstraction which characterizes the …
An Experimental Comparison of Sliding Mode and Immersion and Invariance Adaptive Controllers forPosition-feedback Tracking of a Simple Mechanical System with Friction
Tecnológico Nacional de México / I.T. de La Laguna（墨西哥国立理工学院拉古纳技术学院）；Tecnológico Nacional de México / I.T. de La Paz（墨西哥国立理工学院拉巴斯技术学院）；Departamento de Ingeniería Eléctrica y Electrónica / I.T. Autónomo de México（墨西哥自治理工学院电气与电子工程系）
Research Institute of Trustworthy Autonomous Systems, SUSTech（南方科技大学可信自主系统研究院）；School of Automation, Hangzhou Dianzi University（杭州电子科技大学自动化学院）；Verimag, Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学Verimag实验室）
We design and experimentally evaluate a hybrid safe-by-construction collision avoidance controller for autonomous vehicles. The controller combines into a single architecture the respective advantages of an adaptive controller and a discrete safe controller. The adaptive controller relies on model predictive control to achieve optimal efficiency in nominal conditions. The safe controller avoids collision by applying two different policies, for nominal and out-of-nominal conditions, respectively. We present design principles for both the adaptive and the safe controller and show how each one ca…
We present a novel class of nonlinear controllers that interpolates among differently behaving linear controllers as a case study for recently proposed Linear and Nonlinear System Level Synthesis framework. The structure of the nonlinear controller allows for simultaneously satisfying performance and safety objectives defined for small- and large-disturbance regimes. The proposed controller is distributed, handles delays, sparse actuation, and localizes disturbances. We show our nonlinear controller always outperforms its linear counterpart for constrained LQR problems. We further demonstrate …
Learning-based controllers, such as neural network (NN) controllers, can show high empirical performance but lack formal safety guarantees. To address this issue, control barrier functions (CBFs) have been applied as a safety filter to monitor and modify the outputs of learning-based controllers in order to guarantee the safety of the closed-loop system. However, such modification can be myopic with unpredictable long-term effects. In this work, we propose a safe-by-construction NN controller which employs differentiable CBF-based safety layers, and investigate the performance of safe-by-const…
University of Michigan（密歇根大学）；Plus.ai Inc.（智加科技公司）；California Institute of Technology（加州理工学院）；Visteon Corporation（伟世通公司）
Accomplishing safe and efficient driving is one of the predominant challenges in the controller design of connected automated vehicles (CAVs). It is often more convenient to address these goals separately and integrate the resulting controllers. In this study, we propose a controller integration scheme to fuse performance-based controllers and safety-oriented controllers safely for the longitudinal motion of a CAV. The resulting structure is compatible with a large class of controllers, and offers flexibility to design each controller individually without affecting the performance of the other…
In this paper we propose sufficient conditions to synthesizing reach-avoid controllers for deterministic systems modelled by ordinary differential equations and stochastic systems modeled by stochastic differential equations based on the notion of control guidance-barrier functions. We begin with considering deterministic systems. Given an open safe set, a target set and a nominal controller, we attempt to synthesize a reach-avoid controller, which modifies the nominal controller in a minimal way enforcing the reach-avoid objective, i.e., the system will enter the target set eventually while s…
In this paper, we provide a direct data-driven approach to synthesize safety controllers for unknown linear systems affected by unknown-but-bounded disturbances, in which identifying the unknown model is not required. First, we propose a notion of $γ$-robust safety invariant ($γ$-RSI) sets and their associated state-feedback controllers, which can be applied to enforce invariance properties. Then, we formulate a data-driven computation of these sets in terms of convex optimization problems with linear matrix inequalities (LMI) as constraints, which can be solved based on a finite number of dat…
Underactuated systems pose the challenge of being able to control a plant whose degrees of freedom are not necessarily directly linked to an actuator or where such a relationship is not straightforward. Rotary inverted pendulum is an example of such systems, which on its simplest representation consists of a pendulum whose vertical angle should be taken up to the upward unstable position based on the impulse given from another bar and an appropriate control strategy, bar that is controlled by an electrical motor. This problem is often tackled by linear control theory with state-feedback contro…
This paper proposes a data-driven state feedback controller that enables reference tracking for nonlinear discrete-time systems. The controller is designed based on the identified inverse model of the system and a given reference model, assuming that the identification of the inverse model is carried out using only the system's state/input measurements. When its results are provided, we present conditions that guarantee a certain level of reference tracking performance, regardless of the identification method employed for the inverse model. Specifically, when Gaussian process regression (GPR) …
Symbolic control is an abstraction-based controller synthesis approach that provides, algorithmically, certifiable-by-construction controllers for cyber-physical systems. Symbolic control approaches usually assume that full-state information is available which is not suitable for many real-world applications with partially-observable states or output information. This article introduces a framework for output-feedback symbolic control. We propose relations between original systems and their symbolic models based on outputs. They enable designing symbolic controllers and refining them to enforc…
Researchers have developed various cascaded controllers and non-cascaded controllers for the navigation and control of quadrotors in recent years. It is vital to ensure the safety of a quadrotor both in normal state and in abnormal state if a controller tends to make the quadrotor unsafe. To this end, this paper proposes a non-cascaded Control Barrier Function (CBF) for a quadrotor controlled by either cascaded controllers or a non-cascaded controller. Incorporated with a Quadratic Programming (QP), the non-cascaded CBF can simultaneously regulate the magnitude of the total thrust and the torq…
Fast and safe voltage regulation algorithms can serve as fundamental schemes for achieving a high level of renewable penetration in the modern distribution power grids. Faced with uncertain or even unknown distribution grid models and fast-changing power injections, model-free deep reinforcement learning (DRL) algorithms have been proposed to find the reactive power injections for inverters while optimizing the voltage profiles. However, such data-driven controllers can not guarantee satisfaction of the hard operational constraints, such as maintaining voltage profiles within a certain range o…
The well-known quote from George Box states that: "All models are wrong, but some are useful." To develop more useful models, we quantify the inaccuracy with which a given model represents a system of interest, so that we may leverage this quantity to facilitate controller synthesis and verification. Specifically, we develop a procedure that identifies a sim2real gap that holds with a minimum probability. Augmenting the nominal model with our identified sim2real gap produces an uncertain model which we prove is an accurate representor of system behavior. We leverage this uncertain model to syn…
This paper addresses the problem of designing an optimal output feedback controller with a specified controller structure for linear time-invariant (LTI) systems to maximize the passivity level for the closed-loop system, in both continuous-time (CT) and discrete-time (DT). Specifically, the set of controllers under consideration is linearly parameterized with constrained parameters. Both input feedforward passivity (IFP) and output feedback passivity (OFP) indices are used to capture the level of passivity. Given a set of stabilizing controllers, a necessary and sufficient condition is propos…
