Safety and optimality in learning-based control at low computational cost
Aalto University（阿alto大学）；Uppsala University（乌普萨拉大学）；Wrocław University of Science and Technology（沃斯克拉大学科学与技术大学）；KTH Royal Institute of Technology（皇家理工学院）；Eindhoven University of Technology（埃因霍温理工大学）
Dominik Baumann, Krzysztof Kowalczyk, Cristian R. Rojas, Koen Tiels, Pawel Wachel
备注：
Accepted final version to appear in the IEEE Transactions on Automatic Control
Applying machine learning methods to physical systems that are supposed to act in the real world requires providing safety guarantees. However, methods that include such guarantees often come at a high computational cost, making them inapplicable to large datasets and embedded devices with low computational power. In this paper, we propose CoLSafe, a computationally lightweight safe learning algorithm whose computational complexity grows sublinearly with the number of data points. We derive both safety and optimality guarantees and showcase the effectiveness of our algorithm on a seven-degrees-of-freedom robot arm.
For a broad class of nonlinear systems, we formulate the problem of guaranteeing safety with optimality under constraints. Specifically, we define controlled safety for differential inclusions with constraints on the states and the inputs. Through the use of nonsmooth analysis tools, we show that a continuous optimal control law can be selected from a set-valued constraint capturing the system constraints and conditions guaranteeing safety using control barrier functions. Our results guarantee optimality and safety via a continuous state-feedback law designed using nonsmooth control barrier fu…
Norwegian University of Technology, NTNU（挪威科技大学）；IMT School for Advanced Studies Lucca（卢卡IMT高等研究院）
For all its successes, Reinforcement Learning (RL) still struggles to deliver formal guarantees on the closed-loop behavior of the learned policy. Among other things, guaranteeing the safety of RL with respect to safety-critical systems is a very active research topic. Some recent contributions propose to rely on projections of the inputs delivered by the learned policy into a safe set, ensuring that the system safety is never jeopardized. Unfortunately, it is unclear whether this operation can be performed without disrupting the learning process. This paper addresses this issue. The problem i…
Leveraging recent developments in black-box risk-aware verification, we provide three algorithms that generate probabilistic guarantees on (1) optimality of solutions, (2) recursive feasibility, and (3) maximum controller runtimes for general nonlinear safety-critical finite-time optimal controllers. These methods forego the usual (perhaps) restrictive assumptions required for typical theoretical guarantees, e.g. terminal set calculation for recursive feasibility in Nonlinear Model Predictive Control, or convexification of optimal controllers to ensure optimality. Furthermore, we show that the…
Safe Learning Control with Optimality and Stability Guarantees
Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments, School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, Guangdong 518055, China（深圳先进 motion 控制与现代自动化装备重点实验室，智能科学与工程学院，哈尔滨工业大学，深圳，广东 518055，中国）；School of Data Science, Lingnan University, Hong Kong（数据科学学院，岭南大学，香港）；School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（电气与电子工程学院，南洋理工大学，新加坡）；MIT CSAIL
ETH Zürich（苏黎世联邦理工学院）；Massachusetts Institute of Technology（麻省理工学院）
In tunnel construction projects, delays induce high costs. Thus, tunnel boring machines (TBM) operators aim for fast advance rates, without safety compromise, a difficult mission in uncertain ground environments. Finding the optimal control parameters based on the TBM sensors' measurements remains an open research question with large practical relevance. In this paper, we propose an intelligent decision support system developed in three steps. First past projects performances are evaluated with an optimality score, taking into account the advance rate and the working pressure safety. Then, a d…
This work proposes an optimal safe controller minimizing an infinite horizon cost functional subject to control barrier functions (CBFs) safety conditions. The constrained optimal control problem is reformulated as a minimization problem of the Hamilton-Jacobi-Bellman (HJB) equation subjected to the safety constraints. By solving the optimization problem, we are able to construct a closed form solution that satisfies optimality and safety conditions. The proposed solution is shown to be continuous and thus it renders the safe set forward invariant while minimizing the given cost. Hence, optima…
This paper presents a scheme for dual robust control of batch processes under parametric uncertainty. The dual-control paradigm arises in the context of adaptive control. A trade-off should be decided between the control actions that (robustly) optimize the plant performance and between those that excite the plant such that unknown plant model parameters can be learned precisely enough to increase the robust performance of the plant. Some recently proposed approaches can be used to tackle this problem, however, this will be done at the price of conservativeness or significant computational bur…
Optimal Parameter Adaptation for Safety-Critical Control via Safe Barrier Bayesian Optimization
University of Electronic Science and Technology of China（电子科技大学）；University of Exeter（埃克塞特大学）；Australian AI Institute（澳大利亚人工智能研究所）；University of Technology Sydney（悉尼科技大学）；Hunan University（湖南大学）；China University of Mining and Technology（中国矿业大学）；Southeast University（东南大学）
CNRS（法国国家科学研究中心）；University of Rennes（雷恩大学）；Inria（法国国家信息与自动化研究所）；IRISA（法国信息与随机系统研究所）；University of Innsbruck（因斯布鲁克大学）
This paper presents an approach to deal with safety of dynamical systems in presence of multiple non-convex unsafe sets. While optimal control and model predictive control strategies can be employed in these scenarios, they suffer from high computational complexity in case of general nonlinear systems. Leveraging control barrier functions, on the other hand, results in computationally efficient control algorithms. Nevertheless, when safety guarantees have to be enforced alongside stability objectives, undesired asymptotically stable equilibrium points have been shown to arise. We propose a com…
Across many disciplines, chemical reaction networks (CRNs) are an established population model defined as a system of coupled nonlinear ordinary differential equations. In many applications, for example, in systems biology and epidemiology, CRN parameters such as the kinetic reaction rates can be used as control inputs to steer the system toward a given target. Unfortunately, the resulting optimal control problem is nonlinear, therefore, computationally very challenging. We address this issue by introducing an optimality-preserving reduction algorithm for CRNs. The algorithm partitions the ori…
This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finite-horizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model. This can be achieved by selecting a proper stage cost and terminal cost for the OCP. A very useful particular case of OCP is a Model Predictive Control (MPC) scheme where a deterministic (possibly nonlinear) model is used to reduce the computational complexity. This observation leads us to parameterize an MPC scheme fully, including the cost function. In pract…
This work presents a decentralized allocation algorithm of safety-critical application on parallel computing architectures, where individual Computational Units can be affected by faults. The described method consists in representing the architecture by an abstract graph where each node represents a Computational Unit. Applications are also represented by the graph of Computational Units they require for execution. The problem is then to decide how to allocate Computational Units to applications to guarantee execution of the safety-critical application. The problem is formulated as an optimiza…
We propose a computationally efficient approach to safe reinforcement learning (RL) for frequency regulation in power systems with high levels of variable renewable energy resources. The approach draws on set-theoretic control techniques to craft a neural network-based control policy that is guaranteed to satisfy safety-critical state constraints, without needing to solve a model predictive control or projection problem in real time. By exploiting the properties of robust controlled-invariant polytopes, we construct a novel, closed-form "safety-filter" that enables end-to-end safe learning usi…
Maximal Controlled Invariant-MPC: Enhancing Feasibility and Reducing Conservatism through Terminal CBF Constraint in Safety-Critical Control
Graduate Student, Daniel Guggenheim School of Aerospace Engineering（丹尼尔·古吉恩航空航天工程学院研究生）；Assistant Professor, Director, Aerospace Robotics Lab, Daniel Guggenheim School of Aerospace Engineering（丹尼尔·古吉恩航空航天工程学院助理教授、主任、航空航天机器人实验室）
This paper is concerned with optimality and stability analysis of a family of ensemble Kalman filter (EnKF) algorithms. EnKF is commonly used as an alternative to the Kalman filter for high-dimensional problems, where storing the covariance matrix is computationally expensive. The algorithm consists of an ensemble of interacting particles driven by a feedback control law. The control law is designed such that, in the linear Gaussian setting and asymptotic limit of infinitely many particles, the mean and covariance of the particles follow the exact mean and covariance of the Kalman filter. The …
The task assignment problem is fundamental in combinatorial optimisation, aiming at allocating one or more tasks to a number of agents while minimizing the total cost or maximizing the overall assignment benefit. This problem is known to be computationally hard since it is usually formulated as a mixed-integer programming problem. In this paper, we consider a novel time-triggered dimension reduction algorithm (TTDRA). We propose convexification approaches to convexify both the constraints and the cost function for the general non-convex assignment problem. The computational speed is accelerate…
