We propose Kernel Predictive Control (KPC), a learning-based predictive control strategy that enjoys deterministic guarantees of safety. Noise-corrupted samples of the unknown system dynamics are used to learn several models through the formalism of non-parametric kernel regression. By treating each prediction step individually, we dispense with the need of propagating sets through highly non-linear maps, a procedure that often involves multiple conservative approximation steps. Finite-sample error bounds are then used to enforce state-feasibility by employing an efficient robust formulation. We then present a relaxation strategy that exploits on-line data to weaken the optimization problem constraints while preserving safety. Two numerical examples are provided to illustrate the applicability of the proposed control method.
Smooth Sampling-Based Model Predictive Control Using Deterministic Samples
Intelligent Sensor-Actuator-Systems Laboratory (ISAS), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（智能传感器执行器系统实验室（ISAS）， anthropomatics 与机器人学研究所，卡尔斯鲁厄理工学院）；Autonomous Learning Robots (ALR), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（自主学习机器人（ALR），anthropomatics 与机器人学研究所，卡尔斯鲁厄理工学院）
Data-enabled predictive control (DeePC) is a recently proposed approach that combines system identification, estimation and control in a single optimization problem, for which only recorded input/output data of the examined system is required. The same premise holds for the subspace predictive control (SPC) method in which a multi-step prediction model is identified from the same data as required for DeePC. This model is then used to formulate a similar optimal control problem. In this work we investigate the relationship between DeePC and SPC. Our primary contribution is to show that SPC is e…
In this work, an evaluation of Chance-Constrained Model Predictive Control (CC-MPC) in sewer systems over the use of the classical deterministic Model Predictive Control (MPC) is presented. The focus of this evaluation is on the avoidance of weir overflow when uncertainty is present. Furthermore, the design formulation of CC-MPC is presented with a comparison to the design of MPC. For the evaluation, a simplified model of the Barcelona sewer network case study is utilized. Our comparison shows that for sewer systems with uncertain inflows, a CC-MPC allows for better statistical guarantees for …
A powerful result from behavioral systems theory known as the fundamental lemma allows for predictive control akin to Model Predictive Control (MPC) for linear time invariant (LTI) systems with unknown dynamics purely from data. While most of data-driven predictive control literature focuses on robustness with respect to measurement noise, only few works consider exploiting probabilistic information of disturbances for performance-oriented control as in stochastic MPC. In this work, we propose a novel data-driven stochastic predictive control scheme for chance-constrained LTI systems subject t…
Willems' fundamental lemma and system level synthesis both characterize a linear dynamic system by its input/output sequences. In this work, we extend the application of the fundamental lemma from deterministic to uncertain LTI systems and then further prove this extension to be equivalent to system level synthesis. Based on this uncertain extension, a robust closed-loop data-enabled predictive control scheme is proposed, where a causal feedback control law is further derived. Two numerical experiments, including the temperature control of a single-zone building, are carried out to validate th…
Sampling-based Model Predictive Control Using Trust Regions
Intelligent Sensor-Actuator-Systems Laboratory (ISAS), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（智能传感器执行器系统实验室，人机学与机器人研究所，卡尔斯鲁厄理工学院）
We consider the problem of data-driven predictive control for an unknown discrete-time linear time-periodic (LTP) system of known period. Our proposed strategy generalizes both Data-enabled Predictive Control (DeePC) and Subspace Predictive Control (SPC), which are established data-driven control techniques for linear time-invariant (LTI) systems. The approach is supported by an extensive theoretical development of behavioral systems theory for LTP systems, culminating in a generalization of the fundamental lemma. Our algorithm produces results identical to standard Model Predictive Control (M…
We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.
This paper presents two stochastic model predictive control methods for linear time-invariant systems subject to unbounded additive uncertainties. The new methods are developed by formulating the chance constraints into deterministic form, which are treated in analogy with robust constraints, by using the probabilistic reachable set. Firstly, the probabilistically resolvable time-varying tube-based stochastic model predictive control algorithm is designed by employing the time-varying probabilistic reachable sets as the tubes. Secondly, by utilizing the probabilistic positively invariant set, …
Many systems such as autonomous vehicles and quadrotors are subject to parametric uncertainties and external disturbances. These uncertainties can lead to undesired performance degradation and safety issues. Therefore, it is important to design robust control strategies to safely regulate the dynamics of a system. This paper presents a novel framework for chance-constrained stochastic model predictive control of dynamic systems with non-Gaussian correlated probabilistic uncertainties. We develop a new stochastic Galerkin method to propagate the uncertainties using a new type of basis functions…
This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finite-horizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model. This can be achieved by selecting a proper stage cost and terminal cost for the OCP. A very useful particular case of OCP is a Model Predictive Control (MPC) scheme where a deterministic (possibly nonlinear) model is used to reduce the computational complexity. This observation leads us to parameterize an MPC scheme fully, including the cost function. In pract…
State-of-the-art Model Predictive Control (MPC) applications for building heating adopt either a deterministic controller together with a nonlinear model or a linearized model with a stochastic MPC controller. However, deterministic MPC only considers one single realization of the disturbances and its performance strongly depends on the quality of the forecast of the disturbances, which can lead to low performance. In fact, inadequate building energy management can lead to high energy costs and CO$_2$ emissions. On the other hand, a linearized model can fail to capture some dynamics and behavi…
Many systems exhibit a mixture of continuous and discrete dynamics. We consider a family of mixed-integer non-convex non-linear optimisation problems obtained in discretisations of optimal control of such systems. For this family, a branch-and-bound algorithm solves the discretised problem to global optimality. As an example, we consider water systems, where variations in flow and variations in water levels are continuous, while decisions related to fixed-speed pumps and whether gates that may be opened and closed are discrete. We show that the related optimal-control problems come from the fa…
In this work, a revised formulation of Chance-Constrained (CC) Model Predictive Control (MPC) is presented. The focus of this work is on the mathematical formulation of the revised CC-MPC, and the reason behind the need for its revision. The revised formulation is given in the context of sewer systems, and their weir overflow structures. A linear sewer model of the Astlingen Benchmark sewer model is utilized to illustrate the application of the formulation, both mathematically and performance-wise through simulations. Based on the simulations, a comparison of performance is done between the re…
Approximate non-linear model predictive control with safety-augmented neural networks
Institute for Data Science in Mechanical Engineering, RWTH Aachen University（机械工程数据科学研究所，亚琛RWTH大学）；Institute for Dynamic Systems and Control, ETH Zürich（动态系统与控制研究所，苏黎世联邦理工学院）
We present a general framework for risk semantics on Signal Temporal Logic (STL) specifications for stochastic dynamical systems using axiomatic risk theory. We show that under our recursive risk semantics, risk constraints on STL formulas can be expressed in terms of risk constraints on atomic predicates. We then show how this allows a (stochastic) STL risk constraint to be transformed into a risk-tightened deterministic STL constraint on a related deterministic nominal system, enabling the application of existing STL methods. For affine predicate functions and a (coherent) Distributionally R…
Stochastic Model Predictive Control has proved to be an efficient method to plan trajectories in uncertain environments, e.g., for autonomous vehicles. Chance constraints ensure that the probability of collision is bounded by a predefined risk parameter. However, considering chance constraints in an optimization problem can be challenging and computationally demanding. In this paper, we present a grid-based Stochastic Model Predictive Control approach. This approach allows to determine a simple deterministic reformulation of the chance constraints and reduces the computational effort, while co…
Human-leading truck platooning systems have been proposed to leverage the benefits of both human supervision and vehicle autonomy. Equipped with human guidance and autonomous technology, human-leading truck platooning systems are more versatile to handle uncertain traffic conditions than fully automated platooning systems. This paper presents a novel distributed stochastic model predictive control (DSMPC) design for a human-leading heavy-duty truck platoon. The proposed DSMPC design integrates the stochastic driver behavior model of the human-driven leader truck with a distributed formation co…
In this technical note we analyse the performance improvement and optimality properties of the Learning Model Predictive Control (LMPC) strategy for linear deterministic systems. The LMPC framework is a policy iteration scheme where closed-loop trajectories are used to update the control policy for the next execution of the control task. We show that, when a Linear Independence Constraint Qualification (LICQ) condition holds, the LMPC scheme guarantees strict iterative performance improvement and optimality, meaning that the closed-loop cost evaluated over the entire task converges asymptotica…
This study compares Deep Reinforcement Learning (DRL) and Model Predictive Control (MPC) for Adaptive Cruise Control (ACC) design in car-following scenarios. A first-order system is used as the Control-Oriented Model (COM) to approximate the acceleration command dynamics of a vehicle. Based on the equations of the control system and the multi-objective cost function, we train a DRL policy using Deep Deterministic Policy Gradient (DDPG) and solve the MPC problem via Interior-Point Optimization (IPO). Simulation results for the episode costs show that, when there are no modeling errors and the t…
Control of nonlinear uncertain systems is a common challenge in the robotics field. Nonlinear latent force models, which incorporate latent uncertainty characterized as Gaussian processes, carry the promise of representing such systems effectively, and we focus on the control design for them in this work. To enable the design, we adopt the state-space representation of a Gaussian process to recast the nonlinear latent force model and thus build the ability to predict the future state and uncertainty concurrently. Using this feature, a stochastic model predictive control problem is formulated. …
Gaussian Process Model Predictive Control of Unknown Nonlinear Systems
School of Engineering and Advanced Technology, Massey University（梅西大学工程与先进技术学院）；Department of Information Technology and Software Engineering, Auckland University of Technology（奥克兰理工大学信息技术与软件工程系）
University of Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立理工学院）；GIPSA-lab（格勒诺布尔自动控制与信号处理实验室）
In this paper, a simple heuristic is proposed for the design of uncertainty aware predictive controllers for nonlinear models involving uncertain parameters. The method relies on Machine Learning-based approximation of ideal deterministic MPC solutions with perfectly known parameters. An efficient construction of the learning data set from these off-line solutions is proposed in which each solution provides many samples in the learning data. This enables a drastic reduction of the required number of Non Linear Programming problems to be solved off-line while explicitly exploiting the statistic…
Beijing Jiaotong Univeristy（北京交通大学）；Beijing Institude of Petrochemical Technology（北京石油化工学院）
In this paper, two novel practical methods of Reinforcement Learning from Demonstration (RLfD) are developed and applied to automatic berthing control systems for Unmanned Surface Vessel. A new expert data generation method, called Model Predictive Based Expert (MPBE) which combines Model Predictive Control and Deep Deterministic Policy Gradient, is developed to provide high quality supervision data for RLfD algorithms. A straightforward RLfD method, model predictive Deep Deterministic Policy Gradient (MP-DDPG), is firstly introduced by replacing the RL agent with MPBE to directly interact wit…
The problem of synthesizing stochastic explicit model predictive control policies is known to be quickly intractable even for systems of modest complexity when using classical control-theoretic methods. To address this challenge, we present a scalable alternative called stochastic parametric differentiable predictive control (SP-DPC) for unsupervised learning of neural control policies governing stochastic linear systems subject to nonlinear chance constraints. SP-DPC is formulated as a deterministic approximation to the stochastic parametric constrained optimal control problem. This formulati…
Information-Theoretic Adaptive Cooling for Deterministic MPPI via Entropy Feedback
School of Automation & Intelligent Sensing, Shanghai Jiao Tong University（自动化与智能感知学院，上海交通大学）；Department of Physics, The Hong Kong University of Science and Technology（物理系，香港科技大学）
