We present differentiable predictive control (DPC) as a deep learning-based alternative to the explicit model predictive control (MPC) for unknown nonlinear systems. In the DPC framework, a neural state-space model is learned from time-series measurements of the system dynamics. The neural control policy is then optimized via stochastic gradient descent approach by differentiating the MPC loss function through the closed-loop system dynamics model. The proposed DPC method learns model-based control policies with state and input constraints, while supporting time-varying references and constraints. In embedded implementation using a Raspberry-Pi platform, we experimentally demonstrate that it is possible to train constrained control policies purely based on the measurements of the unknown nonlinear system. We compare the control performance of the DPC method against explicit MPC and report efficiency gains in online computational demands, memory requirements, policy complexity, and construction time. In particular, we show that our method scales linearly compared to exponential scalability of the explicit MPC solved via multiparametric programming.
The aim of this paper is to propose a new data-driven control scheme for multi-input-multi-output linear time-invariant systems whose system model are completely unknown. Using a non-minimal input-output realization, the proposed method can be applied to the case where the system order is unknown, provided that its upper bound is known. A workaround against measurement noise is proposed and it is shown through simulation study that the proposed method is superior to the conventional methods when dealing with input/output data corrupted by measurement noise.
We propose a hierarchical learning architecture for predictive control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. A parameterized environment model generates state constraints specific to each task, which are satisfied by the stored trajectories. Our goal is to find a feasible trajectory for a new task in an unknown environment. From stored data, we learn strategies in the form of target sets in a reduced-order state space. These strategies are applied to …
This paper investigates the data-driven predictive control problems for a class of continuous-time industrial processes with completely unknown dynamics. The proposed approach employs the data-driven technique to get the system matrices online, using input-output measurements. Then, a model-free predictive control approach is designed to implement the receding-horizon optimization and realize the reference tracking. Feasibility of the proposed algorithm and stability of the closed-loop control systems are analyzed, respectively. Finally, a simulation example is provided to demonstrate the effe…
We present a solution for modeling and online identification for heating, ventilation, and air conditioning (HVAC) control in buildings. Our approach comprises: (a) a resistance-capacitance (RC) model based on first order energy balance for deriving the zone temperature dynamics, and (b) a neural network for modeling HVAC dynamics. State estimation and model identification are simultaneously performed using nonlinear moving horizon estimation (MHE) with physical constraints for system states. We leverage the identified model in model predictive control (MPC) for occupant comfort satisfaction a…
In this paper, the model predictive control is designed for an interval type-2 Takagi-Sugeno (T-S) system with unknown time-varying delay in state and input vectors. The time-varying delay is a weird phenomenon that is appeared in almost all systems. It can make many problems and instability while the system is working. In this paper, the time-varying delay is considered in both states and input vectors and is the sensible difference between the proposed method here and previous algorithms, besides, it is unknown but bounded. To solve the problem, the Razumikhin approach is applied to the prop…
Algorithms for motion planning in unknown environments are generally limited in their ability to reason about the structure of the unobserved environment. As such, current methods generally navigate unknown environments by relying on heuristic methods to choose intermediate objectives along frontiers. We present a unified method that combines map prediction and motion planning for safe, time-efficient autonomous navigation of unknown environments by dynamically-constrained robots. We propose a data-driven method for predicting the map of the unobserved environment, using the robot's observatio…
Gaussian Process Model Predictive Control of Unknown Nonlinear Systems
School of Engineering and Advanced Technology, Massey University（梅西大学工程与先进技术学院）；Department of Information Technology and Software Engineering, Auckland University of Technology（奥克兰理工大学信息技术与软件工程系）
Velocity-Form Data-Enabled Predictive Control of Soft Robots under Unknown External Payloads
Department of Mechanical Engineering, Michigan State University（机械工程系，密歇根州立大学）；Department of Mechanical Engineering, University of California Riverside（机械工程系，加州大学河滨分校）；Department of Electrical and Computer Engineering, Michigan State University（电气与计算机工程系，密歇根州立大学）；Department of Electrical and Computer Engineering, University of Michigan（电气与计算机工程系，密歇根大学）
In this work, we propose a distributionally robust stochastic model predictive control (DR-SMPC) algorithm to address the problem of two-sided chance constrained discrete-time linear system corrupted by additive noise. The prevalent mechanism to cope with two-sided chance constraints is the so-called risk allocation approach, which conservatively approximates the two-sided chance constraints with two single chance constraints by applying the Boole's inequality. In this proposed DR-SMPC framework, an exact tractable second-order cone (SOC) approach is adopted to abstract the two-sided chance co…
This article proposes a hierarchical learning architecture for safe data-driven control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. In addition to task-invariant system state and input constraints, a parameterized environment model generates task-specific state constraints, which are satisfied by the stored trajectories. Our goal is to use these trajectories to find a safe and high-performing policy for a new task in a new, unknown environment. We propose u…
We provide a comprehensive review and practical implementation of a recently developed model predictive control (MPC) framework for controlling unknown systems using only measured data and no explicit model knowledge. Our approach relies on an implicit system parametrization from behavioral systems theory based on one measured input-output trajectory. The presented MPC schemes guarantee closed-loop stability for unknown linear time-invariant (LTI) systems, even if the data are affected by noise. Further, we extend this MPC framework to control unknown nonlinear systems by continuously updating…
Distributed switched large-scale systems are composed by dynamically coupled subsystems, in which interactions among subsystems vary over time according to an exogenous input signal named switching signal. In this paper, we present a distributed robust switched model-based predictive control (DSwMPC) to control such systems that are subject to local state and input constraints. The proposed method guarantees stabilizing the origin of the whole closed-loop system and ensures the constraints satisfaction in the presence of an unknown switching signal. In the distributed model-based predictive co…
This paper proposes a hybrid Gaussian process (GP) approach to robust economic model predictive control under unknown future disturbances in order to reduce the conservatism of the controller. The proposed hybrid GP is a combination of two well-known methods, namely, kernel composition and nonlinear auto-regressive. A switching mechanism is employed to select one of these methods for disturbance prediction after analyzing the prediction outcomes. The hybrid GP is intended to detect not only patterns but also unexpected behaviors in the unknown disturbances by using past disturbance measurement…
Reinforcement Learning Compensated Model Predictive Control for Off-road Driving on Unknown Deformable Terrain
Department of Automotive Engineering, Clemson University（汽车工程系，克莱姆森大学）；Ground Vehicle Systems Center, U.S. Army Combat Capabilities Development Command（地面车辆系统中心，美国陆军作战能力发展指挥部）
This article proposes a new safety concept: backup plan safety. The backup plan safety is defined as the ability to complete one of the alternative missions in the case of primary mission abortion. To incorporate this new safety concept in control problems, we formulate a feasibility maximization problem that adopts additional (virtual) input horizons toward the alternative missions on top of the input horizon toward the primary mission. Cost functions for the primary and alternative missions construct multiple objectives, and multi-horizon inputs evaluate them. To address the feasibility maxi…
Robust adaptive model predictive control (RAMPC) is a novel control method that combines robustness guarantees with respect to unknown parameters and bounded disturbances into a model predictive control scheme. However, RAMPC has so far only been developed in theory. The goal of this paper is to apply RAMPC to a physical quadrotor experiment. To the best of our knowledge this is the first time that RAMPC has been applied in practice using a state space formulation. In doing so, we highlight important practical challenges such as computation of $λ$-contractive polytopes and dealing with measure…
This work presents DMPC (Data-and Model-Driven Predictive Control) to solve control problems in which some of the constraints or parts of the objective function are known, while others are entirely unknown to the controller. It is assumed that there is an exogenous ``black box'' system, e.g. a machine learning technique, that predicts the value of the unknown functions for a given trajectory. DMPC (1) provides an approach to merge both the model-based and black-box systems; (2) can cope with very little data and is sample efficient, building its solutions based on recently generated trajectori…
University of California at Berkeley（加州大学伯克利分校）；California Institute of Technology（加州理工学院）
We present a Model Predictive Control (MPC) strategy for unknown input-affine nonlinear dynamical systems. A non-parametric method is used to estimate the nonlinear dynamics from observed data. The estimated nonlinear dynamics are then linearized over time varying regions of the state space to construct an Affine Time Varying (ATV) model. Error bounds arising from the estimation and linearization procedure are computed by using sampling techniques. The ATV model and the uncertainty sets are used to design a robust Model Predictive Control (MPC) problem which guarantees safety for the unknown s…
An important issue in model-based control design is that an accurate dynamic model of the system is generally nonlinear, complex, and costly to obtain. This limits achievable control performance in practice. Gaussian process (GP) based estimation of system models is an effective tool to learn unknown dynamics directly from input/output data. However, conventional GP-based control methods often ignore the computational cost associated with accumulating data during the operation of the system and how to handle forgetting in continuous adaption. In this paper, we present a novel Dual Gaussian Pro…
Data-driven predictive control (DPC) is a feedback control method for systems with unknown dynamics. It repeatedly optimizes a system's future trajectories based on past input-output data. We develop a numerical method that computes poisoning attacks that inject additive perturbations to the online output data to change the trajectories optimized by DPC. This method is based on implicitly differentiating the solution map of the trajectory optimization in DPC. We demonstrate that the resulting attacks can cause an output tracking error one order of magnitude higher than random perturbations in …
An important issue in quadcopter control is that an accurate dynamic model of the system is nonlinear, complex, and costly to obtain. This limits achievable control performance in practice. Gaussian process (GP) based estimation is an effective tool to learn unknown dynamics from input/output data. However, conventional GP-based control methods often ignore the computational cost associated with accumulating data during the operation of the system and how to handle forgetting in continuous adaption. In this paper, we present a novel Dual Gaussian Process (DGP) based model predictive control st…
We present a novel tube-based data-driven predictive control method for linear systems affected by a bounded addictive disturbance. Our method leverages recent results in the reachability analysis of unknown linear systems to formulate and solve a robust tube-based predictive control problem. More precisely, our approach consists in deriving, from the collected data, a zonotope that includes the true state error set. We show how to guarantee the stability of the resulting error zonotope, which can be exploited to increase the computational efficiency of existing zonotopic data-driven MPC formu…
This paper considers the control of uncertain systems that are operated under limited resource factors, such as battery life or hardware longevity. We consider here resource-aware self-triggered control techniques that schedule system operation non-uniformly in time in order to balance performance against resource consumption. When running in an uncertain environment, unknown disturbances may deteriorate system performance by acting adversarially against the planned event triggering schedule. In this work, we propose a resource-aware stochastic predictive control scheme to tackle this challeng…
Dual control explicitly addresses the problem of trading off active exploration and exploitation in the optimal control of partially unknown systems. While the problem can be cast in the framework of stochastic dynamic programming, exact solutions are only tractable for discrete state and action spaces of very small dimension due to a series of nested minimization and expectation operations. We propose an approximate dual control method for systems with continuous state and input domain based on a rollout dynamic programming approach, splitting the control horizon into a dual and an exploitati…
We present a data-driven model predictive control scheme for chance-constrained Markovian switching systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are estimated which include the true conditional probability distributions with high probability. These sets are updated online and used to formulate a time-varying, risk-averse optimal control problem. We prove recursive feasibility of the resulting MPC scheme and show that the original chance constraints remain satisfied at every time step. Furthermore, we show…
Adaptive Legged Locomotion via Online Learning for Model Predictive Control
Department of Aerospace Engineering, University of Michigan（密歇根大学航空航天工程系）；Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）
We consider the problem of data-driven predictive control for an unknown discrete-time linear time-periodic (LTP) system of known period. Our proposed strategy generalizes both Data-enabled Predictive Control (DeePC) and Subspace Predictive Control (SPC), which are established data-driven control techniques for linear time-invariant (LTI) systems. The approach is supported by an extensive theoretical development of behavioral systems theory for LTP systems, culminating in a generalization of the fundamental lemma. Our algorithm produces results identical to standard Model Predictive Control (M…
National University of Defense Technology（国防科技大学）；Jilin University（吉林大学）
Robust model predictive control (MPC) is a well-known control technique for model-based control with constraints and uncertainties. In classic robust tube-based MPC approaches, an open-loop control sequence is computed via periodically solving an online nominal MPC problem, which requires prior model information and frequent access to onboard computational resources. In this paper, we propose an efficient robust MPC solution based on receding horizon reinforcement learning, called r-LPC, for unknown nonlinear systems with state constraints and disturbances. The proposed r-LPC utilizes a Koopma…
We present a data-driven nonlinear predictive control approach for the class of discrete-time multi-input multi-output feedback linearizable nonlinear systems. The scheme uses a non-parametric predictive model based only on input and noisy output data along with a set of basis functions that approximate the unknown nonlinearities. Despite the noisy output data as well as the mismatch caused by the use of basis functions, we show that the proposed multistep robust data-driven nonlinear predictive control scheme is recursively feasible and renders the closed-loop system practically exponentially…
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics for a class of discrete-time systems that are nominally linear with an additive nonlinear component. Such systems commonly model the nonlinear effects of an unknown environment on a nominal system. We optimize over a class of nonlinear feedback policies inspired by certainty equivalent "estimate-and-cancel" control laws pioneered in classical adaptive control to achieve significant performance improvements in the presence of uncertainties of large magnitude, a setting in…
We propose Kernel Predictive Control (KPC), a learning-based predictive control strategy that enjoys deterministic guarantees of safety. Noise-corrupted samples of the unknown system dynamics are used to learn several models through the formalism of non-parametric kernel regression. By treating each prediction step individually, we dispense with the need of propagating sets through highly non-linear maps, a procedure that often involves multiple conservative approximation steps. Finite-sample error bounds are then used to enforce state-feasibility by employing an efficient robust formulation. …
In this paper, it is shown that a simple formulation of Economic Model Predictive Control can be used which possesses two features that are generally viewed as mutually exclusive, namely, a rather short prediction horizon (reachability-compatible) on one side, and the absence of final constraint on the other side. The result holds for discrete-time models that are originated from continuous-time models by means of discretization. Practical stability at a small neighborhood of the optimal unknown steady-state pair is shown when some parameters increase and sampling time decreases. A commonly us…
