This paper proposes a differentiable linear quadratic Model Predictive Control (MPC) framework for safe imitation learning. The infinite-horizon cost is enforced using a terminal cost function obtained from the discrete-time algebraic Riccati equation (DARE), so that the learned controller can be proven to be stabilizing in closed-loop. A central contribution is the derivation of the analytical derivative of the solution of the DARE, thereby allowing the use of differentiation-based learning methods. A further contribution is the structure of the MPC optimization problem: an augmented Lagrangian method ensures that the MPC optimization is feasible throughout training whilst enforcing hard constraints on state and input, and a pre-stabilizing controller ensures that the MPC solution and derivatives are accurate at each iteration. The learning capabilities of the framework are demonstrated in a set of numerical studies.
We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from the observed system's dynamics, and the neural control law is optimized offline by leveraging the differentiable closed-loop system model. The combination of a differentiable closed-loop syst…
Designing control inputs for a system that involves dynamical responses in multiple timescales is nontrivial. This paper proposes a parameterized time-warping function to enable a non-uniformly sampling along a prediction horizon given some parameters. The horizon should capture the responses under faster dynamics in the near future and preview the impact from slower dynamics in the distant future. Then a variable sampling MPC (VS-MPC) strategy is proposed to jointly determine optimal control and sampling parameters at each timestamp. VS-MPC adapts how it samples along the horizon and determin…
The use of derivative-based solvers to compute solutions to optimal control problems with non-differentiable cost or dynamics often requires reformulations or relaxations that complicate the implementation or increase computational complexity. We present an initial framework for using the derivative-free Mesh Adaptive Direct Search (MADS) algorithm to solve Nonlinear Model Predictive Control problems with non-differentiable features without the need for reformulation. The MADS algorithm performs a structured search of the input space by simulating selected system trajectories and computing the…
Predictive control is frequently used for control problems involving constraints. Being an optimization based technique utilizing a user specified so-called stage cost, performance properties, i.e., bounds on the infinite horizon accumulated stage cost, aside closed-loop stability are of interest. To achieve good performance and to influence the region of attraction associated with the prediction horizon, the terminal cost of the predictive controller's optimization objective is a key design factor. Approximate dynamic programming refers to one particular approximation paradigm that pursues it…
First-order optimization solvers, such as the Fast Gradient Method, are increasingly being used to solve Model Predictive Control problems in resource-constrained environments. Unfortunately, the convergence rate of these solvers is significantly affected by the conditioning of the problem data, with ill-conditioned problems requiring a large number of iterations. To reduce the number of iterations required, we present a simple method for computing a horizon-independent preconditioning matrix for the Hessian of the condensed problem. The preconditioner is based on the block Toeplitz structure …
Stability under model predictive control (MPC) schemes is frequently ensured by terminal ingredients. Employing a (control) Lyapunov function as the terminal cost constitutes a common choice. Learning-based methods may be used to construct the terminal cost by relating it to, for instance, an infinite-horizon optimal control problem in which the optimal cost is a Lyapunov function. Value iteration, an approximate dynamic programming (ADP) approach, refers to one particular cost approximation technique. In this work, we merge the results of terminally unconstrained predictive control and approx…
Many chemical processes exhibit diverse timescale dynamics with a strong coupling between timescale sensitive variables. Model predictive control with a non-uniformly spaced optimisation horizon is an effective approach to multi-timescale control and offers opportunities for reduced computational complexity. In such an approach the fast, moderate and slow dynamics can be included in the optimisation problem by implementing smaller time intervals earlier in the prediction horizon and increasingly larger intervals towards the end of the prediction. In this paper, a reference-flexible condition i…
This paper proposes an algorithm that combines Fast Moving Horizon Parameter Estimation and Model Predictive Control subject to an observability constraint designed to ensure a lower bound on the performance of the parameter estimator. Output-feedback stability is proved through input-to-state stability of the state/error system under a small noise and initial error assumption. Numerical experiments have been carried out in the case of Active Simultaneous Localisation and Mapping (SLAM).
In this paper, model predictive control (MPC) strategies are proposed for dead-beat control of linear systems with and without state and control constraints. In unconstrained MPC, deadbeat performance can be guaranteed by setting the control horizon to the system dimension, and adding an terminal equality constraint. It is proved that the unconstrained deadbeat MPC is equivalent to linear deadbeat control. The proposed constrained deadbeat MPC is designed by setting the control horizon equal to the system dimension and penalizing only the terminal cost. The recursive feasibility and deadbeat p…
Adaptive Horizon Model Predictive Control (AHMPC) is a scheme for varying as needed the horizon length of Model Predictive Control (MPC). Its goal is to achieve stabilization with horizons as small as possible so that MPC can be used on faster or more complicated dynamic processes. Beside the standard requirements of MPC including a terminal cost that is a control Lyapunov function, AHMPC requires a terminal feedback that turns the control Lyapunov function into a standard Lyapunov function in some domain around the operating point. But this domain need not be known explicitly. MPC does not co…
Improving greenhouse fruit-production control by integrating reinforcement learning into short-horizon model predictive control
Wageningen University & Research（瓦赫宁根大学与研究）；Delft Center for Systems and Control (DCSC), Delft University of Technology（代尔夫特理工大学系统与控制中心）；Wageningen Research（瓦赫宁根研究）
We develop a one-Newton-step-per-horizon, online, lag-$L$, model predictive control (MPC) algorithm for solving discrete-time, equality-constrained, nonlinear dynamic programs. Based on recent sensitivity analysis results for the target problems class, we prove that the approach exhibits a behavior that we call superconvergence; that is, the tracking error with respect to the full horizon solution is not only stable for successive horizon shifts, but also decreases with increasing shift order to a minimum value that decays exponentially in the length of the receding horizon. The key analytical…
A standard way of finding a feedback law that stabilizes a control system to an operating point is to recast the problem as an infinite horizon optimal control problem. If the optimal cost and the optmal feedback can be found on a large domain around the operating point then a Lyapunov argument can be used to verify the asymptotic stability of the closed loop dynamics. The problem with this approach is that is usually very difficult to find the optimal cost and the optmal feedback on a large domain for nonlinear problems with or without constraints. Hence the increasing interest in Model Predi…
This paper investigates the problem of energy-optimal control for autonomous underwater vehicles (AUVs). To improve the endurance of AUVs, we propose a novel energy-optimal control scheme based on the economic model predictive control (MPC) framework. We first formulate a cost function that computes the energy spent for vehicle operation over a finite-time prediction horizon. Then, to account for the energy consumption beyond the prediction horizon, a terminal cost that approximates the energy to reach the goal (energy-to-go) is incorporated into the MPC cost function.
This letter considers model predictive control of a tandem-rotor helicopter. The error is formulated using the matrix Lie group $SE_2(3)$. A reference trajectory to a target is calculated using a quartic guidance law, leveraging the differentially flat properties of the system, and refined using a finite-horizon linear quadratic regulator. The nonlinear system is linearized about the reference trajectory enabling the formulation of a quadratic program with control input, attitude keep-in zone, and attitude error constraints. A non-uniformly spaced prediction horizon is leveraged to capture the…
The stability analysis of model predictive control schemes without terminal constraints and/or costs has attracted considerable attention during the last years. We pursue a recently proposed approach which can be used to determine a suitable optimization horizon length for nonlinear control systems governed by ordinary differential equations. In this context, we firstly show how the essential growth assumption involved in this methodology can be derived and demonstrate this technique by means of a numerical example. Secondly, inspired by corresponding results, we develop an algorithm which all…
This work presents a suboptimality study of a particular model predictive control with a stage cost shaping based on the ideas of reinforcement learning. The focus of the suboptimality study is to derive quantities relating the infinite-horizon cost function under the said variant of model predictive control to the respective infinite-horizon value function. The basis control scheme involves usual stabilizing constraints comprising of a terminal set and a terminal cost in the form of a local Lyapunov function. The stage cost is adapted using the principles of Q-learning, a particular approach …
The core of the Model Predictive Control (MPC) method in every step of the algorithm consists in solving a time-dependent optimization problem on the prediction horizon of the MPC algorithm, and then to apply a portion of the optimal control over the application horizon to obtain the new state. To solve this problem efficiently, we propose a time-adaptive residual a-posteriori error control concept based on the optimality system of this optimal control problem. This approach not only delivers a tailored time discretization of the the prediction horizon, but also suggests a tailored length of t…
The local stability and convergence for Model Predictive Control (MPC) of unconstrained nonlinear dynamics based on a linear time-invariant plant model is studied. Based on the long-time behavior of the solution of the Riccati Differential Equation (RDE), explicit error estimates are derived that clearly demonstrate the influence of the two critical parameters in MPC: the prediction horizon $T$ and the control horizon $τ$. In particular, if the MPC-controller has access to an exact (linear) plant model, the MPC-controls and the corresponding optimal state trajectories converge exponentially to…
Mixed integer predictive control deals with optimizing integer and real control variables over a receding horizon. The mixed integer nature of controls might be a cause of intractability for instances of larger dimensions. To tackle this little issue, we propose a decomposition method which turns the original $n$-dimensional problem into $n$ indipendent scalar problems of lot sizing form. Each scalar problem is then reformulated as a shortest path one and solved through linear programming over a receding horizon. This last reformulation step mirrors a standard procedure in mixed integer progra…
Developing safe automated vehicles that can be proactive, safe, and comfortable in mixed traffic requires improved planning methods that are risk-averse and that account for predictions of the motion of other road users. To consider these criteria, in this article, we propose a non-linear model-predictive trajectory generator scheme, which couples the longitudinal and lateral motion of the vehicle to steer the vehicle with minimal risk, while progressing towards the goal state. The proposed method takes into account the infrastructure, surrounding objects, and predictions of the objects' state…
Common approaches for direct model predictive control (MPC) for current reference tracking in power electronics suffer from the high computational complexity encountered when solving integer optimal control problems over long prediction horizons. We propose an efficient alternative method based on approximate dynamic programming, greatly reducing the computational burden and enabling sampling times below 25 $μ$s. Our approach is based on the offline estimation of an infinite horizon value function which is then utilized as the tail cost of an MPC problem. This allows us to reduce the controlle…
Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving this problem leads to an optimal probing nature of the resulting control law, called dual control, which trades off benefits of exploration and exploitation. In practice, intractability of Stochastic Model Predictive Control is typically overcome by replacement of the underlying Stochastic Optimal Control problem by more amenable approximate surrogate proble…
Carnegie Mellon University（卡内基梅隆大学）；Georgia Tech（佐治亚理工学院）；Bosch Center for AI（博世人工智能中心）
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by using the KKT conditions of the convex approximation at a fixed point of the controller. Using this strategy, we are able to learn the cost and dynamics of a controller via end-to-end learning. Our experiments focus on imitation learning in the pendulum and cartpole domains, where we…
This paper is concerned with the problem of Model Predictive Control and Rolling Horizon Control of discrete-time systems subject to possibly unbounded random noise inputs, while satisfying hard bounds on the control inputs. We use a nonlinear feedback policy with respect to noise measurements and show that the resulting mathematical program has a tractable convex solution in both cases. Moreover, under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state, under the resulting Model Predictive Control and Rolling Horizon Contr…
