The closed-loop performance of model predictive controllers (MPCs) is sensitive to the choice of prediction models, controller formulation, and tuning parameters. However, prediction models are typically optimized for prediction accuracy instead of performance, and MPC tuning is typically done manually to satisfy (probabilistic) constraints. In this work, we demonstrate a general approach for automating the tuning of MPC under uncertainty. In particular, we formulate the automated tuning problem as a constrained black-box optimization problem that can be tackled with derivative-free optimization. We rely on a constrained variant of Bayesian optimization (BO) to solve the MPC tuning problem that can directly handle noisy and expensive-to-evaluate functions. The benefits of the proposed automated tuning approach are demonstrated on a benchmark continuously stirred tank reactor example.
This paper investigates the combination of model predictive control (MPC) concepts and posterior sampling techniques and proposes a simple constraint tightening technique to introduce cautiousness during explorative learning episodes. The provided theoretical analysis in terms of cumulative regret focuses on previously stated sufficient conditions of the resulting `Cautious Bayesian MPC' algorithm and shows Lipschitz continuity of the future reward function in the case of linear MPC problems. In the case of nonlinear MPC problems, it is shown that commonly required assumptions for nonlinear MP…
We present a Bayesian optimization (BO) framework for tuning model predictive controllers (MPC) of central heating, ventilation, and air conditioning (HVAC) plants. This approach treats the functional relationship between the closed-loop performance of MPC and its tuning parameters as a black-box. The approach is motivated by the observation that evaluating the closed-loop performance of MPC by trial-and-error is time-consuming (e.g., every closed-loop simulation can involve solving thousands of optimization problems). The proposed BO framework seeks to quickly identify the optimal tuning para…
Tight performance specifications in combination with operational constraints make model predictive control (MPC) the method of choice in various industries. As the performance of an MPC controller depends on a sufficiently accurate objective and prediction model of the process, a significant effort in the MPC design procedure is dedicated to modeling and identification. Driven by the increasing amount of available system data and advances in the field of machine learning, data-driven MPC techniques have been developed to facilitate the MPC controller design. While these methods are able to lev…
In this paper, we develop an optimal weight adaptation strategy of model predictive control (MPC) for connected and automated vehicles (CAVs) in mixed traffic. We model the interaction between a CAV and a human-driven vehicle (HDV) as a simultaneous game and formulate a game-theoretic MPC problem to find a Nash equilibrium of the game. In the MPC problem, the weights in the HDV's objective function can be learned online using moving horizon inverse reinforcement learning. Using Bayesian optimization, we propose a strategy to optimally adapt the weights in the CAV's objective function so that t…
In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model free methods, especially in simulated robotics tasks. Probabilistic ensembles with trajectory sampling (PETS) is a leading type of MBRL, which employs Bayesian inference to dynamics modeling and model predictive control (MPC) with stochastic optimization via the cross entropy method (CEM). In this paper, we propose a novel extension to the uncertainty-aware MBRL. Our main cont…
This paper tackles the multi-objective optimization of the cost functional of a path-following model predictive control for vehicle longitudinal and lateral control. While the inherent optimal character of the model predictive control and the direct consideration of constraints gives a very powerful tool for many applications, is the determination of an appropriate cost functional a non-trivial task. This results on the one hand from the number of degrees of freedom or the multitude of adjustable parameters and on the other hand from the coupling of these. To overcome this situation a Bayesian…
Intersection of Reinforcement Learning and Bayesian Optimization for Intelligent Control of Industrial Processes: A Safe MPC-based DPG using Multi-Objective BO
Department of Mechanical Engineering, Clemson University（机械工程系，克莱姆森大学）
Understanding the probabilistic traffic environment is a vital challenge for the motion planning of autonomous vehicles. To make feasible control decisions, forecasting future trajectories of adjacent cars is essential for intelligent vehicles to assess potential conflicts and react to reduce the risk. This paper first introduces a Bayesian Long Short-term Memory (BLSTM) model to learn human drivers' behaviors and habits from their historical trajectory data. The model predicts the probability distribution of surrounding vehicles' positions, which are used to estimate dynamic conflict risks. N…
Fine-Tuning of Neural Network Approximate MPC without Retraining via Bayesian Optimization
Institute for Data Science in Mechanical Engineering (DSME)（机械工程数据科学研究所）；RWTH Aachen University（亚琛工业大学）；Learning Systems and Robotics Lab（学习系统与机器人实验室）；Technical University of Munich（慕尼黑技术大学）
Mobile manipulation in robotics is challenging due to the need of solving many diverse tasks, such as opening a door or picking-and-placing an object. Typically, a basic first-principles system description of the robot is available, thus motivating the use of model-based controllers. However, the robot dynamics and its interaction with an object are affected by uncertainty, limiting the controller's performance. To tackle this problem, we propose a Bayesian multi-task learning model that uses trigonometric basis functions to identify the error in the dynamics. In this way, data from different …
Michigan State University（密歇根州立大学）；Utah State University（犹他州立大学）
This paper presents a sample-efficient data-driven method to design model predictive control (MPC) for cable-actuated soft robotics using Bayesian optimization. Instead of modeling the complex dynamics of the soft robots, the proposed approach uses Bayesian optimization to search the best-guessed low-dimensional prediction model and its associated controller to minimize the objective function of closed-loop responses. The prediction model is updated by Bayesian optimization from the closed-loop input-output data in each iteration. A linear MPC is then designed based on the updated prediction m…
The End-of-Line (EoL) calibration of semi-active suspension systems for road vehicles is usually a critical and expensive task, needing a team of vehicle and control experts as well as many hours of professional driving. In this paper, we propose a purely data-based tuning method enabling the automatic calibration of the parameters of a proprietary suspension controller by relying on little experimental time and exploiting Bayesian Optimization tools. A detailed methodology on how to select the most critical degrees of freedom of the algorithm is also provided. The effectiveness of the propose…
An approach to resilient planning and control of autonomous vehicles in multi-vehicle traffic scenarios is proposed. The proposed method is based on model predictive control (MPC), where alternative predictions of the surrounding traffic are determined automatically such that they are intentionally adversarial to the ego vehicle. This provides robustness against the inherent uncertainty in traffic predictions. To reduce conservatism, an assumption that other agents are of no ill intent is formalized. Simulation results from highway driving scenarios show that the proposed method in real-time n…
Model predictive control (MPC) is a promising approach for the lateral and longitudinal control of autonomous vehicles. However, the parameterization of the MPC with respect to high-level requirements such as passenger comfort as well as lateral and longitudinal tracking is a challenging task. Numerous tuning parameters as well as conflicting requirements need to be considered. This contribution formulates the MPC tuning task as a multi-objective optimization problem. Solving it is challenging for two reasons: First, MPC-parameterizations are evaluated on an computationally expensive simulatio…
An autonomous adaptive MPC architecture is presented for control of heating, ventilation and air condition (HVAC) systems to maintain indoor temperature while reducing energy use. Although equipment use and occupant changes with time, existing MPC methods are not capable of automatically relearning models and computing control decisions reliably for extended periods without intervention from a human expert. We seek to address this weakness. Two major features are embedded in the proposed architecture to enable autonomy: (i) a system identification algorithm from our prior work that periodicall…
This paper presents a systematic method for the selection of the Model Predictive Control (MPC) stage cost. We match the MPC feedback law to a proportional-integral (PI) controller, which we efficiently tune by high-performance Monte Carlo (MC) simulation. The PI tuning offers a wide range of tuning possibilities that is then inherited by the MPC design. The MC simulation tuning of the PI controller is based on the minimization of two different objectives; 1) the 2-norm tracking error, and 2) a bi-objective consisting of the 2-norm tracking error and a 2-norm input rate of movement penalty. We…
Delft University of Technology（代尔夫特理工大学）；University of Eindhoven（埃因霍温大学）
This paper presents a novel methodology to auto-tune an Unscented Kalman Filter (UKF). It involves using a Two-Stage Bayesian Optimisation (TSBO), based on a t-Student Process to optimise the process noise parameters of a UKF for vehicle sideslip angle estimation. Our method minimises performance metrics, given by the average sum of the states' and measurement' estimation error for various vehicle manoeuvres covering a wide range of vehicle behaviour. The predefined cost function is minimised through a TSBO which aims to find a location in the feasible region that maximises the probability of …
Bosch Center for Artificial Intelligence, Robert Bosch GmbH（博世人工智能中心，罗伯特·博世股份有限公司）；Institute for Dynamic Systems and Control, ETH Zurich（苏黎世联邦理工学院动态系统与控制研究所）
Bayesian Optimization (BO) is an effective method for optimizing expensive-to-evaluate black-box functions with a wide range of applications for example in robotics, system design and parameter optimization. However, scaling BO to problems with large input dimensions (>10) remains an open challenge. In this paper, we propose to leverage results from optimal control to scale BO to higher dimensional control tasks and to reduce the need for manually selecting the optimization domain. The contributions of this paper are twofold: 1) We show how we can make use of a learned dynamics model in combin…
Our paper proposes a model predictive controller as a single-task formulation that simultaneously optimizes wheel and torso motions. This online joint velocity and ground reaction force optimization integrates a kinodynamic model of a wheeled quadrupedal robot. It defines the single rigid body dynamics along with the robot's kinematics while treating the wheels as moving ground contacts. With this approach, we can accurately capture the robot's rolling constraint and dynamics, enabling automatic discovery of hybrid maneuvers without needless motion heuristics. The formulation's generality thro…
Direct transfer of optimized controllers to similar systems using dimensionless MPC
Faculty of Electrical Engineering and Computing, University of Zagreb（电子工程与计算学院，泽格里布大学）；Department of Engineering Cybernetics, Norwegian University of Science and Technology (NTNU)（工程 cybernetics 系，挪威科学与技术大学（NTNU））
Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics
1 Learning \& Adaptive Systems Group, Department of Computer Science, ETH Zurich, Switzerland. 2 Dynamic Systems Lab, Institute for Aerospace Studies, University of Toronto, Canada.
We present differentiable predictive control (DPC), a method for learning constrained neural control policies for linear systems with probabilistic performance guarantees. We employ automatic differentiation to obtain direct policy gradients by backpropagating the model predictive control (MPC) loss function and constraints penalties through a differentiable closed-loop system dynamics model. We demonstrate that the proposed method can learn parametric constrained control policies to stabilize systems with unstable dynamics, track time-varying references, and satisfy nonlinear state and input …
