Model Predictive Control has been recently proposed as policy approximation for Reinforcement Learning, offering a path towards safe and explainable Reinforcement Learning. This approach has been investigated for Q-learning and actor-critic methods, both in the context of nominal Economic MPC and Robust (N)MPC, showing very promising results. In that context, actor-critic methods seem to be the most reliable approach. Many applications include a mixture of continuous and integer inputs, for which the classical actor-critic methods need to be adapted. In this paper, we present a policy approximation based on mixed-integer MPC schemes, and propose a computationally inexpensive technique to generate exploration in the mixed-integer input space that ensures a satisfaction of the constraints. We then propose a simple compatible advantage function approximation for the proposed policy, that allows one to build the gradient of the mixed-integer MPC-based policy.
Machine Learning Algorithms for Improving Exact Classical Solvers in Mixed Integer Continuous Optimization
Fakultät für Mathematik, Universität Wien（维也纳大学数学系）；Department of Computer Science, Czech Technical University（捷克技术大学计算机科学系）；CEO Olicheza Limited（Olicheza有限公司）
A new motion planning framework for automated highway merging is presented in this paper. To plan the merge and predict the motion of the neighboring vehicle, the ego automated vehicle solves a joint optimization of both vehicle costs over a receding horizon. The non-convex nature of feasible regions and lane discipline is handled by introducing integer decision variables resulting in a mixed integer quadratic programming (MIQP) formulation of the model predictive control (MPC) problem. Furthermore, the ego uses an inverse optimal control approach to impute the weights of neighboring vehicle c…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Today's fast linear algebra and numerical optimization tools have pushed the frontier of model predictive control (MPC) forward, to the efficient control of highly nonlinear and hybrid systems. The field of hybrid MPC has demonstrated that exact optimal control law can be computed, e.g., by mixed-integer programming (MIP) under piecewise-affine (PWA) system models. Despite the elegant theory, online solving hybrid MPC is still out of reach for many applications. We aim to speed up MIP by combining geometric insights from hybrid MPC, a simple-yet-effective learning algorithm, and MIP warm start…
Real-time motion planning and decision-making for a group of differential drive robots under connectivity constraints using robust MPC and mixed-integer programming
In this paper, we develop an optimal weight adaptation strategy of model predictive control (MPC) for connected and automated vehicles (CAVs) in mixed traffic. We model the interaction between a CAV and a human-driven vehicle (HDV) as a simultaneous game and formulate a game-theoretic MPC problem to find a Nash equilibrium of the game. In the MPC problem, the weights in the HDV's objective function can be learned online using moving horizon inverse reinforcement learning. Using Bayesian optimization, we propose a strategy to optimally adapt the weights in the CAV's objective function so that t…
In hybrid Model Predictive Control (MPC), a Mixed-Integer Quadratic Program (MIQP) is solved at each sampling time to compute the optimal control action. Although these optimizations are generally very demanding, in MPC we expect consecutive problem instances to be nearly identical. This paper addresses the question of how computations performed at one time step can be reused to accelerate (warm start) the solution of subsequent MIQPs. Reoptimization is not a rare practice in integer programming: for small variations of certain problem data, the branch-and-bound algorithm allows an efficient r…
The design of cooperative adaptive cruise control is critical in mixed traffic flow, where connected and automated vehicles (CAVs) and human-driven vehicles (HDVs) coexist. Compared with pure CAVs, the major challenge is how to handle the prediction uncertainty of HDVs, which can cause significant state deviation of CAVs from planned trajectories. In most existing studies, model predictive control (MPC) is utilized to replan CAVs' trajectories to mitigate the deviation at each time step. However, as the replan process is usually conducted by solving an optimization problem with information thr…
We present a new mixed integer formulation for the discrete informative path planning problem in random fields. The objective is to compute a budget constrained path while collecting measurements whose linear estimate results in minimum error over a finite set of prediction locations. The problem is known to be NP-hard. However, we strive to compute optimal solutions by leveraging advances in mixed integer optimization. Our approach is based on expanding the search space so we optimize not only over the collected measurement subset, but also over the class of all linear estimators. This allows…
In this paper we propose a technique that assigns obstacles to clusters used for collision avoidance via Mixed-Integer Programming. This strategy enables a reduction in the number of binary variables used for collision avoidance, thus entailing a decrease in computational cost, which has been a hindrance to the application of Model Predictive Control approaches with Mixed-Integer Programming formulations in real-time. Moreover, the assignment of obstacles to clusters and the sizes of the clusters are decided within the same optimization problem that performs the trajectory planning, thus yield…
Mix-CALADIN: A Distributed Algorithm for Consensus Mixed-Integer Optimization
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；KTH Royal Institute of Technology（瑞典皇家理工学院）；The Hong Kong University of Science and Technology（香港科技大学）
We consider the problem of computing a mixed-strategy generalized Nash equilibrium (MS-GNE) for a class of games where each agent has both continuous and integer decision variables. Specifically, we propose a novel Bregman forward-reflected-backward splitting and design distributed algorithms that exploit the problem structure. Technically, we prove convergence to a variational MS-GNE under mere monotonicity and Lipschitz continuity assumptions, which are typical of continuous GNE problems. Finally, we show the performance of our algorithms via numerical experiments.
This note discusses properties of parametric discrete-time Mixed-Integer Optimal Control Problems (MIOCPs) as they often arise in mixed-integer NMPC. We argue that in want for a handle on similarity properties of parametric MIOCPs the classical turnpike notion from optimal control is helpful. We provide sufficient turnpike conditions based on a dissipativity notion of MIOCPs, and we show that the turnpike property allows specific and accurate guesses for the integer-valued controls. Moreover, we show how the turnpike property can be used to derive efficient node-weighted branch-and-bound schem…
Beijing Research Institute of UBTECH Robotics（优必选机器人北京研究院）；Tsinghua University（清华大学）
The hierarchical quadratic programming (HQP) is commonly applied to consider strict hierarchies of multi-tasks and robot's physical inequality constraints during whole-body compliance. However, for the one-step HQP, the solution can oscillate when it is close to the boundary of constraints. It is because the abrupt hit of the bounds gives rise to unrealisable jerks and even infeasible solutions. This paper proposes the mixed control, which blends the single-axis model predictive control (MPC) and proportional derivate (PD) control for the whole-body compliance to overcome these deficiencies. T…
Discovering governing equations of complex dynamical systems directly from data is a central problem in scientific machine learning. In recent years, the sparse identification of nonlinear dynamics (SINDy) framework, powered by heuristic sparse regression methods, has become a dominant tool for learning parsimonious models. We propose an exact formulation of the SINDy problem using mixed-integer optimization (MIO) to solve the sparsity constrained regression problem to provable optimality in seconds. On a large number of canonical ordinary and partial differential equations, we illustrate the …
This paper presents an efficient suboptimal model predictive control (MPC) algorithm for nonlinear switched systems subject to minimum dwell time constraints (MTC). While MTC are required for most physical systems due to stability, power and mechanical restrictions, MPC optimization problems with MTC are challenging to solve. To efficiently solve such problems, the on-line MPC optimization problem is decomposed into a sequence of simpler problems, which include two nonlinear programs (NLP) and a rounding step, as typically done in mixed-integer optimal control (MIOC). Unlike the classical appr…
We propose a supervised learning framework for computing solutions of multi-parametric Mixed Integer Linear Programs (MILPs) that arise in Model Predictive Control. Our approach also quantifies sub-optimality for the computed solutions. Inspired by Branch-and-Bound techniques, the key idea is to train a Neural Network/Random Forest, which for a given parameter, predicts a strategy consisting of (1) a set of Linear Programs (LPs) such that their feasible sets form a partition of the feasible set of the MILP and (2) a candidate integer solution. For control computation and sub-optimality quantif…
We formulate for the first time the economic dispatch problem in an integrated electrical and gas distribution system as a game equilibrium problem between distributed prosumers. Specifically, by approximating the non-linear gas-flow equations either with a mixed-integer second order cone or a piece-wise affine model and by assuming that electricity and gas prices depend linearly on the total consumption we obtain a potential mixed-integer game. To compute an approximate generalized Nash equilibrium, we propose an iterative two-stage method that exploits a problem convexification and the gas f…
In this paper, we propose an approach to formulate the Weapon Target Assignment (WTA) problem with physical and seeker interference constraints which is solvable in Mixed Integer Linear Programming (MILP). To handle the interference constraint which is intractable in continuous time domain, we discretize the time window and generate predicted intercept point (PIP) set. Also, to handle interference constraints to the MILP formulation, the interference tables that explain which pairs of assignments make interference, are made prematurely before the solver executed. The simulation results whether…
Many systems exhibit a mixture of continuous and discrete dynamics. We consider a family of mixed-integer non-convex non-linear optimisation problems obtained in discretisations of optimal control of such systems. For this family, a branch-and-bound algorithm solves the discretised problem to global optimality. As an example, we consider water systems, where variations in flow and variations in water levels are continuous, while decisions related to fixed-speed pumps and whether gates that may be opened and closed are discrete. We show that the related optimal-control problems come from the fa…
We present a methodology for computing the flexibility index when uncertainty is characterized using multivariate Gaussian random variables. Our approach computes the flexibility index by solving a mixed-integer conic program (MICP). This methodology directly characterizes ellipsoidal sets to capture correlations in contrast to previous methodologies that employ approximations. We also show that, under a Gaussian representation, the flexibility index can be used to obtain a lower bound for the so-called stochastic flexibility index (i.e., the probability of having feasible operation). Our resu…
Convex MPC and Thrust Allocation with Deadband for Spacecraft Rendezvous
Instituto Superior Técnico（高等技术学院）；University of Lisbon（里斯本大学）；Institute for Systems and Robotics（系统与机器人研究所）；School of Science and Technology from the NOVA University of Lisbon（里斯本新大学科学与技术学院）；Lusófona University（卢索福纳大学）；GMV（GMV公司）
We consider potential games with mixed-integer variables, for which we propose two distributed, proximal-like equilibrium seeking algorithms. Specifically, we focus on two scenarios: i) the underlying game is generalized ordinal and the agents update through iterations by choosing an exact optimal strategy; ii) the game admits an exact potential and the agents adopt approximated optimal responses. By exploiting the properties of integer-compatible regularization functions used as penalty terms, we show that both algorithms converge to either an exact or an $ε$-approximate equilibrium. We corro…
