Linear Model Predictive Control (MPC) is a widely used method to control systems with linear dynamics. Efficient interior-point methods have been proposed which leverage the block diagonal structure of the quadratic program (QP) resulting from the receding horizon control formulation. However, they require two matrix factorizations per interior-point method iteration, one each for the computation of the dual and the primal. Recently though an interior point method based on the null-space method has been proposed which requires only a single decomposition per iteration. While the then used null-space basis leads to dense null-space projections, in this work we propose a sparse null-space basis which preserves the block diagonal structure of the MPC matrices. Since it is based on the inverse of the transfer matrix we introduce the notion of so-called virtual controls which enables just that invertibility. A combination of the reduced number of factorizations and omission of the evaluation of the dual lets our solver outperform others in terms of computational speed by an increasing margin dependent on the number of state and control variables.
In this work, in the context of Linear and Quadratic Programming, we interpret Primal Dual Regularized Interior Point Methods (PDR-IPMs) in the framework of the Proximal Point Method. The resulting Proximal Stabilized IPM (PS-IPM) is strongly supported by theoretical results concerning convergence and the rate of convergence, and can handle degenerate problems. Moreover, in the second part of this work, we analyse the interactions between the regularization parameters and the computational foot-print of the linear algebra routines used to solve the Newton linear systems. In particular, when th…
In this paper we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM). The resulting algorithm (IP-PMM) is interpreted as a primal-dual regularized IPM, suitable for solving linearly constrained convex quadratic programming problems. We apply few iterations of the interior point method to each sub-problem of the proximal method of multipliers. Once a satisfactory solution of the PMM sub-problem is found, we update the PMM parameters, form a new IPM neighbourhood and repeat this process. Given this framework, we prove polynomial complexity of the algor…
In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in [An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming, Computational Optimization and Applications, 78, 307--351 (2021)] for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. In particular, we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM) and interpret the algorithm (IP-PMM) as a primal-dual regularized IPM, suitable for sol…
Quantum linear system algorithms (QLSAs) have the potential to speed up algorithms that rely on solving linear systems. Interior Point Methods (IPMs) yield a fundamental family of polynomial-time algorithms for solving optimization problems. IPMs solve a Newton linear system at each iteration to find the search direction, and thus QLSAs can potentially speed up IPMs. Due to the noise in contemporary quantum computers, such quantum-assisted IPM (QIPM) only allows an inexact solution for the Newton linear system. Typically, an inexact search direction leads to an infeasible solution. In our work…
This paper studies the worst case iteration complexity of an infeasible interior point method (IPM) for seconder order cone programming (SOCP), which is more convenient for warmstarting compared with feasible IPMs. The method studied bases on the homogeneous and self-dual model and the Monteiro-Zhang family of searching directions. Its worst case iteration complexity is $O\left(k^{1/2}\log\left(ε^{-1}\right)\right)$, to reduce the primal residual, dual residual, and complementarity gap by a factor of $ε$, where $k$ is the number of cone constraints. The result is the same as the best known res…
We analyze sequences generated by interior point methods (IPMs) in convex and nonconvex settings. We prove that moving the primal feasibility at the same rate as the barrier parameter $μ$ ensures the Lagrange multiplier sequence remains bounded, provided the limit point of the primal sequence has a Lagrange multiplier. This result does not require constraint qualifications. We also guarantee the IPM finds a solution satisfying strict complementarity if one exists. On the other hand, if the primal feasibility is reduced too slowly, then the algorithm converges to a point of minimal complementar…
The interior-point method (IPM) has become the workhorse method for nonlinear programming. The performance of IPM is directly related to the linear solver employed to factorize the Karush--Kuhn--Tucker (KKT) system at each iteration of the algorithm. When solving large-scale nonlinear problems, state-of-the art IPM solvers rely on efficient sparse linear solvers to solve the KKT system. Instead, we propose a novel reduced-space IPM algorithm that condenses the KKT system into a dense matrix whose size is proportional to the number of degrees of freedom in the problem. Depending on where the re…
We develop a new interior-point method (IPM) for symmetric-cone optimization, a common generalization of linear, second-order-cone, and semidefinite programming. In contrast to classical IPMs, we update iterates with a geodesic of the cone instead of the kernel of the linear constraints. This approach yields a primal-dual-symmetric, scale-invariant, and line-search-free algorithm that uses just half the variables of a standard primal-dual IPM. With elementary arguments, we establish polynomial-time convergence matching the standard square-root-n bound. Finally, we prove global convergence of a…
We consider a class of optimization problems for sparse signal reconstruction which arise in the field of Compressed Sensing (CS). A plethora of approaches and solvers exist for such problems, for example GPSR, FPC AS, SPGL1, NestA, $\ell_{1}_\ell_{s}$, PDCO to mention a few. Compressed Sensing applications lead to very well conditioned optimization problems and therefore can be solved easily by simple first-order methods. Interior point methods (IPMs) rely on the Newton method hence they use the second-order information. They have numerous advantageous features and one clear drawback: being t…
The DC network security constraints have been extensively studied in numerous power system problems, such as optimal power flow (OPF), security-constrained economic dispatch (SCED), and security-constrained unit commitment (SCUC). Linear shift factors, i.e., power transfer distribution factors (PTDFs), are widely applied to replace DC power flow constraints. However, the PTDF matrix is extremely dense, making it difficult to solve security-constraint optimization problems. This paper analyzes/investigates the computational inefficiency of PTDF-based security constraints from the sparse structu…
We propose a new framework to implement interior point method (IPM) to solve very large linear programs (LP). Traditional IPMs typically use Newton's method to approximately solve a subproblem that aims to minimize a log-barrier penalty function at each iteration. Due its connection to Newton's method, IPM is often classified as second-order method -- a genre that is attached with stability and accuracy at the expense of scalability. Indeed, computing a Newton step amounts to solving a large linear system, which can be efficiently implemented if the input data are reasonably-sized and/or spars…
The work of Wachter and Biegler suggests that infeasible-start interior point methods (IPMs) developed for linear programming cannot be adapted to nonlinear optimization without significant modification, i.e., using a two-phase or penalty method. We propose an IPM that, by careful initialization and updates of the slack variables, is guaranteed to find a first-order certificate of local infeasibility, local optimality or unboundedness of the (shifted) feasible region. Our proposed algorithm differs from other IPM methods for nonconvex programming because we reduce primal feasibility at the sam…
Dynamic economic dispatch with valve-point effect (DED-VPE) is a non-convex and non-differentiable optimization problem which is difficult to solve efficiently. In this paper, a hybrid mixed integer linear programming (MILP) and interior point method (IPM), denoted by MILP-IPM, is proposed to solve such a DED-VPE problem, where the complicated transmission loss is also included. Due to the non-differentiable characteristic of DED-VPE, the classical derivative-based optimization methods can not be used any more. With the help of model reformulation, a differentiable non-linear programming (NLP)…
This paper introduces HPIPM, a high-performance framework for quadratic programming (QP), designed to provide building blocks to efficiently and reliably solve model predictive control problems. HPIPM currently supports three QP types, and provides interior point method (IPM) solvers as well (partial) condensing routines. In particular, the IPM for optimal control QPs is intended to supersede the HPMPC solver, and it largely improves robustness while keeping the focus on speed. Numerical experiments show that HPIPM reliably solves challenging QPs, and that it outperforms other state-of-the-art…
We propose an interior point method (IPM) for solving semidefinite programming problems (SDPs). The standard interior point algorithms used to solve SDPs work in the space of positive semidefinite matrices. Contrary to that the proposed algorithm works in the cone of matrices of constant \emph{factor width}. This adaptation makes the proposed method more suitable for parallelization than the standard IPM. We prove global convergence and provide a complexity analysis. Our work is inspired by a series of papers by Ahmadi, Dash, Majumdar and Hall, and builds upon a recent preprint by Roig-Solvas …
Due to critical environmental issues, the power systems have to accommodate a significant level of penetration of renewable generation which requires smart approaches to the power grid control. Associated optimal control problems are large-scale nonlinear optimization problems with up to hundreds of millions of variables and constraints. The interior point methods become computationally intractable, mainly due to the solution of large linear systems. This document addresses the computational bottlenecks of the interior point method during the solution of the security constrained optimal power …
Interior point methods (IPMs) are a common approach for solving linear programs (LPs) with strong theoretical guarantees and solid empirical performance. The time complexity of these methods is dominated by the cost of solving a linear system of equations at each iteration. In common applications of linear programming, particularly in machine learning and scientific computing, the size of this linear system can become prohibitively large, requiring the use of iterative solvers, which provide an approximate solution to the linear system. However, approximately solving the linear system at each …
Powerful interior-point methods (IPM) based commercial solvers, such as Gurobi and Mosek, have been hugely successful in solving large-scale linear programming (LP) problems. The high efficiency of these solvers depends critically on the sparsity of the problem data and advanced matrix factorization techniques. For a large scale LP problem with data matrix $A$ that is dense (possibly structured) or whose corresponding normal matrix $AA^T$ has a dense Cholesky factor (even with re-ordering), these solvers may require excessive computational cost and/or extremely heavy memory usage in each inter…
Interior point methods are among the most popular techniques for large scale nonlinear optimization, owing to their intrinsic ability of scaling to arbitrary large problem sizes. Their efficiency has attracted in recent years a lot of attention due to increasing demand for large scale optimization in industry and engineering. A parallel interior point method is discussed that exploits the intrinsic structure of large-scale nonlinear optimization problems so that the solution process can employ massively parallel high-performance computing infastructures. Since the overall performance of interi…
This paper introduces a computationally efficient approach for solving Model Predictive Control (MPC) reference tracking problems with state and control constraints. The approach consists of three key components: First, a log-domain interior-point quadratic programming method that forms the basis of the overall approach; second, a method of warm-starting this optimizer by using the MPC solution from the previous timestep; and third, a computational governor that bounds the suboptimality of the warm-start by altering the reference command provided to the MPC problem. As a result, the closed-loo…
A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
Department of Mathematics, University of California, Berkeley（加州大学伯克利分校数学系）；Department of IEOR, University of California, Berkeley（加州大学伯克利分校工业工程与运营研究系）；School of Industrial and Systems Engineering, Georgia Tech（佐治亚理工学院工业与系统工程学院）；Department of EECS, University of California, Berkeley（加州大学伯克利分校电子工程与计算机科学系）
Computational acceleration strategies for large-scale energy system optimization: a comparative study of GPU-accelerated and distributed-memory solvers
Freie Universität Berlin（柏林自由大学）；Deutsches Zentrum für Luft- und Raumfahrt e.V.（德国航空航天中心）；Zuse Institute Berlin（祖斯研究所柏林）；GAMS Software GmbH（GAMS软件有限公司）
We present a faster interior-point method for optimizing sum-of-squares (SOS) polynomials, which are a central tool in polynomial optimization and capture convex programming in the Lasserre hierarchy. Let $p = \sum_i q^2_i$ be an $n$-variate SOS polynomial of degree $2d$. Denoting by $L := \binom{n+d}{d}$ and $U := \binom{n+2d}{2d}$ the dimensions of the vector spaces in which $q_i$'s and $p$ live respectively, our algorithm runs in time $\tilde{O}(LU^{1.87})$. This is polynomially faster than state-of-art SOS and semidefinite programming solvers, which achieve runtime $\tilde{O}(L^{0.5}\min\{…
Input constrained Model predictive control (MPC) includes an optimization problem which should iteratively be solved at each time-instance. The well-known drawback of model predictive control is the computational cost of the optimization problem. This results in restriction of the application of MPC to systems with slow dynamics, e.g., process control systems and small-scale problems. Therefore, implementing fast numerical optimization algorithms has been a point of interest. Interior-point methods are proved to be appropriate algorithms, from computational cost point-of-vie, to solve input-co…
We propose a primal-dual interior-point (PDIP) method for solving quadratic programming problems with linear inequality constraints that typically arise form MPC applications. We show that the solver converges (locally) quadratically to a suboptimal solution of the MPC problem. PDIP solvers rely on two phases: the damped and the pure Newton phases. Compared to state-of-the-art PDIP methods, our solver replaces the initial damped Newton phase (usually used to compute a medium-accuracy solution) with a dual solver based on Nesterov's fast gradient scheme (DFG) that converges with a sublinear con…
