Compact mixed-integer programming relaxations in quadratic optimization
Ben Beach, Robert Hildebrand, Joey Huchette
备注：
48 pages, 4 figures. Submitted to the Journal of Global Optimization, 3-27-2021. This replacement is an upgrade from a conference paper to a journal article, and is greatly revised and expanded in scope
We present a technique for producing valid dual bounds for nonconvex quadratic optimization problems. The approach leverages an elegant piecewise linear approximation for univariate quadratic functions due to Yarotsky, formulating this (simple) approximation using mixed-integer programming (MIP). Notably, the number of constraints, binary variables, and auxiliary continuous variables used in this formulation grows logarithmically in the approximation error. Combining this with a diagonal perturbation technique to convert a nonseparable quadratic function into a separable one, we present a mixed-integer convex quadratic relaxation for nonconvex quadratic optimization problems. We study the strength (or sharpness) of our formulation and the tightness of its approximation. Further, we show that our formulation represents feasible points via a Gray code. We close with computational results on problems with quadratic objectives and/or constraints, showing that our proposed method i) across the board outperforms existing MIP relaxations from the literature, and ii) on hard instances produces better bounds than exact solvers within a fixed time budget.
It has recently been shown (Burer, Math. Program Ser. A 120:479-495, 2009) that a large class of NP-hard nonconvex quadratic programming problems can be modeled as so called completely positive programming problems, which are convex but still NP-hard in general. A basic tractable relaxation is gotten by doubly nonnegative relaxation, resulting in a doubly nonnegative programming. In this paper, we prove that doubly nonnegative relaxation for binary quadratic programming (BQP) problem is equivalent to a tighter semidifinite relaxation for it. When problem (BQP) reduces to max-cut (MC) problem, …
This paper studies a class of so-called linear semi-infinite polynomial programming (LSIPP) problems. It is a subclass of linear semi-infinite programming problems whose constraint functions are polynomials in parameters and index sets are basic semialgebraic sets. We present a hierarchy of semidefinite programming (SDP) relaxations for LSIPP problems. Convergence rate analysis of the SDP relaxations is established based on some existing results. We show how to verify the compactness of feasible sets of LSIPP problems. In the end, we extend the SDP relaxation method to more general semi-infini…
Binary Integer Programming (BIP) problems are of interest due in part to the difficulty they pose and because of their various applications, including those in graph theory, combinatorial optimization and network optimization. In this note, we explicitly state the Lovasz-Schrijver Semidefinite Programming (SDP) relaxation (in primal-standard form) for a BIP problem, a relaxation that yields a tighter upper-bound than the canonical Linear Programming relaxation.
Among many approaches to increase the computational efficiency of semidefinite programming (SDP) relaxation for quadratic constrained quadratic programming problems (QCQPs), exploiting the aggregate sparsity of the data matrices in the SDP by Fukuda et al. (2001) and second-order cone programming (SOCP) relaxation have been popular. In this paper, we exploit the aggregate sparsity of SOCP relaxation of QCQPs. Specifically, we prove that exploiting the aggregate sparsity reduces the number of second-order cones in the SOCP relaxation, and that we can simplify the matrix completion procedure by …
This paper studies how to solve semi-infinite polynomial programming (SIPP) problems by semidefinite relaxation method. We first introduce two SDP relaxation methods for solving polynomial optimization problems with finitely many constraints. Then we propose an exchange algorithm with SDP relaxations to solve SIPP problems with compact index set. At last, we extend the proposed method to SIPP problems with noncompact index set via homogenization. Numerical results show that the algorithm is efficient in practice.
We extend a relaxation technique due to Bertsimas and Nino-Mora for the restless bandit problem to the case where arbitrary costs penalize switching between the bandits. We also construct a one-step lookahead policy using the solution of the relaxation. Computational experiments and a bound for approximate dynamic programming provide some empirical support for the heuristic.
In this paper, we consider the quadratic programming problems under finitely many convex quadratic constraints in Hilbert spaces. By using the Legendre property of quadratic forms or the compactness of operators in the presentations of quadratic forms, we establish some sufficient conditions for the solution existence of the considered problems. As special cases, we obtain some existence solution results for the quadratic programming problems under linear constraints in Hilbert spaces.
Superlinear and quadratic convergence of a stabilized sequential quadratic semidefinite programming method for nonlinear semidefinite programming problems
We study linear programming relaxations of nonconvex quadratic programs given by the reformulation-linearization technique (RLT), referred to as RLT relaxations. We investigate the relations between the polyhedral properties of the feasible regions of a quadratic program and its RLT relaxation. We establish various connections between recession directions, boundedness, and vertices of the two feasible regions. Using these properties, we present necessary and sufficient exactness conditions for RLT relaxations. We then give a thorough discussion of how our results can be converted into simple a…
This paper focuses on the study of a mathematical program with equilibrium constraints, where the objective and the constraint functions are all polynomials. We present a method for finding its global minimizers and global minimum using a hierarchy of semidefinite programming (SDP) relaxations and prove the convergence result for the method. Numerical experiments are presented to show the efficiency of the proposed algorithm.
We consider the global optimization of nonconvex mixed-integer quadratic programs with linear equality constraints. In particular, we present a new class of convex quadratic relaxations which are derived via quadratic cuts. To construct these quadratic cuts, we solve a separation problem involving a linear matrix inequality with a special structure that allows the use of specialized solution algorithms. Our quadratic cuts are nonconvex, but define a convex feasible set when intersected with the equality constraints. We show that our relaxations are an outer-approximation of a semi-infinite con…
The geometric measure of entanglement is a widely used entanglement measure for quantum pure states. The key problem of computation of the geometric measure is to calculate the entanglement eigenvalue, which is equivalent to computing the largest unitary eigenvalue of a corresponding complex tensor. In this paper, we propose a Jacobian semidefinite programming relaxation method to calculate the largest unitary eigenvalue of a complex tensor. For this, we first introduce the Jacobian semidefinite programming relaxation method for a polynomial optimization with equality constraint, and then conv…
We consider a parametric convex quadratic programming, CQP, relaxation for the quadratic knapsack problem, QKP. This relaxation maintains partial quadratic information from the original QKP by perturbing the objective function to obtain a concave quadratic term. The nonconcave part generated by the perturbation is then linearized by a standard approach that lifts the problem to the matrix space. We present a primal-dual interior point method to optimize the perturbation of the quadratic function, in a search for the tightest upper bound for the \QKPp. We prove that the same perturbation approa…
In this paper, we consider the nonconvex quadratically constrained quadratic programming (QCQP) with one quadratic constraint. By employing the conjugate gradient method, an efficient algorithm is proposed to solve QCQP that exploits the sparsity of the involved matrices and solves the problem via solving a sequence of positive definite system of linear equations after identifying suitable generalized eigenvalues. Some numerical experiments are given to show the effectiveness of the proposed method and to compare it with some recent algorithms in the literature.
We introduce the Suggest-and-Improve framework for general nonconvex quadratically constrained quadratic programs (QCQPs). Using this framework, we generalize a number of known methods and provide heuristics to get approximate solutions to QCQPs for which no specialized methods are available. We also introduce an open-source Python package QCQP, which implements the heuristics discussed in the paper.
We study convex relaxations of nonconvex quadratic programs. We identify a family of so-called feasibility preserving convex relaxations, which includes the well-known copositive and doubly nonnegative relaxations, with the property that the convex relaxation is feasible if and only if the nonconvex quadratic program is feasible. We observe that each convex relaxation in this family implicitly induces a convex underestimator of the objective function on the feasible region of the quadratic program. This alternative perspective on convex relaxations enables us to establish several useful proper…
We consider the global optimization of nonconvex quadratic programs and mixed-integer quadratic programs. We present a family of convex quadratic relaxations which are derived by convexifying nonconvex quadratic functions through perturbations of the quadratic matrix. We investigate the theoretical properties of these quadratic relaxations and show that they are equivalent to some particular semidefinite programs. We also introduce novel branching variable selection strategies which can be used in conjunction with the quadratic relaxations investigated in this paper. We integrate the proposed …
We consider the problem of approximately solving a standard bi-quadratic programming (StBQP), which is NP-hard. After reformulating the original problem as an equivalent copositive tensor programming, we show how to approximate the optimal solution by approximating the cone of copositive tensors via a serial polyhedral cones. The established quality of approximation shows that, a polynomial time approximation scheme (PTAS) for solving StBQP exists and can be extended to solving standard multi-quadratic programming. Some numerical examples are provided to illustrate our approach.
Quadratic programs with box constraints involve minimizing a possibly nonconvex quadratic function subject to lower and upper bounds on each variable. This is a well-known NP-hard problem that frequently arises in various applications. We focus on two convex relaxations, namely the RLT (Reformulation-Linearization Technique) relaxation and the SDP-RLT relaxation obtained by adding semidefinite constraints to the RLT relaxation. Both relaxations yield lower bounds on the optimal value of a quadratic program with box constraints. We present complete algebraic descriptions of the set of instances…
In this paper, we analyze in depth a simplicial decomposition like algorithmic framework for large scale convex quadratic programming. In particular, we first propose two tailored strategies for handling the master problem. Then, we describe a few techniques for speeding up the solution of the pricing problem. We report extensive numerical experiments on both real portfolio optimization and general quadratic programming problems, showing the efficiency and robustness of the method when compared to Cplex.
Multicriterion optimization and Pareto optimality are fundamental tools in economics. In this paper we propose a new relaxation method for solving multiple objective quadratic programming problems. Exploiting the technique of the linear weighted sum method, we reformulate the original multiple objective quadratic programming problems into a single objective one. Since such single objective quadratic programming problem is still nonconvex and NP-hard in general. By using the techniques of lifting and doubly nonnegative relaxation, respectively, this single objective quadratic programming proble…
An equivalence between attainability of simultaneous diagonalization (SD) and hidden convexity in quadratically constrained quadratic programming (QCQP) stimulates us to investigate necessary and sufficient SD conditions, which is one of the open problems posted by Hiriart-Urruty (SIAM Rev., 49 (2007), pp. 255-273) nine years ago. In this paper we give a necessary and sufficient SD condition for any two real symmetric matrices and offer a necessary and sufficient SD condition for any finite collection of real symmetric matrices under the existence assumption of a semi-definite matrix pencil. M…
The radius of the outer Dikin ellipsoid of the intersection of $m$ ellipsoids due to Fu et al. (J. Comb. Optim., 2, 29-50, 1998) is corrected from $m$ to $\sqrt{m^2+m}$. The approximation bound for the general convex quadratic constrained nonconvex quadratic program is correspondingly corrected.
In this paper, we consider a bilevel polynomial optimization problem where the objective and the constraint functions of both the upper and the lower level problems are polynomials. We present methods for finding its global minimizers and global minimum using a sequence of semidefinite programming (SDP) relaxations and provide convergence results for the methods. Our scheme for problems with a convex lower-level problem involves solving a transformed equivalent single-level problem by a sequence of SDP relaxations; whereas our approach for general problems involving a non-convex polynomial low…
We study semidefinite programming (SDP) relaxations for the NP-hard problem of globally optimizing a quadratic function over the Stiefel manifold. We introduce a strengthened relaxation based on two recent ideas in the literature: (i) a tailored SDP for objectives with a block-diagonal Hessian; (ii) and the use of the Kronecker matrix product to construct SDP relaxations. Using synthetic instances on four problem classes, we show that, in general, our relaxation significantly strengthens existing relaxations, although at the expense of longer solution times.
In this paper, we study some bounds for nonconvex quadratically constrained quadratic programs. We propose two types of bounds for quadratically constrained quadratic programs, quadratic and cubic bounds. For quadratic bounds, we use affine functions as Lagrange multipliers. We demonstrate that most semi-definite relaxations can be obtained as the dual of a quadratic bound. In addition, we study bounds obtained by changing the ground set. For cubic bounds, in addition to affine multipliers we employ quadratic functions. We provide a comparison between the proposed cubic bound and typical bound…
In this paper, by improving the variable-splitting approach, we propose a new semidefinite programming (SDP) relaxation for the nonconvex quadratic optimization problem over the $\ell_1$ unit ball (QPL1). It dominates the state-of-the-art SDP-based bound for (QPL1). As extensions, we apply the new approach to the relaxation problem of the sparse principal component analysis and the nonconvex quadratic optimization problem over the $\ell_p$ ($1< p<2$) unit ball and then show the dominance of the new relaxation.
We present a short step interior point method for solving a class of nonlinear programming problems with quadratic objective function. Convex quadratic programming problems can be reformulated as problems in this class. The method is shown to have weak polynomial time complexity. A complete proof of the numerical stability of the method is provided. No requirements on feasibility, row-rank of the constraint Jacobian, strict complementarity, or conditioning of the problem are made. Infeasible problems are solved to an optimal interior least-squares solution.
The purpose of this paper is to study a non-convex fuzzy multi-objective quadratic programming problem, in which both the technological coefficients and resources are fuzzy with nonlinear membership function. A computational procedure to find a fuzzy efficient solution of this problem is developed. A numerical example is given to illustrate the procedure.
This paper provides the first meaningful documentation and analysis of an established technique which aims to obtain an approximate solution to linear programming problems prior to applying the primal simplex method. The underlying algorithm is a penalty method with naive approximate minimization in each iteration. During initial iterations an approach similar to augmented Lagrangian is used. Later the technique corresponds closely to a classical quadratic penalty method. There is also a discussion of the extent to which it can be used to obtain fast approximate solutions of LP problems, in pa…
In this paper, we give a new penalized semidefinite programming approach for non-convex quadratically-constrained quadratic programs (QCQPs). We incorporate penalty terms into the objective of convex relaxations in order to retrieve feasible and near-optimal solutions for non-convex QCQPs. We introduce a generalized linear independence constraint qualification (GLICQ) criterion and prove that any GLICQ regular point that is sufficiently close to the feasible set can be used to construct an appropriate penalty term and recover a feasible solution. Inspired by these results, we develop a heurist…
We present an efficient algorithm for solving fractional programming problems whose objective functions are the ratio of a low-rank quadratic to a positive definite quadratic with convex constraints. The proposed algorithm for these convex-convex problems is based on the Shen-Yu Quadratic Transform which finds stationary points of concave-convex sum-of-ratios problems. We further use elements of the algorithm proposed in [arXiv:1802.10192] and the classic Dinkelbach approach to ensure convergence. We show that our algorithm performs better than previous algorithms for low-rank problems.
Linear programming (LP) relaxation is a standard technique for solving hard combinatorial optimization (CO) problems. Here we present a gradient descent algorithm which exploits the special structure of some LP relaxations induced by CO problems. The algorithm can be run in parallel mode and was implemented as CUDA C/C++ program to be executed on GPU. We exemplify efficiency of the algorithm by solving a fractional 2-matching problem. Our results demonstrate that a fractional 2-matching problem with 100,000 nodes is solved by our algorithm on a modern GPU on a scale of a second while solving t…
We extend the notion of some energy-type expressions based on two sets, developed in the abstract potential theory. We also give the discretized version of the quantities defined, similar to Chebyshev constant. This extension allows to apply the potential-theoretic results to infinite quadratic programming problems. Together with a cutting plane algorithm, the Chebyshev-constant method ensures that under certain conditions, the infinite problem can be reduced to semi-infinite or to finite problems.
This paper aims to find efficient solutions to a multi-objective optimization problem (MP) with convex polynomial data. To this end, a hybrid method, which allows us to transform problem (MP) into a scalar convex polynomial optimization problem (P$_z$) and does not destroy the properties of convexity, is considered. First, we show an existence result for efficient solutions to problem (MP) under some mild assumption. Then, for problem (P$_z$), we establish two kinds of representations of non-negativity of convex polynomials over convex semi-algebraic sets, and propose two kinds of finite conve…
We consider the problem of partitioning the node set of a graph into $k$ sets of given sizes in order to \emph{minimize the cut} obtained using (removing) the $k$-th set. If the resulting cut has value $0$, then we have obtained a vertex separator. This problem is closely related to the graph partitioning problem. In fact, the model we use is the same as that for the graph partitioning problem except for a different \emph{quadratic} objective function. We look at known and new bounds obtained from various relaxations for this NP-hard problem. This includes: the standard eigenvalue bound, proje…
The alternating-current optimal power flow (ACOPF) is one of the best known non-convex non-linear optimisation problems. We present a novel re-formulation of ACOPF, which is based on lifting the rectangular power-voltage rank-constrained formulation, and makes it possible to derive alternative SDP relaxations. For those, we develop a first-order method based on the parallel coordinate descent with a novel closed-form step based on roots of cubic polynomials.
Sequential Quadratic Programming (SQP) is a powerful class of algorithms for solving nonlinear optimization problems. Local convergence of SQP algorithms is guaranteed when the Hessian approximation used in each Quadratic Programming subproblem is close to the true Hessian. However, a good Hessian approximation can be expensive to compute. Low cost Hessian approximations only guarantee local convergence under some assumptions, which are not always satisfied in practice. To address this problem, this paper proposes a simple method to guarantee local convergence for SQP with poor Hessian approxi…
This paper considers a quadratically-constrained cardinality minimization problem with applications to digital filter design, subset selection for linear regression, and portfolio selection. Two relaxations are investigated: the continuous relaxation of a mixed integer formulation, and an optimized diagonal relaxation that exploits a simple special case of the problem. For the continuous relaxation, an absolute upper bound on the optimal cost is derived, suggesting that the continuous relaxation tends to be a relatively poor approximation. In computational experiments, diagonal relaxations oft…
A novel algorithm to solve the quadratic programming problem over ellipsoids is proposed. This is achieved by splitting the problem into two optimisation sub-problems, quadratic programming over a sphere and orthogonal projection. Next, an augmented-Lagrangian algorithm is developed for this multiple constraint optimisation. Benefit from the fact that the QP over a single sphere can be solved in a closed form by solving a secular equation, we derive a tighter bound of the minimiser of the secular equation. We also propose to generate a new psd matrix with a low condition number from the matric…
A standard quadratic program is an optimization problem that consists of minimizing a (nonconvex) quadratic form over the unit simplex. We focus on reformulating a standard quadratic program as a mixed integer linear programming problem. We propose two alternative mixed integer linear programming formulations. Our first formulation is based on casting a standard quadratic program as a linear program with complementarity constraints. We then employ binary variables to linearize the complementarity constraints. For the second formulation, we first derive an overestimating function of the objecti…
In this paper, we present new convex relaxations for nonconvex quadratically constrained quadratic programming (QCQP) problems. While recent research has focused on strengthening convex relaxations using reformulation-linearization technique (RLT), the state-of-the-art methods lose their effectiveness when dealing with (multiple) nonconvex quadratic constraints in QCQP. In this research, we decompose and relax each nonconvex constraint to two second order cone (SOC) constraints and then linearize the products of the SOC constraints and linear constraints to construct some effective new valid c…
This paper develops new semidefinite programming (SDP) relaxation techniques for two classes of mixed binary quadratically constrained quadratic programs (MBQCQP) and analyzes their approximation performance. The first class of problem finds two minimum norm vectors in $N$-dimensional real or complex Euclidean space, such that $M$ out of $2M$ concave quadratic functions are satisfied. By employing a special randomized rounding procedure, we show that the ratio between the norm of the optimal solution of this model and its SDP relaxation is upper bounded by $\frac{54M^2}π$ in the real case and …
We prove that a "first-order" Sequential Quadratic Programming (SQP) algorithm for equality constrained optimization has local linear convergence with rate $(1-1/κ_R)^k$, where $κ_R$ is the condition number of the Riemannian Hessian, and global convergence with rate $k^{-1/4}$. Our analysis builds on insights from Riemannian optimization -- we show that the SQP and Riemannian gradient methods have nearly identical behavior near the constraint manifold, which could be of broader interest for understanding constrained optimization.
We propose a feasible active set method for convex quadratic programming problems with non-negativity constraints. This method is specifically designed to be embedded into a branch-and-bound algorithm for convex quadratic mixed integer programming problems. The branch-and-bound algorithm generalizes the approach for unconstrained convex quadratic integer programming proposed by Buchheim, Caprara and Lodi to the presence of linear constraints. The main feature of the latter approach consists in a sophisticated preprocessing phase, leading to a fast enumeration of the branch-and-bound nodes. Mor…
For nonconvex quadratically constrained quadratic programs (QCQPs), we first show that, under certain feasibility conditions, the standard semidefinite (SDP) relaxation is exact for QCQPs with bipartite graph structures. The exact optimal solutions are obtained by examining the dual SDP relaxation and the rank of the optimal solution of this dual SDP relaxation under strong duality. Our results on the QCQPs generalize the results on QCQP with sign-definite bipartite graph structures, QCQPs with forest structures, and QCQPs with nonpositive off-diagonal data elements. Second, we propose a conve…
We study the exactness of the semidefinite programming (SDP) relaxation of quadratically constrained quadratic programs (QCQPs). With the aggregate sparsity matrix from the data matrices of a QCQP with $n$ variables, the rank and positive semidefiniteness of the matrix are examined. We prove that if the rank of the aggregate sparsity matrix is not less than $n-1$ and the matrix remains positive semidefinite after replacing some off-diagonal nonzero elements with zeros, then the standard SDP relaxation provides an exact optimal solution for the QCQP under feasibility assumptions. In particular,…
