In this paper, we give a new penalized semidefinite programming approach for non-convex quadratically-constrained quadratic programs (QCQPs). We incorporate penalty terms into the objective of convex relaxations in order to retrieve feasible and near-optimal solutions for non-convex QCQPs. We introduce a generalized linear independence constraint qualification (GLICQ) criterion and prove that any GLICQ regular point that is sufficiently close to the feasible set can be used to construct an appropriate penalty term and recover a feasible solution. Inspired by these results, we develop a heuristic sequential procedure that preserves feasibility and aims to improve the objective value at each iteration. Numerical experiments on large-scale system identification problems as well as benchmark instances from the library of quadratic programming (QPLIB) demonstrate the ability of the proposed penalized semidefinite programs in finding near-optimal solutions for non-convex QCQP.
Low-rank methods for semidefinite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-based or Schatten-norm penalties, which are hard to implement in practice due to high computational efforts. In this paper, we propose Entropy Penalized Semi-definite programming (EP-SDP) which provides a unified framework for a wide class of penalty functions used in practice to promote a low-rank solution. We show that EP-SDP problems admit efficient numerical algorithm having (almost) linear time complexity of the …
We consider the efficient minimization of a nonlinear, strictly convex functional with $\ell_1$-penalty term. Such minimization problems appear in a wide range of applications like Tikhonov regularization of (non)linear inverse problems with sparsity constraints. In (2015 Inverse Problems (31) 025005), a globalized Bouligand-semismooth Newton method was presented for $\ell_1$-Tikhonov regularization of linear inverse problems. Nevertheless, a technical assumption on the accumulation point of the sequence of iterates was necessary to prove global convergence. Here, we generalize this method to …
A quadratically convergent semismooth Newton method for nonlinear semidefinite programming without generalized Jacobian regularity
Institute of Applied Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院应用数学研究所）；School of Mathematical Sciences, University of Chinese Academy of Science（中国科学院大学数学科学学院）；School of Data Science, Fudan University（复旦大学数据科学系）
This paper examines the nonconvex quadratically constrained quadratic programming (QCQP) problems using an iterative method. One of the existing approaches for solving nonconvex QCQP problems relaxes the rank one constraint on the unknown matrix into semidefinite constraint to obtain the bound on the optimal value without finding the exact solution. By reconsidering the rank one matrix, an iterative rank minimization (IRM) method is proposed to gradually approach the rank one constraint. Each iteration of IRM is formulated as a convex problem with semidefinite constraints. An augmented Lagrang…
We study the exactness of the semidefinite programming (SDP) relaxation of quadratically constrained quadratic programs (QCQPs). With the aggregate sparsity matrix from the data matrices of a QCQP with $n$ variables, the rank and positive semidefiniteness of the matrix are examined. We prove that if the rank of the aggregate sparsity matrix is not less than $n-1$ and the matrix remains positive semidefinite after replacing some off-diagonal nonzero elements with zeros, then the standard SDP relaxation provides an exact optimal solution for the QCQP under feasibility assumptions. In particular,…
University of Texas Arlington（德克萨斯大学阿灵顿分校）；Cognitiv；University of California Berkeley（加州大学伯克利分校）
In the first part of this work [32], we introduce a convex parabolic relaxation for quadratically-constrained quadratic programs, along with a sequential penalized parabolic relaxation algorithm to recover near-optimal feasible solutions. In this second part, we show that starting from a feasible solution or a near-feasible solution satisfying certain regularity conditions, the sequential penalized parabolic relaxation algorithm convergences to a point which satisfies Karush-Kuhn-Tucker optimality conditions. Next, we present numerical experiments on benchmark non-convex QCQP problems as well …
This paper is concerned with optimal power flow (OPF), which is the problem of optimizing the transmission of electricity in power systems. Our main contributions are as follows: (i) we propose a novel parabolic relaxation, which transforms non-convex OPF problems into convex quadratically-constrained quadratic programs (QCQPs) and can serve as an alternative to the common practice semidefinite programming (SDP) and second-order cone programming (SOCP) relaxations, (ii) we propose a penalization technique which is compatible with the SDP, SOCP, and parabolic relaxations and guarantees the reco…
We study a class of quadratically constrained quadratic programs (QCQPs), called {\em diagonal QCQPs\/}, which contain no off-diagonal terms $x_j x_k$ for $j \ne k$, and we provide a sufficient condition on the problem data guaranteeing that the basic Shor semidefinite relaxation is exact. Our condition complements and refines those already present in the literature and can be checked in polynomial time. We then extend our analysis from diagonal QCQPs to general QCQPs, i.e., ones with no particular structure. By reformulating a general QCQP into diagonal form, we establish new, polynomial-time…
We show that (i) any constrained polynomial optimization problem (POP) has an equivalent formulation on a variety contained in an Euclidean sphere and (ii) the resulting semidefinite relaxations in the moment-SOS hierarchy have the constant trace property (CTP) for the involved matrices. We then exploit the CTP to avoid solving the semidefinite relaxations via interior-point methods and rather use ad-hoc spectral methods that minimize the largest eigenvalue of a matrix pencil. Convergence to the optimal value of the semidefinite relaxation is guaranteed. As a result we obtain a hierarchy of no…
We prove that every semidefinite moment relaxation of a polynomial optimization problem (POP) with a ball constraint can be reformulated as a semidefinite program involving a matrix with constant trace property (CTP). As a result such moment relaxations can be solved efficiently by first-order methods that exploit CTP, e.g., the conditional gradient-based augmented Lagrangian method. We also extend this CTP-exploiting framework to large-scale POPs with different sparsity structures. The efficiency and scalability of our framework are illustrated on second-order moment relaxations for various r…
University of Texas, Arlington（德克萨斯大学阿灵顿分校）；Cognitiv；University of California, Berkeley（加州大学伯克利分校）
For general quadratically-constrained quadratic programming (QCQP), we propose a parabolic relaxation described with convex quadratic constraints. An interesting property of the parabolic relaxation is that the original non-convex feasible set is contained on the boundary of the parabolic relaxation. Under certain assumptions, this property enables one to recover near-optimal feasible points via objective penalization. Moreover, through an appropriate change of coordinates that requires a one-time computation of an optimal basis, the easier-to-solve parabolic relaxation can be made as strong a…
Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the penalty formulation of appropriately rank-constrained SDPs as long as the number of constraints scales sub-quadratically with the desired rank of the optimal solution. Our result is based on a simple penalty function formulation of the rank-constrained SDP along with a smoothed ana…
We develop a spatial branch-and-cut approach for nonconvex Quadratically Constrained Quadratic Programs with bounded complex variables (CQCQP). Linear valid inequalities are added at each node of the search tree to strengthen semidefinite programming relaxations of CQCQP. These valid inequalities are derived from the convex hull description of a nonconvex set of $2 \times 2$ positive semidefinite Hermitian matrices subject to a rank-one constraint. We propose branching rules based on an alternative to the rank-one constraint that allows for local measurement of constraint violation. Closed-for…
This two-part paper is concerned with the problem of minimizing a linear objective function subject to a bilinear matrix inequality (BMI) constraint. In this part, we first consider a family of convex relaxations which transform BMI optimization problems into polynomial-time solvable surrogates. As an alternative to the state-of-the-art semidefinite programming (SDP) and second-order cone programming (SOCP) relaxations, a computationally efficient parabolic relaxation is developed, which relies on convex quadratic constraints only. Next, we developed a family of penalty functions, which can be…
Common computational problems, such as parameter estimation in dynamic models and PDE constrained optimization, require data fitting over a set of auxiliary parameters subject to physical constraints over an underlying state. Naive quadratically penalized formulations, commonly used in practice, suffer from inherent ill-conditioning. We show that surprisingly the partial minimization technique regularizes the problem, making it well-conditioned. This viewpoint sheds new light on variable projection techniques, as well as the penalty method for PDE constrained optimization, and motivates robust…
Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study conditions under which the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then using this framework, we show that the SDP relaxation is tight whenever the quadratic eigenvalue multiplicity, a parameter capturing the amount of symmetry present in a given problem, is large enough. We present similar sufficient conditions under which the…
Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study sufficient conditions for a convex hull result that immediately implies that the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then using this framework, we show that the convex hull result holds whenever the quadratic eigenvalue multiplicity, a parameter capturing the amount of symmetry present in a given problem, is large enough. O…
For nonconvex quadratically constrained quadratic programs (QCQPs), we first show that, under certain feasibility conditions, the standard semidefinite (SDP) relaxation is exact for QCQPs with bipartite graph structures. The exact optimal solutions are obtained by examining the dual SDP relaxation and the rank of the optimal solution of this dual SDP relaxation under strong duality. Our results on the QCQPs generalize the results on QCQP with sign-definite bipartite graph structures, QCQPs with forest structures, and QCQPs with nonpositive off-diagonal data elements. Second, we propose a conve…
Solving Sparse Mixed-Integer Quadratic Problems: Application to the Unit Commitment Problem with Optimal Power Flow
University of Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）；CITMAga (Galician Center for Mathematical Research and Technology)（加利西亚数学研究与技术中心）；Politecnico di Milano（米兰理工大学）；Ivey Business School, Western University（韦仕敦大学艾维商学院）；IE University（IE大学）
Purdue University（普渡大学）；University of Melbourne（墨尔本大学）；Massachusetts Institute of Technology（麻省理工学院）；UC Berkeley（加州大学伯克利分校）；Rice University（莱斯大学）；Carnegie Mellon University（卡内基梅隆大学）
Semidefinite programming (SDP) is a unifying framework that generalizes both linear programming and quadratically-constrained quadratic programming, while also yielding efficient solvers, both in theory and in practice. However, there exist known impossibility results for approximating the optimal solution when constraints for covering SDPs arrive in an online fashion. In this paper, we study online covering linear and semidefinite programs in which the algorithm is augmented with advice from a possibly erroneous predictor. We show that if the predictor is accurate, we can efficiently bypass t…
Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems. In a QCQP, we are asked to minimize a (possibly nonconvex) quadratic function subject to a number of (possibly nonconvex) quadratic constraints. Such problems arise naturally in many areas of operations research, computer science, and engineering. Although QCQPs are NP-hard to solve in general, they admit a natural convex relaxation via the standard (Shor) semidefinite program (SDP) relaxation. In this tutorial, we will study the SDP relaxation for general QCQPs, present various exactness con…
Quadratically constrained quadratic programs (QCQPs) are a highly expressive class of nonconvex optimization problems. While QCQPs are NP-hard in general, they admit a natural convex relaxation via the standard (Shor) semidefinite program (SDP) relaxation. Towards understanding when this relaxation is exact, we study general QCQPs and their (projected) SDP relaxations. We present sufficient (and in some cases, also necessary) conditions for objective value exactness (the condition that the objective values of the QCQP and its SDP relaxation coincide) and convex hull exactness (the condition th…
Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection
Department of Mathematics, Washington State University（数学系，华盛顿州立大学）；Department of Statistics, University of Connecticut（统计系，康涅狄格大学）；Department of Industrial and Systems Engineering, University of Wisconsin-Madison（工业与系统工程系，威斯康星大学麦迪逊分校）
