We show that a class of semidefinite programs (SDP) admits a solution that is a positive semidefinite matrix of rank at most $r$, where $r$ is the rank of the matrix involved in the objective function of the SDP. The optimization problems of this class are semidefinite packing problems, which are the SDP analogs to vector packing problems. Of particular interest is the case in which our result guarantees the existence of a solution of rank one: we show that the computation of this solution actually reduces to a Second Order Cone Program (SOCP). We point out an application in statistics, in the optimal design of experiments.
This paper presents various worst-case results on the positive semidefinite (psd) rank of a nonnegative matrix, primarily in the context of polytopes. We prove that the psd rank of a generic n-dimensional polytope with v vertices is at least (nv)^(1/4) improving on previous lower bounds. For polygons with v vertices, we show that psd rank cannot exceed 4ceil(v/6) which in turn shows that the psd rank of a p by q matrix of rank three is at most 4ceil(min{p,q}/6). In general, a nonnegative matrix of rank (k+1 choose 2) has psd rank at least k and we pose the problem of deciding whether the psd r…
Let M be a p-by-q matrix with nonnegative entries. The positive semidefinite rank (psd rank) of M is the smallest integer k for which there exist positive semidefinite matrices $A_i, B_j$ of size $k \times k$ such that $M_{ij} = \text{trace}(A_i B_j)$. The psd rank has many appealing geometric interpretations, including semidefinite representations of polyhedra and information-theoretic applications. In this paper we develop and survey the main mathematical properties of psd rank, including its geometry, relationships with other rank notions, and computational and algorithmic aspects.
We consider a new semidefinite programming (SDP) relaxation of the symmetric traveling salesman problem (TSP) that may be obtained via an SDP relaxation of the more general quadratic assignment problem (QAP). We show that the new relaxation dominates the one in [D. Cvetkovic, M. Cangalovic, and V. Kovacevic-Vujcic, Semidefinite programming methods for the symmetric traveling salesman problem, in Proc. 7th Int. IPCO Conference, Springer, London, 1999, pp. 126--136]. Unlike the bound of Cvetkovic et al., the new SDP bound is not dominated by the Held-Karp linear programming bound, or vice versa.
The positive semidefinite rank of a nonnegative $(m\times n)$-matrix~$S$ is the minimum number~$q$ such that there exist positive semidefinite $(q\times q)$-matrices $A_1,\dots,A_m$, $B_1,\dots,B_n$ such that $S(k,\ell) = \mbox{tr}(A_k^* B_\ell)$. The most important, lower bound technique for nonnegative rank is solely based on the support of the matrix S, i.e., its zero/non-zero pattern. In this paper, we characterize the power of lower bounds on positive semidefinite rank based on solely on the support.
This paper is a tutorial in a general and explicit procedure to simplify semidefinite programs which are invariant under the action of a symmetry group. The procedure is based on basic notions of representation theory of finite groups. As an example we derive the block diagonalization of the Terwilliger algebra of the binary Hamming scheme in this framework. Here its connection to the orthogonal Hahn and Krawtchouk polynomials becomes visible.
We study the closure of the projection of the (nonconvex) cone of rank restricted positive semidefinite matrices onto subsets of the matrix entries. This defines the feasible sets for semidefinite completion problems with restrictions on the ranks. Applications include conditions for low-rank completions using the nuclear norm heuristic.
Disjointly constrained multilinear programming concerns the problem of maximizing a multilinear function on the product of finitely many disjoint polyhedra. While maximizing a linear function on a polytope (linear programming) is known to be solvable in polynomial time, even bilinear programming is NP-hard. Based on a reformulation of the problem in terms of sum-of-squares polynomials, we study a hierarchy of semidefinite relaxations to the problem. It follows from the general theory that the sequence of optimal values converges asymptotically to the optimal value of the multilinear program. W…
Semidefinite programming (SDP) is a powerful framework from convex optimization that has striking potential for data science applications. This paper develops a provably correct randomized algorithm for solving large, weakly constrained SDP problems by economizing on the storage and arithmetic costs. Numerical evidence shows that the method is effective for a range of applications, including relaxations of MaxCut, abstract phase retrieval, and quadratic assignment. Running on a laptop equivalent, the algorithm can handle SDP instances where the matrix variable has over $10^{14}$ entries.
In the last years many results in the area of semidefinite programming were obtained for invariant (finite dimensional, or infinite dimensional) semidefinite programs - SDPs which have symmetry. This was done for a variety of problems and applications. The purpose of this handbook chapter is to give the reader the necessary background for dealing with semidefinite programs which have symmetry. Here the basic theory is given and it is illustrated in applications from coding theory, combinatorics, geometry, and polynomial optimization.
A real symmetric matrix $M$ is completely positive semidefinite if it admits a Gram representation by (Hermitian) positive semidefinite matrices of any size $d$. The smallest such $d$ is called the (complex) completely positive semidefinite rank of $M$, and it is an open question whether there exists an upper bound on this number as a function of the matrix size. We construct completely positive semidefinite matrices of size $4k^2+2k+2$ with complex completely positive semidefinite rank $2^k$ for any positive integer $k$. This shows that if such an upper bound exists, it has to be at least exp…
In this paper, we propose a new sequential quadratic semidefinite programming (SQSDP) method for solving degenerate nonlinear semidefinite programs (NSDPs), in which we produce iteration points by solving a sequence of stabilized quadratic semidefinite programming (QSDP) subproblems, which we derive from the minimax problem associated with the NSDP. Unlike the existing SQSDP methods, the proposed one allows us to solve those QSDP subproblems inexactly, and each QSDP is feasible. One more remarkable point of the proposed method is that constraint qualifications (CQs) or boundedness of Lagrange …
The nonnegative and positive semidefinite (PSD-) ranks are closely connected to the nonnegative and positive semidefinite extension complexities of a polytope, which are the minimal dimensions of linear and SDP programs which represent this polytope. Though some exponential lower bounds on the nonnegative and PSD- ranks has recently been proved for the slack matrices of some particular polytopes, there are still no tight bounds for these quantities. We explore some existing bounds on the PSD-rank and prove that they cannot give exponential lower bounds on the extension complexity. Our approach…
Lecture notes for the tutorial at the workshop HPOPT 2008 - 10th International Workshop on High Performance Optimization Techniques (Algebraic Structure in Semidefinite Programming), June 11th to 13th, 2008, Tilburg University, The Netherlands.
In this paper, we investigate the generalized low rank approximation to the symmetric positive semidefinite matrix in the Frobenius norm: $$\underset{ rank(X)\leq k}{\min} \sum^m_{i=1}\left \Vert A_i - B_i XB_i^T \right \Vert^2_F,$$ where $X$ is an unknown symmetric positive semidefinite matrix and $k$ is a positive integer. We firstly use the property of a symmetric positive semidefinite matrix $X=YY^T$, $Y$ with order $n\times k$, to convert the generalized low rank approximation into unconstraint generalized optimization problem. Then we apply the nonlinear conjugate gradient method to solv…
We propose a method for low-rank semidefinite programming in application to the semidefinite relaxation of unconstrained binary quadratic problems. The method improves an existing solution of the semidefinite programming relaxation to achieve a lower rank solution. This procedure is computationally efficient as it does not require projecting on the cone of positive-semidefinite matrices. Its performance in terms of objective improvement and rank reduction is tested over multiple graphs of large-scale Gset graph collection and over binary optimization problems from the Biq Mac collection.
Superlinear and quadratic convergence of a stabilized sequential quadratic semidefinite programming method for nonlinear semidefinite programming problems
Given a nonnegative matrix M with rational entries, we consider two quantities: the usual positive semidefinite (psd) rank, where the matrix is factored through the cone of real symmetric psd matrices, and the rational-restricted psd rank, where the matrix factors are required to be rational symmetric psd matrices. It is clear that the rational-restricted psd rank is always an upper bound to the usual psd rank. We show that this inequality may be strict by exhibiting a matrix with psd rank four whose rational-restricted psd rank is strictly greater than four.
The positive semidefinite (psd) rank of a polytope is the smallest $k$ for which the cone of $k \times k$ real symmetric psd matrices admits an affine slice that projects onto the polytope. In this paper we show that the psd rank of a polytope is at least the dimension of the polytope plus one, and we characterize those polytopes whose psd rank equals this lower bound. We give several classes of polytopes that achieve the minimum possible psd rank including a complete characterization in dimensions two and three.
Binary Integer Programming (BIP) problems are of interest due in part to the difficulty they pose and because of their various applications, including those in graph theory, combinatorial optimization and network optimization. In this note, we explicitly state the Lovasz-Schrijver Semidefinite Programming (SDP) relaxation (in primal-standard form) for a BIP problem, a relaxation that yields a tighter upper-bound than the canonical Linear Programming relaxation.
The set of matrices of given positive semidefinite rank is semialgebraic. In this paper we study the geometry of this set, and in small cases we describe its boundary. For general values of positive semidefinite rank we provide a conjecture for the description of this boundary. Our proof techniques are geometric in nature and rely on nesting spectrahedra between polytopes.
This paper focuses on the study of a mathematical program with equilibrium constraints, where the objective and the constraint functions are all polynomials. We present a method for finding its global minimizers and global minimum using a hierarchy of semidefinite programming (SDP) relaxations and prove the convergence result for the method. Numerical experiments are presented to show the efficiency of the proposed algorithm.
Grothendieck inequalities are fundamental inequalities which are frequently used in many areas of mathematics and computer science. They can be interpreted as upper bounds for the integrality gap between two optimization problems: a difficult semidefinite program with rank-1 constraint and its easy semidefinite relaxation where the rank constrained is dropped. For instance, the integrality gap of the Goemans-Williamson approximation algorithm for MAX CUT can be seen as a Grothendieck inequality. In this paper we consider Grothendieck inequalities for ranks greater than 1 and we give two applic…
To address difficult optimization problems, convex relaxations based on semidefinite programming are now common place in many fields. Although solvable in polynomial time, large semidefinite programs tend to be computationally challenging. Over a decade ago, exploiting the fact that in many applications of interest the desired solutions are low rank, Burer and Monteiro proposed a heuristic to solve such semidefinite programs by restricting the search space to low-rank matrices. The accompanying theory does not explain the extent of the empirical success. We focus on Synchronization and Communi…
The completely bounded trace and spectral norms, for finite-dimensional spaces, are known to be efficiently expressible by semidefinite programs (J. Watrous, Theory of Computing 5: 11, 2009). This paper presents two new, and arguably much simpler, semidefinite programming formulations of these norms.
Semidefinite programming can be considered over any real closed field, including fields of Puiseux series equipped with their nonarchimedean valuation. Nonarchimedean semidefinite programs encode parametric families of classical semidefinite programs, for sufficiently large values of the parameter. Recently, a correspondence has been established between nonarchimedean semidefinite programs and stochastic mean payoff games with perfect information. This correspondence relies on tropical geometry. It allows one to solve generic nonarchimedean semidefinite feasibility problems, of large scale, by…
We propose a homogeneous primal-dual interior-point method to solve sum-of-squares optimization problems by combining non-symmetric conic optimization techniques and polynomial interpolation. The approach optimizes directly over the sum-of-squares cone and its dual, circumventing the semidefinite programming (SDP) reformulation which requires a large number of auxiliary variables. As a result, it has substantially lower theoretical time and space complexity than the conventional SDP-based approach. Although our approach avoids the semidefinite programming reformulation, an optimal solution to …
We present an algorithm for approximating semidefinite programs with running time that is sublinear in the number of entries in the semidefinite instance. We also present lower bounds that show our algorithm to have a nearly optimal running time.
The T-product for third-order tensors has been used extensively in the literature. In this paper, we first introduce the first-order and second-order T-derivatives for the multi-vector real-valued function with the tensor T-product; and inspired by an equivalent characterization of a twice continuously T-differentiable multi-vector real-valued function being convex, we present a definition of the T-positive semidefiniteness of third-order symmetric tensors. After that, we extend many properties of positive semidefinite matrices to the case of third-order symmetric tensors. In particular, analo…
We consider the decision problem asking whether a partial rational symmetric matrix with an all-ones diagonal can be completed to a full positive semidefinite matrix of rank at most $k$. We show that this problem is $\NP$-hard for any fixed integer $k\ge 2$. Equivalently, for $k\ge 2$, it is $\NP$-hard to test membership in the rank constrained elliptope $\EE_k(G)$, i.e., the set of all partial matrices with off-diagonal entries specified at the edges of $G$, that can be completed to a positive semidefinite matrix of rank at most $k$. Additionally, we show that deciding membership in the conve…
We consider sensitivity of a semidefinite program under perturbations in the case that the primal problem is strictly feasible and the dual problem is weakly feasible. When the coefficient matrices are perturbed, the optimal values can change discontinuously as explained in concrete examples. We show that the optimal value of such a semidefinite program changes continuously under conditions involving the behavior of the minimal faces of the perturbed dual problems. In addition, we determine what kinds of perturbations keep the minimal faces invariant, by using the reducing certificates, which …
This paper presents generalizations of semidefinite programming formulations of 1-norm optimization problems over infinite dictionaries of vectors of complex exponentials, which were recently proposed for superresolution, gridless compressed sensing, and other applications in signal processing. Results related to the generalized Kalman-Yakubovich-Popov lemma in linear system theory provide simple, constructive proofs of the semidefinite representations of the penalty functions used in these applications. The connection leads to several extensions to gauge functions and atomic norms for sets of…
This paper studies a class of so-called linear semi-infinite polynomial programming (LSIPP) problems. It is a subclass of linear semi-infinite programming problems whose constraint functions are polynomials in parameters and index sets are basic semialgebraic sets. We present a hierarchy of semidefinite programming (SDP) relaxations for LSIPP problems. Convergence rate analysis of the SDP relaxations is established based on some existing results. We show how to verify the compactness of feasible sets of LSIPP problems. In the end, we extend the SDP relaxation method to more general semi-infini…
Lovasz and Schrijver, and later Lasserre, proposed hierarchies of semidefinite programming relaxations for general 0/1 linear programming problems. In this paper these two constructions are revisited and two new, block-diagonal hierarchies are proposed. They have the advantage of being computationally less costly while being at least as strong as the Lovasz-Schrijver hierarchy. Our construction is applied to the stable set problem and experimental results for Paley graphs are reported.
A bilevel program is an optimization problem whose constraints involve another optimization problem. This paper studies bilevel polynomial programs (BPPs), i.e., all the functions are polynomials. We reformulate BPPs equivalently as semi-infinite polynomial programs (SIPPs), using Fritz John conditions and Jacobian representations. Combining the exchange technique and Lasserre type semidefinite relaxations, we propose numerical methods for solving both simple and general BPPs. For simple BPPs, we prove the convergence to global optimal solutions. Numerical experiments are presented to show the…
Semidefinite programs (SDPs) often arise in relaxations of some NP-hard problems, and if the solution of the SDP obeys certain rank constraints, the relaxation will be tight. Decomposition methods based on chordal sparsity have already been applied to speed up the solution of sparse SDPs, but methods for dealing with rank constraints are underdeveloped. This paper leverages a minimum rank completion result to decompose the rank constraint on a single large matrix into multiple rank constraints on a set of smaller matrices. The re-weighted heuristic is used as a proxy for rank, and the specific…
The geometric measure of entanglement is a widely used entanglement measure for quantum pure states. The key problem of computation of the geometric measure is to calculate the entanglement eigenvalue, which is equivalent to computing the largest unitary eigenvalue of a corresponding complex tensor. In this paper, we propose a Jacobian semidefinite programming relaxation method to calculate the largest unitary eigenvalue of a complex tensor. For this, we first introduce the Jacobian semidefinite programming relaxation method for a polynomial optimization with equality constraint, and then conv…
In a common formulation of semi-infinite programs, the infinite constraint set is a requirement that a function parametrized by the decision variables is nonnegative over an interval. If this function is sufficiently closely approximable by a polynomial or a rational function, then the semi-infinite program can be reformulated as an equivalent semidefinite program. Solving this semidefinite program is challenging if the polynomials involved are of high degree, due to numerical difficulties and bad scaling arising both from the polynomial approximations and from the fact that the semidefinite p…
Consider a semidefinite program (SDP) involving an $n\times n$ positive semidefinite matrix $X$. The Burer-Monteiro method uses the substitution $X=Y Y^T$ to obtain a nonconvex optimization problem in terms of an $n\times p$ matrix $Y$. Boumal et al. showed that this nonconvex method provably solves equality-constrained SDPs with a generic cost matrix when $p \gtrsim \sqrt{2m}$, where $m$ is the number of constraints. In this note we extend their result to arbitrary SDPs, possibly involving inequalities or multiple semidefinite constraints. We derive similar guarantees for a fixed cost matrix …
Farkas' lemma is a fundamental result from linear programming providing linear certificates for infeasibility of systems of linear inequalities. In semidefinite programming, such linear certificates only exist for strongly infeasible linear matrix inequalities. We provide nonlinear algebraic certificates for all infeasible linear matrix inequalities in the spirit of real algebraic geometry: A linear matrix inequality is infeasible if and only if -1 lies in the quadratic module associated to it. We also present a new exact duality theory for semidefinite programming, motivated by the real radic…
We introduce a notion of matrix valued Gram decompositions for correlation matrices whose study is motivated by quantum information theory. We show that for extremal correlations, the matrices in such a factorization generate a Clifford algebra and thus, their size is exponential in terms of the rank of the correlation matrix. Using this we give a self-contained and succinct proof of the existence of completely positive semidefinite matrices with sub-exponential cpsd-rank, recently derived in the literature. This fact also underlies and generalizes Tsirelson's seminal lower bound on the local …
This paper presents a method to approximately solve stochastic optimal control problems in which the cost function and the system dynamics are polynomial. For stochastic systems with polynomial dynamics, the moments of the state can be expressed as a, possibly infinite, system of deterministic linear ordinary differential equations. By casting the problem as a deterministic control problem in moment space, semidefinite programming is used to find a lower bound on the optimal solution. The constraints in the semidefinite program are imposed by the ordinary differential equations for moment dyna…
This paper studies how to solve semi-infinite polynomial programming (SIPP) problems by semidefinite relaxation method. We first introduce two SDP relaxation methods for solving polynomial optimization problems with finitely many constraints. Then we propose an exchange algorithm with SDP relaxations to solve SIPP problems with compact index set. At last, we extend the proposed method to SIPP problems with noncompact index set via homogenization. Numerical results show that the algorithm is efficient in practice.
In the context of augmented Lagrangian approaches for solving semidefinite programming problems, we investigate the possibility of eliminating the positive semidefinite constraint on the dual matrix by employing a factorization. Hints on how to deal with the resulting unconstrained maximization of the augmented Lagrangian are given. We further use the approximate maximum of the augmented Lagrangian with the aim of improving the convergence rate of alternating direction augmented Lagrangian frameworks. Numerical results are reported, showing the benefits of the approach.
Since more than three decades, interior-point methods proved very useful for optimization, from linear over semidefinite to conic (and partly beyond non-convex) programming; despite the fact that already in the semidefinite case (even when strong duality holds) "hard" problems are known. We shade a light on a rather surprising restriction in the non-commutative world (of semidefinite programming), namely "commutative" paths and propose a new family of solvers that is able to use the full richness of "non-commutative" search directions: (primal) feasible-interior-point methods. Beside a detaile…
We present new constraint qualification conditions for nonlinear semidefinite programming that extend some of the constant rank-type conditions from nonlinear programming. As an application of these conditions, we provide a unified global convergence proof of a class of algorithms to stationary points without assuming neither uniqueness of the Lagrange multiplier nor boundedness of the Lagrange multipliers set. This class of algorithm includes, for instance, general forms of augmented Lagrangian, sequential quadratic programming, and interior point methods. We also compare these new conditions…
The alternating-current optimal power flow (ACOPF) is one of the best known non-convex non-linear optimisation problems. We present a novel re-formulation of ACOPF, which is based on lifting the rectangular power-voltage rank-constrained formulation, and makes it possible to derive alternative SDP relaxations. For those, we develop a first-order method based on the parallel coordinate descent with a novel closed-form step based on roots of cubic polynomials.
The positive semidefinite rank of a convex body $C$ is the size of its smallest positive semidefinite formulation. We show that the positive semidefinite rank of any convex body $C$ is at least $\sqrt{\log d}$ where $d$ is the smallest degree of a polynomial that vanishes on the boundary of the polar of $C$. This improves on the existing bound which relies on results from quantifier elimination. The proof relies on the Bézout bound applied to the Karush-Kuhn-Tucker conditions of optimality. We discuss the connection with the algebraic degree of semidefinite programming and show that the bound …
We propose a very simple preprocessing algorithm for semidefinite programming. Our algorithm inspects the constraints of the problem, deletes redundant rows and columns in the constraints, and reduces the size of the variable matrix. It often detects infeasibility. Our algorithm does not rely on any optimization solver: the only subroutine it needs is Cholesky factorization, hence it can be implemented with a few lines of code in machine precision. We present computational results on a set of problems arising mostly from polynomial optimization.
In semidefinite programming (SDP), unlike in linear programming, Farkas' lemma may fail to prove infeasibility. Here we obtain an exact, short certificate of infeasibility in SDP by an elementary approach: we reformulate any semidefinite system of the form Ai*X = bi (i=1,...,m) (P) X >= 0 using only elementary row operations, and rotations. When (P) is infeasible, the reformulated system is trivially infeasible. When (P) is feasible, the reformulated system has strong duality with its Lagrange dual for all objective functions. As a corollary, we obtain algorithms to generate the constraints of…
In this note, we use a natural desingularization of the conormal variety of the variety of n x n symmetric matrices of rank at most r to find a general formula for the algebraic degree in semidefinite programming.
Hydro unit commitment is the problem of maximizing water use efficiency while minimizing start-up costs in the daily operation of multiple hydro plants, subject to constraints on short-term reservoir operation, and long-term goals. A low-rank rounding heuristic is presented for the semidefinite relaxation of the mixed-integer quadratic-constrained formulation of this problem. In addition to limits on reservoir and generator operation, transmission constraints are represented by an approximate AC power flow model. In our proposed method, the mathematical program is equivalently formulated as a …
Exploring chordal sparsity in semidefinite programming with sparse plus low-rank data matrices
Institute of Operations Research and Analytics, National University of Singapore（新加坡国立大学运筹学与分析研究所）；Department of Mathematics, and Institute of Operations Research and Analytics, National University of Singapore（新加坡国立大学数学系及运筹学与分析研究所）
In contrast with many other convex optimization classes, state-of-the-art semidefinite programming solvers are yet unable to efficiently solve large scale instances. This work aims to reduce this scalability gap by proposing a novel proximal algorithm for solving general semidefinite programming problems. The proposed methodology, based on the primal-dual hybrid gradient method, allows the presence of linear inequalities without the need of adding extra slack variables and avoids solving a linear system at each iteration. More importantly, it does simultaneously compute the dual variables asso…
