The ground state energy of a many-electron system can be approximated by an variational approach in which the total energy of the system is minimized with respect to one and two-body reduced density matrices (RDM) instead of many-electron wavefunctions. This problem can be formulated as a semidefinite programming problem. Due the large size of the problem, the well-known interior point method can only be used to tackle problems with a few atoms. First-order methods such as the the alternating direction method of multipliers (ADMM) have much lower computational cost per iteration. However, their convergence can be slow, especially for obtaining highly accurate approximations. In this paper, we present a practical and efficient second-order semi-smooth Newton type method for solving the SDP formulation of the energy minimization problem. We discuss a number of techniques that can be used to improve the computational efficiency of the method and achieve global convergence. Extensive numerical experiments show that our approach is competitive to the state-of-the-art methods in terms of both accuracy and speed.
The completely bounded trace and spectral norms, for finite-dimensional spaces, are known to be efficiently expressible by semidefinite programs (J. Watrous, Theory of Computing 5: 11, 2009). This paper presents two new, and arguably much simpler, semidefinite programming formulations of these norms.
We provide new tools for worst-case performance analysis of the gradient (or steepest descent) method of Cauchy for smooth strongly convex functions, and Newton's method for self-concordant functions, including the case of inexact search directions. The analysis uses semidefinite programming performance estimation, as pioneered by Drori and Teboulle [Mathematical Programming, 145(1-2):451-482, 2014], and extends recent performance estimation results for the method of Cauchy by the authors [Optimization Letters, 11(7), 1185-1199, 2017]. To illustrate the applicability of the tools, we demonstra…
In this paper, we propose a new sequential quadratic semidefinite programming (SQSDP) method for solving degenerate nonlinear semidefinite programs (NSDPs), in which we produce iteration points by solving a sequence of stabilized quadratic semidefinite programming (QSDP) subproblems, which we derive from the minimax problem associated with the NSDP. Unlike the existing SQSDP methods, the proposed one allows us to solve those QSDP subproblems inexactly, and each QSDP is feasible. One more remarkable point of the proposed method is that constraint qualifications (CQs) or boundedness of Lagrange …
Lecture notes for the tutorial at the workshop HPOPT 2008 - 10th International Workshop on High Performance Optimization Techniques (Algebraic Structure in Semidefinite Programming), June 11th to 13th, 2008, Tilburg University, The Netherlands.
Superlinear and quadratic convergence of a stabilized sequential quadratic semidefinite programming method for nonlinear semidefinite programming problems
Binary Integer Programming (BIP) problems are of interest due in part to the difficulty they pose and because of their various applications, including those in graph theory, combinatorial optimization and network optimization. In this note, we explicitly state the Lovasz-Schrijver Semidefinite Programming (SDP) relaxation (in primal-standard form) for a BIP problem, a relaxation that yields a tighter upper-bound than the canonical Linear Programming relaxation.
We present an algorithm for approximating semidefinite programs with running time that is sublinear in the number of entries in the semidefinite instance. We also present lower bounds that show our algorithm to have a nearly optimal running time.
This paper focuses on the study of a mathematical program with equilibrium constraints, where the objective and the constraint functions are all polynomials. We present a method for finding its global minimizers and global minimum using a hierarchy of semidefinite programming (SDP) relaxations and prove the convergence result for the method. Numerical experiments are presented to show the efficiency of the proposed algorithm.
We propose a method for low-rank semidefinite programming in application to the semidefinite relaxation of unconstrained binary quadratic problems. The method improves an existing solution of the semidefinite programming relaxation to achieve a lower rank solution. This procedure is computationally efficient as it does not require projecting on the cone of positive-semidefinite matrices. Its performance in terms of objective improvement and rank reduction is tested over multiple graphs of large-scale Gset graph collection and over binary optimization problems from the Biq Mac collection.
We propose a homogeneous primal-dual interior-point method to solve sum-of-squares optimization problems by combining non-symmetric conic optimization techniques and polynomial interpolation. The approach optimizes directly over the sum-of-squares cone and its dual, circumventing the semidefinite programming (SDP) reformulation which requires a large number of auxiliary variables. As a result, it has substantially lower theoretical time and space complexity than the conventional SDP-based approach. Although our approach avoids the semidefinite programming reformulation, an optimal solution to …
The T-product for third-order tensors has been used extensively in the literature. In this paper, we first introduce the first-order and second-order T-derivatives for the multi-vector real-valued function with the tensor T-product; and inspired by an equivalent characterization of a twice continuously T-differentiable multi-vector real-valued function being convex, we present a definition of the T-positive semidefiniteness of third-order symmetric tensors. After that, we extend many properties of positive semidefinite matrices to the case of third-order symmetric tensors. In particular, analo…
In this paper, we study randomized and cyclic coordinate descent for convex unconstrained optimization problems. We improve the known convergence rates in some cases by using the numerical semidefinite programming performance estimation method. As a spin-off we provide a method to analyse the worst-case performance of the Gauss-Seidel iterative method for linear systems where the coefficient matrix is positive semidefinite with a positive diagonal.
We consider sensitivity of a semidefinite program under perturbations in the case that the primal problem is strictly feasible and the dual problem is weakly feasible. When the coefficient matrices are perturbed, the optimal values can change discontinuously as explained in concrete examples. We show that the optimal value of such a semidefinite program changes continuously under conditions involving the behavior of the minimal faces of the perturbed dual problems. In addition, we determine what kinds of perturbations keep the minimal faces invariant, by using the reducing certificates, which …
This paper studies a class of so-called linear semi-infinite polynomial programming (LSIPP) problems. It is a subclass of linear semi-infinite programming problems whose constraint functions are polynomials in parameters and index sets are basic semialgebraic sets. We present a hierarchy of semidefinite programming (SDP) relaxations for LSIPP problems. Convergence rate analysis of the SDP relaxations is established based on some existing results. We show how to verify the compactness of feasible sets of LSIPP problems. In the end, we extend the SDP relaxation method to more general semi-infini…
Lovasz and Schrijver, and later Lasserre, proposed hierarchies of semidefinite programming relaxations for general 0/1 linear programming problems. In this paper these two constructions are revisited and two new, block-diagonal hierarchies are proposed. They have the advantage of being computationally less costly while being at least as strong as the Lovasz-Schrijver hierarchy. Our construction is applied to the stable set problem and experimental results for Paley graphs are reported.
To address difficult optimization problems, convex relaxations based on semidefinite programming are now common place in many fields. Although solvable in polynomial time, large semidefinite programs tend to be computationally challenging. Over a decade ago, exploiting the fact that in many applications of interest the desired solutions are low rank, Burer and Monteiro proposed a heuristic to solve such semidefinite programs by restricting the search space to low-rank matrices. The accompanying theory does not explain the extent of the empirical success. We focus on Synchronization and Communi…
A bilevel program is an optimization problem whose constraints involve another optimization problem. This paper studies bilevel polynomial programs (BPPs), i.e., all the functions are polynomials. We reformulate BPPs equivalently as semi-infinite polynomial programs (SIPPs), using Fritz John conditions and Jacobian representations. Combining the exchange technique and Lasserre type semidefinite relaxations, we propose numerical methods for solving both simple and general BPPs. For simple BPPs, we prove the convergence to global optimal solutions. Numerical experiments are presented to show the…
The geometric measure of entanglement is a widely used entanglement measure for quantum pure states. The key problem of computation of the geometric measure is to calculate the entanglement eigenvalue, which is equivalent to computing the largest unitary eigenvalue of a corresponding complex tensor. In this paper, we propose a Jacobian semidefinite programming relaxation method to calculate the largest unitary eigenvalue of a complex tensor. For this, we first introduce the Jacobian semidefinite programming relaxation method for a polynomial optimization with equality constraint, and then conv…
In a common formulation of semi-infinite programs, the infinite constraint set is a requirement that a function parametrized by the decision variables is nonnegative over an interval. If this function is sufficiently closely approximable by a polynomial or a rational function, then the semi-infinite program can be reformulated as an equivalent semidefinite program. Solving this semidefinite program is challenging if the polynomials involved are of high degree, due to numerical difficulties and bad scaling arising both from the polynomial approximations and from the fact that the semidefinite p…
Consider a semidefinite program (SDP) involving an $n\times n$ positive semidefinite matrix $X$. The Burer-Monteiro method uses the substitution $X=Y Y^T$ to obtain a nonconvex optimization problem in terms of an $n\times p$ matrix $Y$. Boumal et al. showed that this nonconvex method provably solves equality-constrained SDPs with a generic cost matrix when $p \gtrsim \sqrt{2m}$, where $m$ is the number of constraints. In this note we extend their result to arbitrary SDPs, possibly involving inequalities or multiple semidefinite constraints. We derive similar guarantees for a fixed cost matrix …
This paper presents a method to approximately solve stochastic optimal control problems in which the cost function and the system dynamics are polynomial. For stochastic systems with polynomial dynamics, the moments of the state can be expressed as a, possibly infinite, system of deterministic linear ordinary differential equations. By casting the problem as a deterministic control problem in moment space, semidefinite programming is used to find a lower bound on the optimal solution. The constraints in the semidefinite program are imposed by the ordinary differential equations for moment dyna…
Recently two approximate Newton methods were proposed for the optimisation of Markov Decision Processes. While these methods were shown to have desirable properties, such as a guarantee that the preconditioner is negative-semidefinite when the policy is $\log$-concave with respect to the policy parameters, and were demonstrated to have strong empirical performance in challenging domains, such as the game of Tetris, no convergence analysis was provided. The purpose of this paper is to provide such an analysis. We start by providing a detailed analysis of the Hessian of a Markov Decision Process…
This paper studies how to solve semi-infinite polynomial programming (SIPP) problems by semidefinite relaxation method. We first introduce two SDP relaxation methods for solving polynomial optimization problems with finitely many constraints. Then we propose an exchange algorithm with SDP relaxations to solve SIPP problems with compact index set. At last, we extend the proposed method to SIPP problems with noncompact index set via homogenization. Numerical results show that the algorithm is efficient in practice.
We show that a class of semidefinite programs (SDP) admits a solution that is a positive semidefinite matrix of rank at most $r$, where $r$ is the rank of the matrix involved in the objective function of the SDP. The optimization problems of this class are semidefinite packing problems, which are the SDP analogs to vector packing problems. Of particular interest is the case in which our result guarantees the existence of a solution of rank one: we show that the computation of this solution actually reduces to a Second Order Cone Program (SOCP). We point out an application in statistics, in the…
In the last years many results in the area of semidefinite programming were obtained for invariant (finite dimensional, or infinite dimensional) semidefinite programs - SDPs which have symmetry. This was done for a variety of problems and applications. The purpose of this handbook chapter is to give the reader the necessary background for dealing with semidefinite programs which have symmetry. Here the basic theory is given and it is illustrated in applications from coding theory, combinatorics, geometry, and polynomial optimization.
In the context of augmented Lagrangian approaches for solving semidefinite programming problems, we investigate the possibility of eliminating the positive semidefinite constraint on the dual matrix by employing a factorization. Hints on how to deal with the resulting unconstrained maximization of the augmented Lagrangian are given. We further use the approximate maximum of the augmented Lagrangian with the aim of improving the convergence rate of alternating direction augmented Lagrangian frameworks. Numerical results are reported, showing the benefits of the approach.
Since more than three decades, interior-point methods proved very useful for optimization, from linear over semidefinite to conic (and partly beyond non-convex) programming; despite the fact that already in the semidefinite case (even when strong duality holds) "hard" problems are known. We shade a light on a rather surprising restriction in the non-commutative world (of semidefinite programming), namely "commutative" paths and propose a new family of solvers that is able to use the full richness of "non-commutative" search directions: (primal) feasible-interior-point methods. Beside a detaile…
We propose a very simple preprocessing algorithm for semidefinite programming. Our algorithm inspects the constraints of the problem, deletes redundant rows and columns in the constraints, and reduces the size of the variable matrix. It often detects infeasibility. Our algorithm does not rely on any optimization solver: the only subroutine it needs is Cholesky factorization, hence it can be implemented with a few lines of code in machine precision. We present computational results on a set of problems arising mostly from polynomial optimization.
Semidefinite programming is an important tool to tackle several problems in data science and signal processing, including clustering and community detection. However, semidefinite programs are often slow in practice, so speed up techniques such as sketching are often considered. In the context of community detection in the stochastic block model, Mixon and Xie \cite{mixon2020sketching} have recently proposed a sketching framework in which a semidefinite program is solved only on a subsampled subgraph of the network, giving rise to significant computational savings. In this short paper, we prov…
In this paper, we present a majorized semismooth Newton-CG augmented Lagrangian method, called SDPNAL$+$, for semidefinite programming (SDP) with partial or full nonnegative constraints on the matrix variable. SDPNAL$+$ is a much enhanced version of SDPNAL introduced by Zhao, Sun and Toh [SIAM Journal on Optimization, 20 (2010), pp.~1737--1765] for solving generic SDPs. SDPNAL works very efficiently for nondegenerate SDPs but may encounter numerical difficulty for degenerate ones. Here we tackle this numerical difficulty by employing a majorized semismooth Newton-CG augmented Lagrangian method…
In contrast with many other convex optimization classes, state-of-the-art semidefinite programming solvers are yet unable to efficiently solve large scale instances. This work aims to reduce this scalability gap by proposing a novel proximal algorithm for solving general semidefinite programming problems. The proposed methodology, based on the primal-dual hybrid gradient method, allows the presence of linear inequalities without the need of adding extra slack variables and avoids solving a linear system at each iteration. More importantly, it does simultaneously compute the dual variables asso…
We present semi-decentralized and distributed algorithms, designed via a preconditioned forward-backward operator splitting, for solving large-scale, decomposable semidefinite programs (SDPs). We exploit a chordal aggregate sparsity pattern assumption on the original SDP to obtain a set of mutually coupled SDPs defined on positive semidefinite (PSD) cones of reduced dimensions. We show that the proposed algorithms converge to a solution of the original SDP via iterations of reasonable computational cost. Finally, we compare the performances of the two proposed algorithms with respect to others…
We derive a stochastic gradient algorithm for semidefinite optimization using randomization techniques. The algorithm uses subsampling to reduce the computational cost of each iteration and the subsampling ratio explicitly controls granularity, i.e. the tradeoff between cost per iteration and total number of iterations. Furthermore, the total computational cost is directly proportional to the complexity (i.e. rank) of the solution. We study numerical performance on some large-scale problems arising in statistical learning.
On Completely Positive and Semidefinite Programs for Moment Bounds on Maximum of Random Quadratics
School of Mathematics and Computer Science, Indian Institute of Technology (IIT) Goa（印度理工学院果阿分校数学与计算机科学学院）；Engineering Systems Design, Singapore University of Technology and Design（新加坡科技设计大学工程设计系统）
We study the ideal generated by polynomials vanishing on a semialgebraic set and propose an algorithm to calculate the generators, which is based on some techniques of the cylindrical algebraic decomposition. By applying these, polynomial optimization problems with polynomial equality constraints can be modified equivalently so that the associated semidefinite programming relaxation problems have no duality gap. Elementary proofs for some criteria on reality of ideals are also given.
In this work, we derive second-order optimality conditions for nonlinear semidefinite programming (NSDP) problems, by reformulating it as an ordinary nonlinear programming problem using squared slack variables. We first consider the correspondence between Karush-Kuhn-Tucker points and regularity conditions for the general NSDP and its reformulation via slack variables. Then, we obtain a pair of "no-gap" second-order optimality conditions that are essentially equivalent to the ones already considered in the literature. We conclude with the analysis of some computational prospects of the squared…
Several algorithms are available in the literature for finding the entire set of Pareto-optimal solutions in MultiObjective Linear Programming (MOLP). However, it has not been proposed so far an interior point algorithm that finds all Pareto-optimal solutions of MOLP. We present an explicit construction, based on a transformation of any MOLP into a finite sequence of SemiDefinite Programs (SDP), the solutions of which give the entire set of Pareto-optimal extreme points solutions of MOLP. These SDP problems are solved by interior point methods; thus our approach provides a pseudo-polynomial in…
This paper studies the problem of stability of a parameterized delay differential equations (DDE see equation (0.1)). After discretizing the DDE (0.1), we show that the problem can be equivalently casted into a semi-definite programming (SDP) see (3.2), which can be solved efficiently through some popular algorithm, e.g., the interior point method [1].
Phase retrieval seeks to recover a signal x from the amplitude |Ax| of linear measurements. We cast the phase retrieval problem as a non-convex quadratic program over a complex phase vector and formulate a tractable relaxation (called PhaseCut) similar to the classical MaxCut semidefinite program. We solve this problem using a provably convergent block coordinate descent algorithm whose structure is similar to that of the original greedy algorithm in Gerchberg-Saxton, where each iteration is a matrix vector product. Numerical results show the performance of this approach over three different p…
Solving semidefinite programs (SDP) in a short time is the key to managing various mathematical optimization problems. The matrix-completion primal-dual interior-point method (MC-PDIPM) extracts a sparse structure of input SDP by factorizing the variable matrices. In this paper, we propose a new factorization based on the inverse of the variable matrix to enhance the performance of MC-PDIPM. We also use multithreaded parallel computing to deal with the major bottlenecks in MC-PDIPM. Numerical results show that the new factorization and multithreaded computing reduce the computation time for SD…
Given a generic semidefinite program, specified by matrices with rational entries, each coordinate of its optimal solution is an algebraic number. We study the degree of the minimal polynomials of these algebraic numbers. Geometrically, this degree counts the critical points attained by a linear functional on a fixed rank locus in a linear space of symmetric matrices. We determine this degree using methods from complex algebraic geometry, such as projective duality, determinantal varieties, and their Chern classes.
We consider a new semidefinite programming (SDP) relaxation of the symmetric traveling salesman problem (TSP) that may be obtained via an SDP relaxation of the more general quadratic assignment problem (QAP). We show that the new relaxation dominates the one in [D. Cvetkovic, M. Cangalovic, and V. Kovacevic-Vujcic, Semidefinite programming methods for the symmetric traveling salesman problem, in Proc. 7th Int. IPCO Conference, Springer, London, 1999, pp. 126--136]. Unlike the bound of Cvetkovic et al., the new SDP bound is not dominated by the Held-Karp linear programming bound, or vice versa.
In this paper we analyze the spectral gap of a weighted graph which is the difference between the smallest positive and largest negative eigenvalue of its adjacency matrix. Such a graph can represent e.g. a chemical organic molecule. Our goal is to construct a new graph by bridging two given weighted graphs over a bipartite graph. The aim is to maximize the spectral gap with respect to a bridging graph. To this end, we construct a mixed integer semidefinite program for maximization of the spectral gap and compute it numerically.
It is known that one can solve semidefinite programs to within fixed accuracy in polynomial time using the ellipsoid method (under some assumptions). In this paper it is shown that the same holds true when one uses the short-step, primal interior point method. The main idea of the proof is to employ Diophantine approximation at each iteration to bound the intermediate bit-sizes of iterates.
This paper is a tutorial in a general and explicit procedure to simplify semidefinite programs which are invariant under the action of a symmetry group. The procedure is based on basic notions of representation theory of finite groups. As an example we derive the block diagonalization of the Terwilliger algebra of the binary Hamming scheme in this framework. Here its connection to the orthogonal Hahn and Krawtchouk polynomials becomes visible.
The constant rank constraint qualification, introduced by Janin in 1984 for nonlinear programming, has been extensively used for sensitivity analysis, global convergence of first- and second-order algorithms, and for computing the derivative of the value function. In this paper we discuss naive extensions of constant rank-type constraint qualifications to second-order cone programming and semidefinite programming, which are based on the Approximate-Karush-Kuhn-Tucker necessary optimality condition and on the application of the reduction approach. Our definitions are strictly weaker than Robins…
