The numerical range of a matrix is studied geometrically via the cone of positive semidefinite matrices (or semidefinite cone for short). In particular it is shown that the feasible set of a two-dimensional linear matrix inequality (LMI), an affine section of the semidefinite cone, is always dual to the numerical range of a matrix, which is therefore an affine projection of the semidefinite cone. Both primal and dual sets can also be viewed as convex hulls of explicit algebraic plane curve components. Several numerical examples illustrate this interplay between algebra, geometry and semidefinite programming duality. Finally, these techniques are used to revisit a theorem in statistics on the independence of quadratic forms in a normally distributed vector.
LAAS-CNRS, University of Toulouse, France（图卢兹大学 LAAS-CNRS）；Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic（布拉格捷克理工大学电气工程学院）
Packing problems in discrete geometry can be modeled as finding independent sets in infinite graphs where one is interested in independent sets which are as large as possible. For finite graphs one popular way to compute upper bounds for the maximal size of an independent set is to use Lasserre's semidefinite programming hierarchy. We generalize this approach to infinite graphs. For this we introduce topological packing graphs as an abstraction for infinite graphs coming from packing problems in discrete geometry. We show that our hierarchy converges to the independence number.
This is a survey article on real algebra and geometry, and in particular on its recent applications in optimization and convexity. We first introduce basic notions and results from the classical theory. We then explain how these relate to optimization, mostly via semidefinite programming. We introduce interesting geometric problems arising from the classification of feasible sets for semidefinite programming. We close with a perspective on the very active area of non-commutative real algebra and geometry.
The numerical performance of algorithms can be studied using test sets or procedures that generate such problems. This paper proposes various methods for generating linear, semidefinite, and second-order cone optimization problems. Specifically, we are interested in problem instances requiring a known optimal solution, a known optimal partition, a specific interior solution, or all these together. In the proposed problem generators, different characteristics of optimization problems, including dimension, size, condition number, degeneracy, optimal partition, and sparsity, can be chosen to faci…
This paper revisits the parametric analysis of semidefinite optimization problems with respect to the perturbation of the objective function along a fixed direction. We review the notions of invariancy set, nonlinearity interval, and transition point of the optimal partition, and we investigate their characterizations. We show that the set of transition points is finite and the continuity of the optimal set mapping, on the basis of Painlevé-Kuratowski set convergence, might fail on a nonlinearity interval. Under a local nonsingularity condition, we then develop a methodology, stemming from num…
Decomposition of large matrix inequalities for matrices with chordal sparsity graph has been recently used by Kojima et al.\ \cite{kim2011exploiting} to reduce problem size of large scale semidefinite optimization (SDO) problems and thus increase efficiency of standard SDO software. A by-product of such a decomposition is the introduction of new dense small-size matrix variables. We will show that for arrow type matrices satisfying suitable assumptions, the additional matrix variables have rank one and can thus be replaced by vector variables of the same dimensions. This leads to significant i…
In this paper, we investigate the generalized low rank approximation to the symmetric positive semidefinite matrix in the Frobenius norm: $$\underset{ rank(X)\leq k}{\min} \sum^m_{i=1}\left \Vert A_i - B_i XB_i^T \right \Vert^2_F,$$ where $X$ is an unknown symmetric positive semidefinite matrix and $k$ is a positive integer. We firstly use the property of a symmetric positive semidefinite matrix $X=YY^T$, $Y$ with order $n\times k$, to convert the generalized low rank approximation into unconstraint generalized optimization problem. Then we apply the nonlinear conjugate gradient method to solv…
Approximate Projections onto the Positive Semidefinite Cone Using Randomization
School of Electrical and Electronic Engineering, The University of Sheffield（谢菲尔德大学电气与电子工程学院）；Department of Electrical Engineering, Columbia University（哥伦比亚大学电气工程系）
The augmented Lagrangian method (ALM) has gained tremendous popularity for its elegant theory and impressive numerical performance since it was proposed by Hestenes and Powell in 1969. It has been widely used in numerous efficient solvers to improve numerical performance to solve many problems. In this paper, without requiring the uniqueness of multipliers, the local (asymptotic Q-superlinear) Q-linear convergence rate of the primal-dual sequences generated by ALM for the nonlinear semidefinite programming (NLSDP) is established by assuming the second-order sufficient condition (SOSC) and the …
In this paper, we propose iterative inner/outer approximations based on a recent notion of block factor-width-two matrices for solving semidefinite programs (SDPs). Our inner/outer approximating algorithms generate a sequence of upper/lower bounds of increasing accuracy for the optimal SDP cost. The block partition in our algorithms offers flexibility in terms of both numerical efficiency and solution quality, which includes the approach of scaled diagonally dominance (SDD) approximation as a special case. We discuss both the theoretical results and numerical implementation in detail. Our main…
The local convergence of alternating optimization methods with overrelaxation for low-rank matrix and tensor problems is established. The analysis is based on the linearization of the method which takes the form of an SOR iteration for a positive semidefinite Hessian and can be studied in the corresponding quotient geometry of equivalent low-rank representations. In the matrix case, the optimal relaxation parameter for accelerating the local convergence can be determined from the convergence rate of the standard method. This result relies on a version of Young's SOR theorem for positive semide…
In the context of augmented Lagrangian approaches for solving semidefinite programming problems, we investigate the possibility of eliminating the positive semidefinite constraint on the dual matrix by employing a factorization. Hints on how to deal with the resulting unconstrained maximization of the augmented Lagrangian are given. We further use the approximate maximum of the augmented Lagrangian with the aim of improving the convergence rate of alternating direction augmented Lagrangian frameworks. Numerical results are reported, showing the benefits of the approach.
The paper proves sum-of-square-of-rational-function based representations (shortly, sosrf-based representations) of polynomial matrices that are positive semidefinite on some special sets: $\mathbb{R}^n;$ $\mathbb{R}$ and its intervals $[a,b]$, $[0,\infty)$; and the strips $[a,b] \times \mathbb{R} \subset \mathbb{R}^2.$ A method for numerically computing such representations is also presented. The methodology is divided into two stages: (S1) diagonalizing the initial polynomial matrix based on the Schmüdgen's procedure \cite{Schmudgen09}; (S2) for each diagonal element of the resulting matrix,…
We investigate the problem of finding inner ap-proximations of positive semidefinite (PSD) cones. We developa novel decomposition framework of the PSD cone by meansof conical combinations of smaller dimensional sub-cones. Weshow that many inner approximation techniques could besummarized within this framework, including the set of (scaled)diagonally dominant matrices, Factor-widthkmatrices, andChordal Sparse matrices. Furthermore, we provide a moreflexible family of inner approximations of the PSD cone, wherewe aim to arrange the sub-cones so that they are maximallyseparated from each other. I…
A weakly infeasible semidefinite program (SDP) has no feasible solution, but it has approximate solutions whose constraint violation is arbitrarily small. These SDPs are ill-posed and numerically often unsolvable. They are also closely related to "bad" linear projections that map the cone of positive semidefinite matrices to a nonclosed set. We describe a simple echelon form of weakly infeasible SDPs with the following properties: (i) it is obtained by elementary row operations and congruence transformations, (ii) it makes weak infeasibility evident, and (iii) it permits us to construct any we…
The generalized problem of moments is a conic linear optimization problem over the convex cone of positive Borel measures with given support. It has a large variety of applications, including global optimization of polynomials and rational functions, options pricing in finance, constructing quadrature schemes for numerical integration, and distributionally robust optimization. A usual solution approach, due to J.B. Lasserre, is to approximate the convex cone of positive Borel measures by finite dimensional outer and inner conic approximations. We will review some results on these approximation…
We use techniques from (tracial noncommutative) polynomial optimization to formulate hierarchies of semidefinite programming lower bounds on matrix factorization ranks. In particular, we consider the nonnegative rank, the positive semidefinite rank, and their symmetric analogues: the completely positive rank and the completely positive semidefinite rank. We study the convergence properties of our hierarchies, compare them extensively to known lower bounds, and provide some (numerical) examples.
Clique tree conversion solves large-scale semidefinite programs by splitting an $n\times n$ matrix variable into up to $n$ smaller matrix variables, each representing a principal submatrix of up to $ω\timesω$. Its fundamental weakness is the need to introduce overlap constraints that enforce agreement between different matrix variables, because these can result in dense coupling. In this paper, we show that by dualizing the clique tree conversion, the coupling due to the overlap constraints is guaranteed to be sparse over dense blocks, with a block sparsity pattern that coincides with the adja…
