Efficient representations of convex sets are of crucial importance for many algorithms that work with them. It is well-known that sometimes, a complicated convex set can be expressed as the projection of a much simpler set in higher dimensions called a lift of the original set. This is a brief survey of recent developments in the topic of lifts of convex sets. Our focus will be on lifts that arise from affine slices of real positive semidefinite cones known as psd or spectrahedral lifts. The main result is that projection representations of a convex set are controlled by factorizations, through closed convex cones, of an operator that comes from the convex set. This leads to several research directions and results that lie at the intersection of convex geometry, combinatorics, real algebraic geometry, optimization, computer science and more.
This paper presents a selected tour through the theory and applications of lifts of convex sets. A lift of a convex set is a higher-dimensional convex set that projects onto the original set. Many convex sets have lifts that are dramatically simpler to describe than the original set. Finding such simple lifts has significant algorithmic implications, particularly for optimization problems. We consider both the classical case of polyhedral lifts, described by linear inequalities, as well as spectrahedral lifts, defined by linear matrix inequalities, with a focus on recent developments related t…
Hyperbolicity cones are convex algebraic cones arising from hyperbolic polynomials. A well-understood subclass of hyperbolicity cones is that of spectrahedral cones and it is conjectured that every hyperbolicity cone is spectrahedral. In this paper we prove a weaker version of this conjecture by showing that every smooth hyperbolicity cone is the linear projection of a spectrahedral cone, that is, a spectrahedral shadow.
Spectrahedral shadows are projections of linear sections of the cone of positive semidefinite matrices. We characterize the polynomials that vanish on the boundaries of these convex sets when both the section and the projection are generic.
Convex regression is the problem of fitting a convex function to a data set consisting of input-output pairs. We present a new approach to this problem called spectrahedral regression, in which we fit a spectrahedral function to the data, i.e. a function that is the maximum eigenvalue of an affine matrix expression of the input. This method represents a significant generalization of polyhedral (also called max-affine) regression, in which a polyhedral function (a maximum of a fixed number of affine functions) is fit to the data. We prove bounds on how well spectrahedral functions can approxima…
We study the convex hull of $SO(n)$, thought of as the set of $n\times n$ orthogonal matrices with unit determinant, from the point of view of semidefinite programming. We show that the convex hull of $SO(n)$ is doubly spectrahedral, i.e. both it and its polar have a description as the intersection of a cone of positive semidefinite matrices with an affine subspace. Our spectrahedral representations are explicit, and are of minimum size, in the sense that there are no smaller spectrahedral representations of these convex bodies.
We give explicit polynomial-sized (in $n$ and $k$) semidefinite representations of the hyperbolicity cones associated with the elementary symmetric polynomials of degree $k$ in $n$ variables. These convex cones form a family of non-polyhedral outer approximations of the non-negative orthant that preserve low-dimensional faces while successively discarding high-dimensional faces. More generally we construct explicit semidefinite representations (polynomial-sized in $k,m$, and $n$) of the hyperbolicity cones associated with $k$th directional derivatives of polynomials of the form $p(x) = \det(\s…
Let $K$ be a compact Lie group and $V$ a finite-dimensional representation of $K$. The orbitope of a vector $x\in V$ is the convex hull $\mathscr O_x$ of the orbit $Kx$ in $V$. We show that if $V$ is polar then $\mathscr O_x$ is a spectrahedron, and we produce an explicit linear matrix inequality representation. We also consider the coorbitope $\mathscr O_x^o$, which is the convex set polar to $\mathscr O_x$. We prove that $\mathscr O_x^o$ is the convex hull of finitely many $K$-orbits, and we identify the cases in which $\mathscr O_x^o$ is itself an orbitope. In these cases one has $\mathscr …
We show that there are many (compact) convex semi-algebraic sets in euclidean space that do not have a semidefinite representation. This gives a negative answer to a question by Nemirovski, resp. it shows that the Helton-Nie conjecture is false.
The Helton-Nie Conjecture (HNC) is the proposition that every convex semialgebraic set is a spectrahedral shadow. Here we prove that HNC is equivalent to another propo- sition related to quadratically constrained quadratic programming. Namely, that the convex hull of the rank-one elements of any spectrahedron is a spectrahedral shadow. In the case of compact convex semialgebraic sets, the spectrahedra may be taken to be compact. We illustrate the relationship between spetrahedra and these convex subsets with examples.
If $X$ is an $n\times n$ symmetric matrix, then the directional derivative of $X \mapsto \det(X)$ in the direction $I$ is the elementary symmetric polynomial of degree $n-1$ in the eigenvalues of $X$. This is a polynomial in the entries of $X$ with the property that it is hyperbolic with respect to the direction $I$. The corresponding hyperbolicity cone is a relaxation of the positive semidefinite (PSD) cone known as the first derivative relaxation (or Renegar derivative) of the PSD cone. A spectrahedal cone is a convex cone that has a representation as the intersection of a subspace with the …
A spectrahedron is a set defined by a linear matrix inequality. Given a spectrahedron we are interested in the question of the smallest possible size $r$ of the matrices in the description by linear matrix inequalities. We show that for the $n$-dimensional unit ball $r$ is at least $\frac{n}{2}$. If $n=2^k+1$, then we actually have $r=n$. The same holds true for any compact convex set in $\mathbb{R}^n$ defined by a quadratic polynomial. Furthermore, we show that for a convex region in $\mathbb{R}^3$ whose algebraic boundary is smooth and defined by a cubic polynomial we have that $r$ is at lea…
Let $C$ be a proper convex cone generated by a compact set which supports a measure $μ$. A construction due to A.Barvinok, E.Veomett and J.B. Lasserre produces, using $μ$, a sequence $(P_k)_{k\in \mathbb{N}}$ of nested spectrahedral cones which contains the cone $C^*$ dual to $C$. We prove convergence results for such sequences of spectrahedra and provide tools for bounding the distance between $P_k$ and $C^*$. These tools are especially useful on cones with enough symmetries and allow us to determine bounds for several cones of interest. We compute such upper bounds for semidefinite approxima…
In this paper we generalize the factorization theorem of Gouveia, Parrilo and Thomas to a broader class of convex sets. Given a general convex set, we define a slack operator associated to the set and its polar according to whether the convex set is full dimensional, whether it is a translated cone and whether it contains lines. We strengthen the condition of a cone lift by requiring not only the convex set is the image of an affine slice of a given closed convex cone, but also its recession cone is the image of the linear slice of the closed convex cone. We show that the generalized lift of a…
In this paper we address the basic geometric question of when a given convex set is the image under a linear map of an affine slice of a given closed convex cone. Such a representation or 'lift' of the convex set is especially useful if the cone admits an efficient algorithm for linear optimization over its affine slices. We show that the existence of a lift of a convex set to a cone is equivalent to the existence of a factorization of an operator associated to the set and its polar via elements in the cone and its dual. This generalizes a theorem of Yannakakis that established a connection be…
A lift-and-permute scheme of alternating direction method of multipliers (ADMM) is proposed for linearly constrained convex programming. It contains not only the newly developed balanced augmented Lagrangian method and its dual-primal variation, but also the proximal ADMM and Douglas-Rachford splitting algorithm. It helps to propose accelerated algorithms with worst-case $O(1/k^2)$ convergence rates in the case that the objective function to be minimized is strongly convex.
Two new optimization techniques based on projections onto convex space (POCS) framework for solving convex and some non-convex optimization problems are presented. The dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. If the cost function is a convex function in R^N the corresponding set is a convex set in R^(N+1). The iterative optimization approach starts with an arbitrary initial estimate in R^(N+1) and an orthogonal projection is performed onto one of the sets in a sequential manner at each step of the optimization problem. The …
A convex optimization problem in conic form involves minimizing a linear functional over the intersection of a convex cone and an affine subspace. In some cases, it is possible to replace a conic formulation using a certain cone, with a 'lifted' conic formulation using another cone that is higher-dimensional, but simpler, in some sense. One situation in which this can be computationally advantageous is when the higher-dimensional cone is a Cartesian product of many 'low-complexity' cones, such as second-order cones, or small positive semidefinite cones. This paper studies obstructions to a con…
Linear matrix Inequalities (LMIs) have had a major impact on control but formulating a problem as an LMI is an art. Recently there is the beginnings of a theory of which problems are in fact expressible as LMIs. For optimization purposes it can also be useful to have "lifts" which are expressible as LMIs. We show here that this is a much less restrictive condition and give methods for actually constructing lifts and their LMI representation.
The training of two-layer neural networks with nonlinear activation functions is an important non-convex optimization problem with numerous applications and promising performance in layerwise deep learning. In this paper, we develop exact convex optimization formulations for two-layer neural networks with second degree polynomial activations based on semidefinite programming. Remarkably, we show that semidefinite lifting is always exact and therefore computational complexity for global optimization is polynomial in the input dimension and sample size for all input data. The developed convex fo…
We consider operators acting on convex subsets of the unit hypercube. These operators are used in constructing convex relaxations of combinatorial optimization problems presented as a 0,1 integer programming problem or a 0,1 polynomial optimization problem. Our focus is mostly on operators that, when expressed as a lift-and-project operator, involve the use of semidefiniteness constraints in the lifted space, including operators due to Lasserre and variants of the Sherali--Adams and Bienstock--Zuckerberg operators. We study the performance of these semidefinite-optimization-based lift-and-proj…
We consider lift-and-project methods for combinatorial optimization problems and focus mostly on those lift-and-project methods which generate polyhedral relaxations of the convex hull of integer solutions. We introduce many new variants of Sherali--Adams and Bienstock--Zuckerberg operators. These new operators fill the spectrum of polyhedral lift-and-project operators in a way which makes all of them more transparent, easier to relate to each other, and easier to analyze. We provide new techniques to analyze the worst-case performances as well as relative strengths of these operators in a uni…
This note focuses on the problem of representing convex sets as projections of the cone of positive semidefinite matrices, in the particular case of sets generated by bivariate polynomials of degree four. Conditions are given for the convex hull of a plane quartic to be exactly semidefinite representable with at most 12 lifting variables. If the quartic is rationally parametrizable, an exact semidefinite representation with 2 lifting variables can be obtained. Various numerical examples illustrate the techniques and suggest further research directions.
Polyhedral results for two classes of submodular sets with GUB constraints
University of Science and Technology of China（中国科学技术大学）；Beijing Institute of Technology（北京理工大学）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
We characterize and compute the maximal admissible positively invariant set for asymptotically stable constrained switching linear systems. Motivated by practical problems found, e.g., in obstacle avoidance, power electronics and nonlinear switching systems, in our setting the constraint set is formed by a finite number of polynomial inequalities. First, we observe that the so-called Veronese lifting allows to represent the constraint set as a polyhedral set. Next, by exploiting the fact that the lifted system dynamics remains linear, we establish a method based on reachability computations to…
General Successive Convex Relaxation Methods (SRCMs) can be used to compute the convex hull of any compact set, in an Euclidean space, described by a system of quadratic inequalities and a compact convex set which is not very complicated. Linear Complementarity Problems (LCPs) make an interesting and rich class of structured nonconvex optimization problems. In this paper, we study a few of the specialized lift-and-project methods and some of the possible ways of applying the general SCRMs to LCPs and related problems.
Two new optimization techniques based on projections onto convex space (POCS) framework for solving convex optimization problems are presented. The dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. If the cost function is a convex function in R^N the corresponding set is also a convex set in R^{N+1}. The iterative optimization approach starts with an arbitrary initial estimate in R^{N+1} and an orthogonal projection is performed onto one of the sets in a sequential manner at each step of the optimization problem. The method provides…
A new deconvolution algorithm based on orthogonal projections onto the epigraph set of a convex cost function is presented. In this algorithm, the dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. As the utilized cost function is a convex function in $R^N$, the corresponding epigraph set is also a convex set in $R^{N+1}$. The deconvolution algorithm starts with an arbitrary initial estimate in $R^{N+1}$. At each step of the iterative algorithm, first deconvolution projections are performed onto the epigraphs, later an orthogonal pro…
Bilkent University（比尔肯特大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）
A new signal processing framework based on making orthogonal Projections onto the Epigraph Set of a Convex cost function (PESC) is developed. In this way it is possible to solve convex optimization problems using the well-known Projections onto Convex Set (POCS) approach. In this algorithm, the dimension of the minimization problem is lifted by one and a convex set corresponding to the epigraph of the cost function is defined. If the cost function is a convex function in $R^N$, the corresponding epigraph set is also a convex set in R^{N+1}. The PESC method provides globally optimal solutions f…
