We review several (and provide new) results on the theory of moments, sums of squares and basic semi-algebraic sets when convexity is present. In particular, we show that under convexity, the hierarchy of semidefinite relaxations for polynomial optimization simplifies and has finite convergence, a highly desirable feature as convex problems are in principle easier to solve. In addition, if a basic semi-algebraic set K is convex but its defining polynomials are not, we provide a certificate of convexity if a sufficient (and almost necessary) condition is satified. This condition can be checked numerically and also provides a new condition for K to have semidefinite representation. For this we use (and extend) some of recent results from the author and Helton and Nie. Finally, we show that when restricting to a certain class of convex polynomials, the celebrated Jensen's inequality in convex analysis can be extended to linear functionals that are not necessarily probability measures.
We provide two certificates of convexity for arbitrary basic semi-algebraic sets of $\R^n$. The first one is based on a necessary and sufficient condition whereas the second one is based on a sufficient (but simpler) condition only. Both certificates are obtained from any feasible solution of a related semidefinite program and so can be obtained numerically (however, up to machine precision).
Chapters 1 to 4 are the lecture notes of my course "Real Algebraic Geometry I" from the winter term 2020/2021. Chapters 5 to 8 are the lecture notes of its continuation "Real Algebraic Geometry II" from the summer term 2021. Chapters 9 and 10 are the lecture notes of its further continuation "Geometry of Linear Matrix Inequalities" from the winter term 2021/2022. These courses have been delivered at the University of Konstanz in Southern Germany. The entirety of these lecture notes is accompanied by a list of 47 long videos which is available from the following YouTube playlist: https://youtub…
The goal of this paper is to measure the non-convexity of compact and smooth connected components of real algebraic plane curves. We study these curves first in a general setting and then in an asymptotic one. In particular, we consider sufficiently small levels of a real bivariate polynomial in a small enough neighbourhood of a strict local minimum at the origin of the real affine plane. We introduce and describe a new combinatorial object, called the Poincare-Reeb graph, whose role is to encode the shape of such curves and to allow us to quantify their non-convexity. Moreover, we prove that …
In the complex plane, the frequency response of a univariate polynomial is the set of values taken by the polynomial when evaluated along the imaginary axis. This is an algebraic curve partitioning the plane into several connected components. In this note it is shown that the component including the origin is exactly representable by a linear matrix inequality if and only if the polynomial is stable, in the sense that all its roots have negative real parts.
In this paper, the quasi-convexity of a sum of quadratic fractions in the form $\sum_{i=1}^n \frac{1+c_i x^2}{\left(1+d_ix\right)^2}$ is demonstrated where $c_i$ and $d_i$ are strictly positive scalars, when defined on the positive real axis $\mathbb{R}^{+}$. It will be shown that this quasi-convexity guarantees it has a unique local (and hence global) minimum. Indeed, this problem arises when considering the optimization of the weighting coefficient in regularized semi-blind channel identification problem, and more generally, is of interest in other contexts where we combine two different est…
We propose a projected semi-stochastic gradient descent method with mini-batch for improving both the theoretical complexity and practical performance of the general stochastic gradient descent method (SGD). We are able to prove linear convergence under weak strong convexity assumption. This requires no strong convexity assumption for minimizing the sum of smooth convex functions subject to a compact polyhedral set, which remains popular across machine learning community. Our PS2GD preserves the low-cost per iteration and high optimization accuracy via stochastic gradient variance-reduced tech…
We present a generalization of the notion of neighborliness to non-polyhedral convex cones. Although a definition of neighborliness is available in the non-polyhedral case in the literature, it is fairly restrictive as it requires all the low-dimensional faces to be polyhedral. Our approach is more flexible and includes, for example, the cone of positive-semidefinite matrices as a special case (this cone is not neighborly in general). We term our generalization Terracini convexity due to its conceptual similarity with the conclusion of Terracini's lemma from algebraic geometry. Polyhedral cone…
This is a survey article on real algebra and geometry, and in particular on its recent applications in optimization and convexity. We first introduce basic notions and results from the classical theory. We then explain how these relate to optimization, mostly via semidefinite programming. We introduce interesting geometric problems arising from the classification of feasible sets for semidefinite programming. We close with a perspective on the very active area of non-commutative real algebra and geometry.
We give sufficient conditions for the expected excess and the upper semideviation of recourse functions to be strongly convex. This is done in the setting of two-stage stochastic programs with complete linear recourse and random right-hand side. This work extends results on strong convexity of risk-neutral models.
The set of controllers stabilizing a linear system is generally non-convex in the parameter space. In the case of two-parameter controller design (e.g. PI control or static output feedback with one input and two outputs), we observe however that quite often for benchmark problem instances, the set of stabilizing controllers seems to be convex. In this note we use elementary techniques from real algebraic geometry (resultants and Bezoutian matrices) to explain this phenomenon. As a byproduct, we derive a convex linear matrix inequality (LMI) formulation of two-parameter fixed-order controller d…
We study the algebraic boundary of a convex semi-algebraic set via duality in convex and algebraic geometry. We generalize the correspondence of facets of a polytope to the vertices of the dual polytope to general semi-algebraic convex bodies. In the general setup, exceptional families of extreme points might exist and we characterize them semi-algebraically. We also give an algorithm to compute a complete list of exceptional families, given the algebraic boundary of the dual convex set.
A class of real functions, which is the generalization of a family of convex functions, is introduced; in this connection, we have defined $X$-convex, strictly $X$-convex, quasi-$X$-convex, strictly quasi-$X$-convex, and semi-strictly quasi-$X$-convex functions. Moreover, in this paper, we give a detailed study of the fundamental properties of these functions with various examples, supporting the concepts. Finally, the study of optimization problems employs quasi-$X$-convex, semistrictly quasi-$X$-convex, and strictly quasi-$X$-convex functions.
$\mathbb B$-convexity was defined in [7] as a suitable Kuratowski-Painlevé upper limit of linear convexities over a finite dimensional Euclidean vector space. Excepted in the special case where convex sets are subsets of $\mathbb R^n_ +$, $\mathbb B$-convexity was not defined with respect to a given explicit algebraic structure. This is done in that paper, which proposes an extension of $\mathbb B$-convexity to the whole Euclidean vector space. An unital idempotent and non-associative magma is defined over the real set and an extended $n$-ary operation is introduced. Along this line, the exist…
Given a polynomial $f$ and a semi-algebraic set $S$, we provide a symbolic algorithm to find the equations and inequalities defining a semi-algebraic set $Q$ which is identical to the closure of the image of $S$ under $f$, i.e., \begin{equation} Q=\overline{f(S)}\,. \end{equation} Consequently, every polynomial optimization problem whose optimum value is finite has an equivalent form with attained optimum value, i.e., \begin{equation} \min \limits_{t\in Q} t =\inf\limits_{x\in S} f(x) \end{equation} whenever the right-hand side is finite. Given $d$ as the upper bound on the degrees of $f$ and …
We introduce and investigate a new generalized convexity notion for functions called prox-convexity. The proximity operator of such a function is single-valued and firmly nonexpansive. We provide examples of (strongly) quasiconvex, weakly convex, and DC (difference of convex) functions that are prox-convex, however none of these classes fully contains the one of prox-convex functions or is included into it. We show that the classical proximal point algorithm remains convergent when the convexity of the proper lower semicontinuous function to be minimized is relaxed to prox-convexity.
We use the framework of a type of abstract convexity ($Φ_{lsc}$-convexity) to investigate properties of lower semicontinuous quadratically minorized functions in Hilbert spaces. A new result, which states that, for every local $Φ_{lsc}$-subgradient there exists a global one is proved and plays a crucial role in our considerations. We deliver conditions for abstract subdifferentiability ($Φ_{lsc}$-subdifferentiability) of locally $C^{1,1}$ functions, twice continuously differentiable functions, prox-regular functions and paraconvex functions. As an application we establish a new sufficient and …
Given a polynomial $x \in {\mathbb R}^n \mapsto p(x)$ in $n=2$ variables, a symbolic-numerical algorithm is first described for detecting whether the connected component of the plane sublevel set ${\mathcal P} = \{x : p(x) \geq 0\}$ containing the origin is rigidly convex, or equivalently, whether it has a linear matrix inequality (LMI) representation, or equivalently, if polynomial $p(x)$ is hyperbolic with respect to the origin. The problem boils down to checking whether a univariate polynomial matrix is positive semidefinite, an optimization problem that can be solved with eigenvalue decomp…
This paper considers the analysis of continuous time gradient-based optimization algorithms through the lens of nonlinear contraction theory. It demonstrates that in the case of a time-invariant objective, most elementary results on gradient descent based on convexity can be replaced by much more general results based on contraction. In particular, gradient descent converges to a unique equilibrium if its dynamics are contracting in any metric, with convexity of the cost corresponding to the special case of contraction in the identity metric. More broadly, contraction analysis provides new ins…
A multivariate polynomial $p(x)=p(x_1,...,x_n)$ is sos-convex if its Hessian $H(x)$ can be factored as $H(x)= M^T(x) M(x)$ with a possibly nonsquare polynomial matrix $M(x)$. It is easy to see that sos-convexity is a sufficient condition for convexity of $p(x)$. Moreover, the problem of deciding sos-convexity of a polynomial can be cast as the feasibility of a semidefinite program, which can be solved efficiently. Motivated by this computational tractability, it has been recently speculated whether sos-convexity is also a necessary condition for convexity of polynomials. In this paper, we give…
The so-called l0 pseudonorm counts the number of nonzero components of a vector of a Euclidian space. It is well-known that the l0 pseudonorm is not convex, as its Fenchel biconjugate is zero. In this paper, we introduce a suitable conjugacy, induced by a novel coupling, E-Capra, that has the property of being constant along primal rays like the l0 pseudonorm. The coupling E-Capra belongs to the class of one-sided linear couplings, that we introduce; we show that they induce conjugacies that share nice properties with the classic Fenchel conjugacy. For the E-Capra conjugacy, induced by the cou…
We define convexity canonically in the setting of monoids. We show that many classical results from convex analysis hold for functions defined on such groups and semigroups, rather than only on vector spaces. Some examples and counter-examples are also discussed.
Our first contribution in this paper is to prove that three natural sum of squares (sos) based sufficient conditions for convexity of polynomials, via the definition of convexity, its first order characterization, and its second order characterization, are equivalent. These three equivalent algebraic conditions, henceforth referred to as sos-convexity, can be checked by semidefinite programming whereas deciding convexity is NP-hard. If we denote the set of convex and sos-convex polynomials in $n$ variables of degree $d$ with $\tilde{C}_{n,d}$ and $\tilde{ΣC}_{n,d}$ respectively, then our main …
Friedrich Schiller University Jena（耶拿弗里德里希·席勒大学）；Technical University of Kaiserslautern（凯泽斯劳滕工业大学）；Data Assessment Solutions GmbH（数据评估解决方案有限责任公司）
The Hessian of a differentiable convex function is positive semidefinite. Therefore, checking the Hessian of a given function is a natural approach to certify convexity. However, implementing this approach is not straightforward since it requires a representation of the Hessian that allows its analysis. Here, we implement this approach for a class of functions that is rich enough to support classical machine learning. For this class of functions, it was recently shown how to compute computational graphs of their Hessians. We show how to check these graphs for positive semidefiniteness. We comp…
In this paper, we study classes of discrete convex functions: submodular functions on modular semilattices and L-convex functions on oriented modular graphs. They were introduced by the author in complexity classification of minimum 0-extension problems. We clarify the relationship to other discrete convex functions, such as $k$-submodular functions, skew-bisubmodular functions, L$^\natural$-convex functions, tree submodular functions, and UJ-convex functions. We show that they are actually viewed as submodular/L-convex functions in our sense. We also prove a sharp iteration bound of the steep…
We prove that the subdifferential of any semi-algebraic extended-real-valued function on $\R^n$ has $n$-dimensional graph. We discuss consequences for generic semi-algebraic optimization problems.
We show that quadratic growth of a semi-algebraic function is equivalent to strong metric subregularity of the subdifferential --- a kind of stability of generalized critical points. In contrast, this equivalence can easily fail outside of the semi-algebraic setting. As a consequence, we derive necessary conditions and sufficient conditions for optimality in subdifferential terms.
This work presents a collection of useful properties of the Moreau envelope for finite-dimensional, proper, lower semicontinuous, convex functions. In particular, gauge functions and piecewise cubic functions are investigated and their Moreau envelopes categorized. Characterizations of convex Moreau envelopes are established; topics include strict convexity, strong convexity and Lipschitz continuity.
Convex optimization is a vibrant and successful area due to the existence of a variety of efficient algorithms that leverage the rich structure provided by convexity. Convexity of a smooth set or a function in a Euclidean space is defined by how it interacts with the standard differential structure in this space -- the Hessian of a convex function has to be positive semi-definite everywhere. However, in recent years, there is a growing demand to understand non-convexity and develop computational methods to optimize non-convex functions. Intriguingly, there is a type of non-convexity that disap…
In this paper, we study the rate of convergence of the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets. We establish an explicit convergence rate estimate which relies on the maximum degree of the polynomials that generate the basic semi-algebraic convex sets and the dimension of the underlying space. We achieve our results by exploiting the algebraic structure of the basic semi-algebraic convex sets.
In 'Tractable semi-algebraic approximation using Christoffel-Darboux kernel' Marx, Pauwels, Weisser, Henrion and Lasserre conjectured, that the approximation rate $\mathcal O(\frac 1 {\sqrt(d)})$ of a Lipschitz functions by a semi-algebraic function induced by a Christoffel- Darboux kernel of degree $d$ in the $L^1$ norm can be improved for more regular functions. Here we will show, that for semi-algebraic and definable functions the results can be strengthened to a rational approximation rate in the $L^\infty$ norm.
