In this paper, we generalize the chance optimization problems and introduce constrained volume optimization where enables us to obtain convex formulation for challenging problems in systems and control. We show that many different problems can be cast as a particular cases of this framework. In constrained volume optimization, we aim at maximizing the volume of a semialgebraic set under some semialgebraic constraints. Building on the theory of measures and moments, a sequence of semidefinite programs are provided, whose sequence of optimal values is shown to converge to the optimal value of the original problem. We show that different problems in the area of systems and control that are known to be nonconvex can be reformulated as special cases of this framework. Particularly, in this work, we address the problems of probabilistic control of uncertain systems as well as inner approximation of region of attraction and invariant sets of polynomial systems. Numerical examples are presented to illustrate the computational performance of the proposed approach.
10 years ago or so Bill Helton introduced me to some mathematical problems arising from semidefinite programming. This paper is a partial account of what was and what is happening with one of these problems, including many open questions and some new results.
In this paper, "chance optimization" problems are introduced, where one aims at maximizing the probability of a set defined by polynomial inequalities. These problems are, in general, nonconvex and computationally hard. With the objective of developing systematic numerical procedures to solve such problems, a sequence of convex relaxations based on the theory of measures and moments is provided, whose sequence of optimal values is shown to converge to the optimal value of the original problem. Indeed, we provide a sequence of semidefinite programs of increasing dimension which can arbitrarily …
Motivated by problems of uncertainty propagation and robust estimation we are interested in computing a polynomial sublevel set of fixed degree and minimum volume that contains a given semialgebraic set K. At this level of generality this problem is not tractable, even though it becomes convex e.g. when restricted to nonnegative homogeneous polynomials. Our contribution is to describe and justify a tractable L^1-norm or trace heuristic for this problem, relying upon hierarchies of linear matrix inequality (LMI) relaxations when K is semialgebraic, and simplifying to linear programming (LP) whe…
We consider the problem of approximating a semialgebraic set with a sublevel-set of a polynomial function. In this setting, it is standard to seek a minimum volume outer approximation and/or maximum volume inner approximation. As there is no known relationship between the coefficients of an arbitrary polynomial and the volume of its sublevel sets, previous works have proposed heuristics based on the determinant and trace objectives commonly used in ellipsoidal fitting. For the case of star-convex semialgebraic sets, we propose a novel objective which yields both an outer and an inner approxima…
Convex inner approximations of nonconvex semialgebraic sets applied to fixed-order controller design
CNRS（法国国家科学研究中心；实验室分析与系统研究所）；LAAS（图卢兹大学）；Université de Toulouse（捷克布拉格捷克理工大学电气工程学院）；UPS, INSA, INP, ISAE；UT1, UTM, LAAS；Faculty of Electrical Engineering, Czech Technical University in Prague
The Helton-Nie Conjecture (HNC) is the proposition that every convex semialgebraic set is a spectrahedral shadow. Here we prove that HNC is equivalent to another propo- sition related to quadratically constrained quadratic programming. Namely, that the convex hull of the rank-one elements of any spectrahedron is a spectrahedral shadow. In the case of compact convex semialgebraic sets, the spectrahedra may be taken to be compact. We illustrate the relationship between spetrahedra and these convex subsets with examples.
We consider the problem of the semidefinite representation of a class of non-compact basic semialgebraic sets. We introduce the conditions of pointedness and closedness at infinity of a semialgebraic set and show that under these conditions our modified hierarchies of nested theta bodies and Lasserre's relaxations converge to the closure of the convex hull of $S$. Moreover, if the PP-BDR property is satisfied, our theta body and Lasserre's relaxation are exact when the order is large enough; if the PP-BDR property does not hold, our hierarchies convergent uniformly to the closure of the convex…
Consider a finite system of non-strict real polynomial inequalities and suppose its solution set $S\subseteq\mathbb R^n$ is convex, has nonempty interior and is compact. Suppose that the system satisfies the Archimedean condition, which is slightly stronger than the compactness of $S$. Suppose that each defining polynomial satisfies a second order strict quasiconcavity condition where it vanishes on $S$ (which is very natural because of the convexity of $S$) or its Hessian has a certain matrix sums of squares certificate for negative-semidefiniteness on $S$ (fulfilled trivially by linear polyn…
In this paper we show that Sum-of-Squares optimization can be used to find optimal semialgebraic representations of sets. These sets may be explicitly defined, as in the case of discrete points or unions of sets; or implicitly defined, as in the case of attractors of nonlinear systems. We define optimality in the sense of minimum volume, while satisfying constraints that can include set containment, convexity, or Lyapunov stability conditions. Our admittedly heuristic approach to volume minimization is based on the use of a determinantlike objective function. We provide numerical examples for …
We address the long-standing problem of computing the region of attraction (ROA) of a target set (e.g., a neighborhood of an equilibrium point) of a controlled nonlinear system with polynomial dynamics and semialgebraic state and input constraints. We show that the ROA can be computed by solving an infinite-dimensional convex linear programming (LP) problem over the space of measures. In turn, this problem can be solved approximately via a classical converging hierarchy of convex finite-dimensional linear matrix inequalities (LMIs). Our approach is genuinely primal in the sense that convexity …
The free closed semialgebraic set $D_f$ determined by a hermitian noncommutative polynomial $f$ is the closure of the connected component of $\{(X,X^*)\mid f(X,X^*)>0\}$ containing the origin. When $L$ is a hermitian monic linear pencil, the free closed semialgebraic set $D_L$ is the feasible set of the linear matrix inequality $L(X,X^*)\geq 0$ and is known as a free spectrahedron. Evidently these are convex and it is well-known that a free closed semialgebraic set is convex if and only it is a free spectrahedron. The main result of this paper solves the basic problem of determining those $f$ …
Motivated by applications in robotics and computer vision, we study problems related to spatial reasoning of a 3D environment using sublevel sets of polynomials. These include: tightly containing a cloud of points (e.g., representing an obstacle) with convex or nearly-convex basic semialgebraic sets, computation of Euclidean distances between two such sets, separation of two convex basic semalgebraic sets that overlap, and tight containment of the union of several basic semialgebraic sets with a single convex one. We use algebraic techniques from sum of squares optimization that reduce all the…
Simple Approximations of Semialgebraic Sets and their Applications to Control
CNR-IEIIT（意大利国家研究委员会工业工程研究所）；Politecnico di Torino（都灵理工大学）；CNRS（法国国家科学研究中心）；LAAS（图卢兹系统分析与实验室）；Université de Toulouse（图卢兹大学）；Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克理工大学电气工程学院）；Electrical Engineering Department, The Pennsylvania State University（宾夕法尼亚州立大学电气工程系）
We characterize the maximum controlled invariant (MCI) set for discrete- as well as continuous-time nonlinear dynamical systems as the solution of an infinite-dimensional linear programming problem. For systems with polynomial dynamics and compact semialgebraic state and control constraints, we describe a hierarchy of finite-dimensional linear matrix inequality (LMI) relaxations whose optimal values converge to the volume of the MCI set; dual to these LMI relaxations are sum-of-squares (SOS) problems providing a converging sequence of outer approximations to the MCI set. The approach is simple…
In this paper, we introduce a new class of nonsmooth convex functions called SOS-convex semialgebraic functions extending the recently proposed notion of SOS-convex polynomials. This class of nonsmooth convex functions covers many common nonsmooth functions arising in the applications such as the Euclidean norm, the maximum eigenvalue function and the least squares functions with $\ell_1$-regularization or elastic net regularization used in statistics and compressed sensing. We show that, under commonly used strict feasibility conditions, the optimal value and an optimal solution of SOS-convex…
We consider the problem of approximating the reachable set of a discrete-time polynomial system from a semialgebraic set of initial conditions under general semialgebraic set constraints. Assuming inclusion in a given simple set like a box or an ellipsoid, we provide a method to compute certified outer approximations of the reachable set. The proposed method consists of building a hierarchy of relaxations for an infinite-dimensional moment problem. Under certain assumptions, the optimal value of this problem is the volume of the reachable set and the optimum solution is the restriction of the …
In Hilbert space, we propose a family of primal-dual dynamical system for affine constrained convex optimization problem. Several damping coefficients, time scaling coefficients, and perturbation terms are thus considered. By constructing the energy functions, we investigate the convergence rates with different choices of the damping coefficients and time scaling coefficients. Our results extend the inertial dynamical approaches for unconstrained convex optimization problems to affine constrained convex optimization problems.
Convergence of a projected stochastic gradient algorithm is demonstrated for convex objective functionals with convex constraint sets in Hilbert spaces. In the convex case, the sequence of iterates ${u_n}$ converges weakly to a point in the set of minimizers with probability one. In the strongly convex case, the sequence converges strongly to the unique optimum with probability one. An application to a class of PDE constrained problems with a convex objective, convex constraint and random elliptic PDE constraints is shown. Theoretical results are demonstrated numerically.
This paper considers a general convex constrained problem setting where functions are not assumed to be differentiable nor Lipschitz continuous. Our motivation is in finding a simple first-order method for solving a wide range of convex optimization problems with minimal requirements. We study the method of weighted dual averages (Nesterov, 2009) in this setting and prove that it is an optimal method.
In this paper, we propose a mixed-binary convex quadratic programming reformulation for the box-constrained nonconvex quadratic integer program and then implement IBM ILOG CPLEX 12.6 to solve the new model. Computational results demonstrate that our approach clearly outperform the very recent state-of-the-art solvers.
We consider the semi-infinite system of polynomial inequalities of the form \[ \mathbf{K}:=\{x\in\mathbb{R}^m\mid p(x,y)\ge 0,\ \ \forall y\in S\subseteq\mathbb{R}^n\}, \] where $p(x,y)$ is a real polynomial in the variables $x$ and the parameters $y$, the index set $S$ is a basic semialgebraic set in $\mathbb{R}^n$, $-p(x,y)$ is convex in $x$ for every $y\in S$. We propose a procedure to construct approximate semidefinite representations of $\mathbf{K}$. There are two indices to index these approximate semidefinite representations. As two indices increase, these semidefinite representation se…
We present a method for solving linearly constrained convex optimization problems, which is based on the application of known algorithms for finding zeros of the sum of two monotone operators (presented by Eckstein and Svaiter) to the dual problem. We establish convergence rates for the new method, and we present applications to TV denoising and compressed sensing problems.
This paper shows that the OSGA algorithm -- which uses first-order information to solve convex optimization problems with optimal complexity -- can be used to efficiently solve arbitrary bound-constrained convex optimization problems. This is done by constructing an explicit method as well as an inexact scheme for solving the bound-constrained rational subproblem required by OSGA. This leads to an efficient implementation of OSGA on large-scale problems in applications arising signal and image processing, machine learning and statistics. Numerical experiments demonstrate the promising performa…
In this paper we study second-order optimality conditions for non-convex set-constrained optimization problems. For a convex set-constrained optimization problem, it is well-known that second-order optimality conditions involve the support function of the second-order tangent set. In this paper we propose two approaches for establishing second-order optimality conditions for the non-convex case. In the first approach we extend the concept of the support function so that it is applicable to general non-convex set-constrained problems, whereas in the second approach we introduce the notion of th…
Motivated by TRACE algorithm [Curtis et al. 2017], we propose a trust region algorithm for finding second order stationary points of a linearly constrained non-convex optimization problem. We show the convergence of the proposed algorithm to (ε_g, ε_H)-second order stationary points in \widetilde{\mathcal{O}}(\max{ε_g^{-3/2}, ε_H^{-3}}) iterations. This iteration complexity is achieved for general linearly constrained optimization without cubic regularization of the objective function.
Chance constraints are a valuable tool for the design of safe decisions in uncertain environments; they are used to model satisfaction of a constraint with a target probability. However, because of possible non-convexity and non-smoothness, optimizing over a chance constrained set is challenging. In this paper, we establish an exact reformulation of chance constrained problems as a bilevel problems with convex lower-levels. We then derive a tractable penalty approach, where the penalized objective is a difference-of-convex function that we minimize with a suitable bundle algorithm. We release …
In this paper, we propose a new inexact version of the projected subgradient method to solve nondifferentiable constrained convex optimization problems. The method combine $ε$-subgradient method with a procedure to obtain a feasible inexact projection onto the constraint set. Asymptotic convergence results and iteration-complexity bounds for the sequence generated by the method employing the well known exogenous stepsizes, Polyak's stepsizes, and dynamic stepsizes are established.
This paper considers a class of constrained convex stochastic composite optimization problems whose objective function is given by the summation of a differentiable convex component, together with a nonsmooth but convex component. The nonsmooth component has an explicit max structure that may not easy to compute its proximal mapping. In order to solve these problems, we propose a mini-batch stochastic Nesterov's smoothing (MSNS) method. Convergence and the optimal iteration complexity of the method are established. Numerical results are provided to illustrate the efficiency of the proposed MSN…
In this paper, we propose an inertial accelerated primal-dual method for the linear equality constrained convex optimization problem. When the objective function has a ``nonsmooth + smooth'' composite structure, we further propose an inexact inertial primal-dual method by linearizing the smooth individual function and solving the subproblem inexactly. Assuming merely convexity, we prove that the proposed methods enjoy $\mathcal{O}(1/k^2)$ convergence rate on the objective residual and the feasibility violation in the primal model. Numerical results are reported to demonstrate the validity of t…
Tractable hierarchies of convex relaxations for polynomial optimization on the nonnegative orthant
Institute of Mathematics, Vietnam Academy of Science and Technology（越南科技科学院数学研究所）；CNRS（法国国家科学研究中心（CNRS）、劳伦斯·亚当斯研究所（LAAS））；LAAS（图卢兹大学；劳伦斯·亚当斯研究所）；Université de Toulouse（新加坡国立大学数学系）；LAAS；Department of Mathematics, National University of Singapore
We investigate non-convex optimization problems in $BV(Ω)$ with two-sided pointwise inequality constraints. We propose a regularization and penalization method to numerically solve the problem. Under certain conditions, weak limit points of iterates are stationary for the original problem. In addition, we prove optimality conditions for the original problem that contain Lagrange multipliers to the inequality constraints. Numerical experiments confirm the theoretical findings.
We study constrained stochastic programs where the decision vector at each time slot cannot be chosen freely but is tied to the realization of an underlying random state vector. The goal is to minimize a general objective function subject to linear constraints. A typical scenario where such programs appear is opportunistic scheduling over a network of time-varying channels, where the random state vector is the channel state observed, and the control vector is the transmission decision which depends on the current channel state. We consider a primal-dual type Frank-Wolfe algorithm that has a lo…
In this paper, we establish the convergence properties for a majorized alternating direction method of multipliers (ADMM) for linearly constrained convex optimization problems whose objectives contain coupled functions. Our convergence analysis relies on the generalized Mean-Value Theorem which plays an important role to properly control the cross terms due to the presence of coupled objective functions. Our results in particular show that directly applying 2-block ADMM with a large step length to the linearly constrained convex optimization problem with a quadratically coupled objective funct…
In this paper, we study the solution uniqueness of an individual feasible vector of a class of convex optimization problems involving convex piecewise affine functions and subject to general polyhedral constraints. This class of problems incorporates many important polyhedral constrained $\ell_1$ recovery problems arising from sparse optimization, such as basis pursuit, LASSO, and basis pursuit denoising, as well as polyhedral gauge recovery. By leveraging the max-formulation of convex piecewise affine functions and convex analysis tools, we develop dual variables based necessary and sufficien…
Alternating Direction Method of Multipliers for nonlinear constrained convex problems and applications to distributed resource allocation and constrained machine learning
Although the classical LQR design method has been very successful in real world engineering designs, in some cases, the classical design method needs modifications because of the saturation in actuators. This modified problem is sometimes called the constrained LQR design. For discrete systems, the constrained LQR design problem is equivalent to a convex quadratic programming problem with box constraints. We will show that the interior-point method is very efficient for this problem because an initial interior point is available, a condition which is not true for general convex quadratic progr…
In this paper, we present new convex relaxations for nonconvex quadratically constrained quadratic programming (QCQP) problems. While recent research has focused on strengthening convex relaxations using reformulation-linearization technique (RLT), the state-of-the-art methods lose their effectiveness when dealing with (multiple) nonconvex quadratic constraints in QCQP. In this research, we decompose and relax each nonconvex constraint to two second order cone (SOC) constraints and then linearize the products of the SOC constraints and linear constraints to construct some effective new valid c…
This paper considers the fixed point problem for a nonexpansive mapping on a real Hilbert space and proposes novel line search fixed point algorithms to accelerate the search. The termination conditions for the line search are based on the well-known Wolfe conditions that are used to ensure the convergence and stability of unconstrained optimization algorithms. The directions to search for fixed points are generated by using the ideas of the steepest descent direction and conventional nonlinear conjugate gradient directions for unconstrained optimization. We perform convergence as well as conv…
We consider cooperative multi-agent consensus optimization problems over both static and time-varying communication networks, where only local communications are allowed. The objective is to minimize the sum of agent-specific possibly non-smooth composite convex functions over agent-specific private conic constraint sets; hence, the optimal consensus decision should lie in the intersection of these private sets. Assuming the sum function is strongly convex, we provide convergence rates in suboptimality, infeasibility and consensus violation; examine the effect of underlying network topology on…
Linear constrained convex programming has many practical applications, including support vector machine and machine learning portfolio problems. We propose the randomized primal-dual coordinate (RPDC) method, a randomized coordinate extension of the first-order primal-dual method by Cohen and Zhu, 1984 and Zhao and Zhu, 2019, to solve linear constrained convex programming. We randomly choose a block of variables based on a uniform distribution, linearize, and apply a Bregman-like function (core function) to the selected block to obtain simple parallel primal-dual decomposition. We then establi…
This paper presents a piecewise convexification method to approximate the whole approximate optimal solution set of non-convex optimization problems with box constraints. In the process of box division, we first classify the sub-boxes and only continue to divide only some sub-boxes in the subsequent division. At the same time, applying the $α$-based Branch-and-Bound ({\rm$α$BB}) method, we construct a series of piecewise convex relax sub-problems, which are collectively called the piecewise convexification problem of the original problem. Then, we define the (approximate) solution set of the p…
We consider the problem of finding an approximate second-order stationary point of a constrained non-convex optimization problem. We first show that, unlike the gradient descent method for unconstrained optimization, the vanilla projected gradient descent algorithm may converge to a strict saddle point even when there is only a single linear constraint. We then provide a hardness result by showing that checking $(ε_g,ε_H)$-second order stationarity is NP-hard even in the presence of linear constraints. Despite our hardness result, we identify instances of the problem for which checking second …
This work proposes an accelerated primal-dual dynamical system for affine constrained convex optimization and presents a class of primal-dual methods with nonergodic convergence rates. In continuous level, exponential decay of a novel Lyapunov function is established and in discrete level, implicit, semi-implicit and explicit numerical discretizations for the continuous model are considered sequentially and lead to new accelerated primal-dual methods for solving linearly constrained optimization problems. Special structures of the subproblems in those schemes are utilized to develop efficient …
Shape-constrained convex regression problem deals with fitting a convex function to the observed data, where additional constraints are imposed, such as component-wise monotonicity and uniform Lipschitz continuity. This paper provides a unified framework for computing the least squares estimator of a multivariate shape-constrained convex regression function in $\mathbb{R}^d$. We prove that the least squares estimator is computable via solving an essentially constrained convex quadratic programming (QP) problem with $(d+1)n$ variables, $n(n-1)$ linear inequality constraints and $n$ possibly non…
We develop two new proximal alternating penalty algorithms to solve a wide range class of constrained convex optimization problems. Our approach mainly relies on a novel combination of the classical quadratic penalty, alternating minimization, Nesterov's acceleration, and adaptive strategy for parameters. The first algorithm is designed to solve generic and possibly nonsmooth constrained convex problems without requiring any Lipschitz gradient continuity or strong convexity, while achieving the best-known $O(\frac{1}{k})$-convergence rate in a non-ergodic sense, where $k$ is the iteration coun…
We consider the problem of minimizing a finite sum of convex functions subject to the set of minimizers of a convex differentiable function. In order to solve the problem, an algorithm combining the incremental proximal gradient method with smooth penalization technique is proposed. We show the convergence of the generated sequence of iterates to an optimal solution of the optimization problems, provided that a condition expressed via the Fenchel conjugate of the constraint function is fulfilled. Finally, the functionality of the method is illustrated by some numerical experiments addressing i…
We introduce a novel primal-dual flow for affine constrained convex optimization problems. As a modification of the standard saddle-point system, our primal-dual flow is proved to possess the exponential decay property, in terms of a tailored Lyapunov function. Then two primal-dual methods are obtained from numerical discretizations of the continuous model, and global nonergodic linear convergence rate is established via a discrete Lyapunov function. Instead of solving the subproblem of the primal variable, we apply the semi-smooth Newton iteration to the subproblem with respect to the multipl…
Second-order dynamical systems are important tools for solving optimization problems, and most of existing works in this field have focused on unconstrained optimization problems. In this paper, we propose an inertial primal-dual dynamical system with constant viscous damping and time scaling for the linear equality constrained convex optimization problem, which consists of a second-order ODE for the primal variable and a first-order ODE for the dual variable. When the scaling satisfies certain conditions, we prove its convergence property without assuming strong convexity. Even the convergenc…
In a real Hilbert space setting, we reconsider the classical Arrow-Hurwicz differential system in view of solving linearly constrained convex minimization problems. We investigate the asymptotic properties of the differential system and provide conditions for which its solutions converge towards a saddle point of the Lagrangian associated with the convex minimization problem. Our convergence analysis mainly relies on a `Lagrangian identity' which naturally extends on the well-known descent property of the classical continuous steepest descent method. In addition, we present asymptotic estimate…
This paper focuses on stochastic proximal gradient methods for optimizing a smooth non-convex loss function with a non-smooth non-convex regularizer and convex constraints. To the best of our knowledge we present the first non-asymptotic convergence results for this class of problem. We present two simple stochastic proximal gradient algorithms, for general stochastic and finite-sum optimization problems, which have the same or superior convergence complexities compared to the current best results for the unconstrained problem setting. In a numerical experiment we compare our algorithms with t…
