This work addresses the occupation measure relaxation of calculus of variations problems, which is an infinite-dimensional linear programming relaxation amenable to numerical approximation by a hierarchy of semidefinite optimization problems. We address the problem of equivalence of this relaxation to the original problem. Our main result provides sufficient conditions for this equivalence. These conditions, revolving around the convexity of the data, are simple and apply in very general settings that may be of arbitrary dimensions and may include pointwise and integral constraints, thereby considerably strengthening the existing results. Our conditions are also extended to optimal control problems. In addition, we demonstrate how these results can be applied in non-convex settings, showing that the occupation measure relaxation is at least as strong as the convexification using the convex envelope; in doing so, we prove that a certain weakening of the occupation measure relaxation is equivalent to the convex envelope. This opens the way to application of the occupation measure relaxation in situations where the convex envelope relaxation is known to be equivalent to the original problem, which includes problems in magnetism and elasticity.
In this paper we introduce the Ameso optimization problem, a special class of discrete optimization problems. We establish its basic properties and investigate the relation between Ameso optimization and the convex optimization. Further, we design an algorithm to solve a multi-dimensional Ameso problem by solving a sequence of one-dimensional Ameso problems. Finally, we demonstrate how the knapsack problem can be solved using the Ameso optimization framework.
We consider the class of nonlinear optimal control problems (OCP) with polynomial data, i.e., the differential equation, state and control con- straints and cost are all described by polynomials, and more generally for OCPs with smooth data. In addition, state constraints as well as state and/or action constraints are allowed. We provide a simple hierarchy of LMI (lin- ear matrix inequality)-relaxations whose optimal values form a nondecreasing sequence of lower bounds on the optimal value. Under some convexity assump- tions, the sequence converges to the optimal value of the OCP. Preliminary …
This paper addresses the problem of solving a class of nonlinear optimal control problems (OCP) with infinite-dimensional linear state constraints involving Riesz-spectral operators. Each instance within this class has time/control dependent polynomial Lagrangian cost and control constraints described by polynomials. We first perform a state-mode discretization of the Riesz-spectral operator. Then, we approximate the resulting finite-dimensional OCPs by using a previously known hierarchy of semidefinite relaxations. Under certain compactness assumptions, we provide a converging hierarchy of se…
Obtaining initial conditions and parameterizations leading to a model consistent with available measurements or safety specifications is important for many applications. Examples include model (in-)validation, prediction, fault diagnosis, and controller design. We present an approach to determine inner- and outer-approximations of the set containing all consistent initial conditions/parameterizations for nonlinear continuous-time systems. These approximations are found by occupation measures that encode the system dynamics and measurements, and give rise to an infinite-dimensional linear progr…
In the first part of the paper, we consider a discrete-time stochastic control system. We show that, under certain conditions, the set of random occupational measures generated by the state-control trajectories of the system as well as the set of their mathematical expectations converge (as the time horizon tends to infinity) to a convex and compact (non-random) set, which is shown to coincide with the set of stationary probabilities of the system. In the second part, we apply the results obtained in the first part to deal with a hybrid system that evolves in continuous time and is subjected t…
Rank and cardinality penalties are hard to handle in optimization frameworks due to non-convexity and discontinuity. Strong approximations have been a subject of intense study and numerous formulations have been proposed. Most of these can be described as separable, meaning that they apply a penalty to each element (or singular value) based on size, without considering the joint distribution. In this paper we present a class of non-separable penalties and give a recipe for computing strong relaxations suitable for optimization. In our analysis of this formulation we first give conditions that …
We consider nonlinear optimal control problems (OCPs) for which all problem data are polynomial. In the first part of the paper, we review how occupation measures can be used to approximate pointwise the optimal value function of a given OCP, using a hierarchy of linear matrix inequality (LMI) relaxations. In the second part, we extend the methodology to approximate the optimal value function on a given set and we use such a function to constructively and computationally derive an almost optimal control law. Numerical examples show the effectiveness of the approach.
The gap between a variational problem and its occupation measure relaxation
CNRS（法国国家科学研究中心；系统分析与实验室）；LAAS（布拉格捷克理工大学电气工程学院）；Faculty of Electrical Engineering, Czech Technical University in Prague（挪威北极大学数学与统计系）；Department of Mathematics and Statistics, UiT The Arctic University of Norway
The standard assumption for proving linear convergence of first order methods for smooth convex optimization is the strong convexity of the objective function, an assumption which does not hold for many practical applications. In this paper, we derive linear convergence rates of several first order methods for solving smooth non-strongly convex constrained optimization problems, i.e. involving an objective function with a Lipschitz continuous gradient that satisfies some relaxed strong convexity condition. In particular, in the case of smooth constrained convex optimization, we provide several…
Point-set registration is a classical image processing problem that looks for the optimal transformation between two sets of points. In this work, we analyze the impact of outliers when finding the optimal rotation between two point clouds. The presence of outliers motivates the use of least unsquared deviation, which is a non-smooth minimization problem over non-convex domain. We compare approaches based on non-convex optimization over special orthogonal group and convex relaxations. We show that if the fraction of outliers is larger than a certain threshold, any naive convex relaxation fails…
In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of subgradient methods with the Polyak step-size, to the situation of inexact information about the value of the minimum, as well as the unknown Lipschitz constant of the objective function. Moreover, the use of local analogs of the global characteristics of the objective function makes it possible to apply the results of this type to wider classes of problems. We…
This paper deals with the unconstrained and constrained cases for continuous-time Markov decision processes under the finite-horizon expected total cost criterion. The state space is denumerable and the transition and cost rates are allowed to be unbounded from above and from below. We give conditions for the existence of optimal policies in the class of all randomized history-dependent policies. For the unconstrained case, using the analogue of the forward Kolmogorov equation in the form of conditional expectation, we show that the finite-horizon optimal value function is the unique solution …
We study compressible and incompressible nonlinear elasticity variational problems in a general context. Our main result gives a sufficient condition for an equilibrium to be a global energy minimizer, in terms of convexity properties of the pressure in the deformed configuration. We also provide a convex relaxation of the problem together with its dual formulation, based on measure-valued mappings, which coincides with the original problem under our condition.
This article investigates the numerical approximation of shape optimization problems with PDE constraint on classes of convex domains. The convexity constraint provides a compactness property which implies well posedness of the problem. Moreover, we prove the convergence of discretizations in two-dimensional situations. A numerical algorithm is devised that iteratively solves the discrete formulation. Numerical experiments show that optimal convex shapes are generally non-smooth and that three-dimensional problems require an appropriate relaxation of the convexity condition.
Finite convergence of the Moment-SOS hierarchy under hidden convexity
CNRS（法国国家科学研究中心）；Nanyang Technological University（南洋理工大学）；LAAS-CNRS（法国国家科学研究中心自动分析与控制研究所）；Toulouse School of Economics (TSE)（图卢兹经济学院）；University of Toulouse（图卢兹大学）
We present a generalization of the notion of neighborliness to non-polyhedral convex cones. Although a definition of neighborliness is available in the non-polyhedral case in the literature, it is fairly restrictive as it requires all the low-dimensional faces to be polyhedral. Our approach is more flexible and includes, for example, the cone of positive-semidefinite matrices as a special case (this cone is not neighborly in general). We term our generalization Terracini convexity due to its conceptual similarity with the conclusion of Terracini's lemma from algebraic geometry. Polyhedral cone…
This paper aims to find efficient solutions to a multi-objective optimization problem (MP) with convex polynomial data. To this end, a hybrid method, which allows us to transform problem (MP) into a scalar convex polynomial optimization problem (P$_z$) and does not destroy the properties of convexity, is considered. First, we show an existence result for efficient solutions to problem (MP) under some mild assumption. Then, for problem (P$_z$), we establish two kinds of representations of non-negativity of convex polynomials over convex semi-algebraic sets, and propose two kinds of finite conve…
Technion - Israel Institute of Technology（以色列理工学院）
We consider convex optimization problems which are widely used as convex relaxations for low-rank matrix recovery problems. In particular, in several important problems, such as phase retrieval and robust PCA, the underlying assumption in many cases is that the optimal solution is rank-one. In this paper we consider a simple and natural sufficient condition on the objective so that the optimal solution to these relaxations is indeed unique and rank-one. Mainly, we show that under this condition, the standard Frank-Wolfe method with line-search (i.e., without any tuning of parameters whatsoever…
We propose a method for verifying that a given feasible point for a polynomial optimization problem is globally optimal. The approach relies on the Lasserre hierarchy and the result of Lasserre regarding the importance of the convexity of the feasible set as opposed to that of the individual constraints. By focusing solely on certifying global optimality and relaxing the Lasserre hierarchy using necessary conditions for positive semidefiniteness based on matrix determinants, the proposed method is implementable as a computationally tractable linear program. We demonstrate this method via appli…
This paper considers the problem of finding a low rank matrix from observations of linear combinations of its elements. It is well known that if the problem fulfills a restricted isometry property (RIP), convex relaxations using the nuclear norm typically work well and come with theoretical performance guarantees. On the other hand these formulations suffer from a shrinking bias that can severely degrade the solution in the presence of noise. In this theoretical paper we study an alternative non-convex relaxation that in contrast to the nuclear norm does not penalize the leading singular value…
We review several (and provide new) results on the theory of moments, sums of squares and basic semi-algebraic sets when convexity is present. In particular, we show that under convexity, the hierarchy of semidefinite relaxations for polynomial optimization simplifies and has finite convergence, a highly desirable feature as convex problems are in principle easier to solve. In addition, if a basic semi-algebraic set K is convex but its defining polynomials are not, we provide a certificate of convexity if a sufficient (and almost necessary) condition is satified. This condition can be checked …
Optimal power flow (OPF) is an important problem in the operation of electric power systems. Due to the OPF problem's non-convexity, there may exist multiple local optima. Certifiably obtaining the global solution is important for certain applications of OPF problems. Many global optimization techniques compute an optimality gap that compares the achievable objective value corresponding to the feasible point from a local solution algorithm with the objective value bound from a convex relaxation technique. Rather than the traditional practice of completely separating the local solution and conv…
We formulate bang-bang, purification, and minimization principles in dual Banach spaces with Gelfand integrals and provide a complete characterization of the saturation property of finite measure spaces. We also present a new application of the relaxation technique to large economies with infinite-dimensional commodity spaces, where the space of agents is modeled as a finite measure space. We propose a "relaxation" of large economies, which is regarded as a reasonable convexification of original economies. Under the saturation hypothesis, the relaxation and purification techniques enable us to…
In this paper, the convergence of alternating minimization is established for non-smooth convex optimization in Banach spaces, and novel rates of convergence are provided. As objective function a composition of a smooth and a non-smooth part is considered with the latter being block-separable, e.g., corresponding to convex constraints or regularization. For the smooth part, three different relaxations of strong convexity are considered: (i) quasi-strong convexity; (ii) quadratic functional growth; and (iii) plain convexity. Linear convergence is established for the first two cases, generalizin…
Optimal power flow (OPF) is a key problem in power system operations. OPF problems that use the nonlinear AC power flow equations to accurately model the network physics have inherent challenges associated with non-convexity. To address these challenges, recent research has applied various convex relaxation approaches to OPF problems. The QC relaxation is a promising approach that convexifies the trigonometric and product terms in the OPF problem by enclosing these terms in convex envelopes. The accuracy of the QC relaxation strongly depends on the tightness of these envelopes. This paper pres…
Signomial programs (SPs) are optimization problems specified in terms of signomials, which are weighted sums of exponentials composed with linear functionals of a decision variable. SPs are non-convex optimization problems in general, and families of NP-hard problems can be reduced to SPs. In this paper we describe a hierarchy of convex relaxations to obtain successively tighter lower bounds of the optimal value of SPs. This sequence of lower bounds is computed by solving increasingly larger-sized relative entropy optimization problems, which are convex programs specified in terms of linear an…
{We consider alternating minimization procedures for convex optimization problems with variable divided in many block, each block being amenable for minimization with respect to its variable with freezed other variables blocks. In the case of two blocks, we prove a linear convergence rate for alternating minimization procedure under Polyak-Lojasiewicz condition, which can be seen as a relaxation of the strong convexity assumption. Under strong convexity assumption in many-blocks setting we provide an accelerated alternating minimization procedure with linear rate depending on the square root o…
We consider several basic questions pertaining to the geometry of image of a general quadratic map. In general the image of a quadratic map is non-convex, although there are several known classes of quadratic maps when the image is convex. Remarkably, even when the image is not convex it often exhibits hidden convexity: a surprising efficiency of convex relaxation to address various geometric questions by reformulating them in terms of convex optimization problems. In this paper we employ this strategy and put forward several algorithms that solve the following problems pertaining to the image…
Optimal control problems involving hybrid binary-continuous control costs are challenging due to their lack of convexity and weak lower semicontinuity. Replacing such costs with their convex relaxation leads to a primal-dual optimality system that allows an explicit pointwise characterization and whose Moreau-Yosida regularization is amenable to a semismooth Newton method in function space. This approach is especially suited for computing switching controls for partial differential equations. In this case, the optimality gap between the original functional and its relaxation can be estimated a…
This paper considers phase retrieval from the magnitude of 1D over-sampled Fourier measurements, a classical problem that has challenged researchers in various fields of science and engineering. We show that an optimal vector in a least-squares sense can be found by solving a convex problem, thus establishing a hidden convexity in Fourier phase retrieval. We also show that the standard semidefinite relaxation approach yields the optimal cost function value (albeit not necessarily an optimal solution) in this case. A method is then derived to retrieve an optimal minimum phase solution in polyno…
We study the use of the System Optimum (SO) Dynamic Traffic Assignment (DTA) problem to design optimal traffic flow controls for freeway networks as modeled by the Cell Transmission Model, using variable speed limit, ramp metering, and routing. We consider two optimal control problems: the DTA problem, where turning ratios are part of the control inputs, and the Freeway Network Control (FNC), where turning ratios are instead assigned exogenous parameters. It is known that relaxation of the supply and demand constraints in the cell-based formulations of the DTA problem results in a linear progr…
In stochastic optimization, particularly in evolutionary computation and reinforcement learning, the optimization of a function $f: Ω\to \mathbb{R}$ is often addressed through optimizing a so-called relaxation $θ\in Θ\mapsto \mathbb{E}_θ(f)$ of $f$, where $Θ$ resembles the parameters of a family of probability measures on $Ω$. We investigate the structure of such relaxations by means of measure theory and Fourier analysis, enabling us to shed light on the success of many associated stochastic optimization methods. The main structural traits we derive and that allow fast and reliable optimizati…
Semidefinite Programming (SDP) and Sums-of-Squares (SOS) relaxations have led to certifiably optimal non-minimal solvers for several robotics and computer vision problems. However, most non-minimal solvers rely on least-squares formulations, and, as a result, are brittle against outliers. While a standard approach to regain robustness against outliers is to use robust cost functions, the latter typically introduce other non-convexities, preventing the use of existing non-minimal solvers. In this paper, we enable the simultaneous use of non-minimal solvers and robust estimation by providing a g…
In many areas of applied mathematics, engineering, and social and natural sciences, decentralization of information is a key aspect determining how to approach a problem. In this review article, we study information structures in a probability theoretic and geometric context. We define information structures, place various topologies on them, and study closedness, compactness and convexity properties on the strategic measures induced by information structures and decentralized control/decision policies under varying degree of relaxations with regard to access to private or common randomness. U…
This work considers the problem of locating a single source from noisy range measurements to a set of nodes in a wireless sensor network. We propose two new techniques that we designate as Source Localization with Nuclear Norm (SLNN) and Source Localization with l1-norm (SL-l1), which extend to arbitrary real dimensions, including 3D, our prior work on 2D source localization formulated in the complex plane. Broadly, our approach is based on formulating a Maximum-Likelihood (ML) estimation problem for the source position, and then using convex relaxation techniques to obtain a semidefinite prog…
