Level-set methods for convex optimization are predicated on the idea that certain problems can be parameterized so that their solutions can be recovered as the limiting process of a root-finding procedure. This idea emerges time and again across a range of algorithms for convex problems. Here we demonstrate that strong duality is a necessary condition for the level-set approach to succeed. In the absence of strong duality, the level-set method identifies $ε$-infeasible points that do not converge to a feasible point as $ε$ tends to zero. The level-set approach is also used as a proof technique for establishing sufficient conditions for strong duality that are different from Slater's constraint qualification.
Let $E$ be a real vector space with dual space $E^*$ and let $C\subset E$ be a convex subset with more than one point. Let $f : C\to\mathbb{R}$ be a function satisfying a mild stability property at 'flat' points of the (relative) boundary of $C$. We show that $f$ is convex if and only if for some linear form $c^*$ on $E$ not constant on $C$, the function $f+λc^*$ is quasiconvex for all $λ\in\mathbb{R}$.
The original motivation for this paper was to provide an efficient quantitative analysis of convex infinite (or semi-infinite) inequality systems whose decision variables run over general infinite-dimensional (resp. finite-dimensional) Banach spaces and that are indexed by an arbitrary fixed set $J$. Parameter perturbations on the right-hand side of the inequalities are required to be merely bounded, and thus the natural parameter space is $l_{\infty}(J)$. Our basic strategy consists of linearizing the parameterized convex system via splitting convex inequalities into linear ones by using the …
The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection o…
We present new results on optimization problems where the involved functions are evenly convex. By means of a generalized conjugation scheme and the perturbation theory introduced by Rockafellar, we propose an alternative dual problem for a general optimization one defined on a separated locally convex topological space. Sufficient conditions for converse and total duality involving the even convexity of the perturbation function and $c$-subdifferentials are given. Formulae for the $c$-subdifferential and biconjugate of the objective function of a general optimization problem are provided, too…
We show that minimizers of convex functions subject to almost all linear perturbations are nondegenerate. An analogous result holds more generally, for lower-C^2 functions.
In a Hilbert space setting $\mathcal H$, we study the fast convergence properties as $t \to + \infty$ of the trajectories of the second-order differential equation $ \ddot{x}(t) + \fracα{t} \dot{x}(t) + \nabla Φ(x(t)) = g(t)$, where $\nablaΦ$ is the gradient of a convex continuously differentiable function $Φ: \mathcal H \to \mathbb R$, $α$ is a positive parameter, and $g: [t_0, + \infty[ \rightarrow \mathcal H$ is a "small" perturbation term. In this damped inertial system, the viscous damping coefficient $\fracα{t}$ vanishes asymptotically, but not too rapidly. For $α\geq 3$, and $\int_{t_0}…
We consider a parametric convex quadratic programming, CQP, relaxation for the quadratic knapsack problem, QKP. This relaxation maintains partial quadratic information from the original QKP by perturbing the objective function to obtain a concave quadratic term. The nonconcave part generated by the perturbation is then linearized by a standard approach that lifts the problem to the matrix space. We present a primal-dual interior point method to optimize the perturbation of the quadratic function, in a search for the tightest upper bound for the \QKPp. We prove that the same perturbation approa…
We propose a generalization of hamiltonian mechanics, as a hamiltonian inclusion with convex dissipation function. We obtain a dynamical version of the approach of Mielke to quasistatic rate-independent processes. Then we show that a class of models of dynamical brittle damage can be formulated in this setting.
Second-order continuous-time dissipative dynamical systems with viscous and Hessian driven damping have inspired effective first-order algorithms for solving convex optimization problems. While preserving the fast convergence properties of the Nesterov-type acceleration, the Hessian driven damping makes it possible to significantly attenuate the oscillations. To study the stability of these algorithms with respect to perturbations, we analyze the behaviour of the corresponding continuous systems when the gradient computation is subject to exogenous additive errors. We provide a quantitative an…
In order to solve the minimization of a nonsmooth convex function, we design an inertial second-order dynamic algorithm, which is obtained by approximating the nonsmooth function by a class of smooth functions. By studying the asymptotic behavior of the dynamic algorithm, we prove that each trajectory of it weakly converges to an optimal solution under some appropriate conditions on the smoothing parameters, and the convergence rate of the objective function values is o(t^-2). We also show that the algorithm is stable, that is, this dynamic algorithm with a perturbation term owns the same conv…
This article is devoted to the stability of error bounds (local and global) for semi-infinite convex constraint systems in Banach spaces. We provide primal characterizations of the stability of local and global error bounds when systems are subject to small perturbations. These characterizations are given in terms of the directional derivatives of the functions that enter into the definition of these systems. It is proved that the stability of error bounds is essentially equivalent to verifying that the optimal values of several minimax problems, defined in terms of the directional derivatives…
We consider the resource allocation problem and its numerical solution. The following constructions are demonstrated: 1) Walrasian price-adjustment mechanism for determining the equilibrium; 2) Decentralized role of the prices; 3) Slater's method for price restrictions (dual Lagrange multipliers); 4) A new mechanism for determining equilibrium prices, in which prices are fully controlled not by Center (Government), but by economic agents -- nodes (factories). In economic literature the convergence of the considered methods is only proved. In contrast, this paper provides an accurate analysis o…
In view of the minimization of a function which is the sum of a differentiable function $f$ and a convex function $g$ we introduce descent methods which can be viewed as produced by inexact auxiliary problem principleor inexact variable metric forward-backward algorithm. Assuming that the global objective function satisfies the Kurdyka-Lojasiewicz inequalitywe prove the convergence of the proposed algorithm weakening assumptions found in previous works.
We study a class of distributed convex constrained optimization problems where a group of agents aim to minimize the sum of individual objective functions while each desires that any information about its objective function is kept private. We prove the impossibility of achieving differential privacy using strategies based on perturbing the inter-agent messages with noise when the underlying noise-free dynamics are asymptotically stable. This justifies our algorithmic solution based on the perturbation of individual functions with Laplace noise. To this end, we establish a general framework fo…
In Hilbert space, we propose a family of primal-dual dynamical system for affine constrained convex optimization problem. Several damping coefficients, time scaling coefficients, and perturbation terms are thus considered. By constructing the energy functions, we investigate the convergence rates with different choices of the damping coefficients and time scaling coefficients. Our results extend the inertial dynamical approaches for unconstrained convex optimization problems to affine constrained convex optimization problems.
The current bottleneck of globally solving mixed-integer (non-convex) quadratically constrained problem (MIQCP) is still to construct strong but computationally cheap convex relaxations, especially when dense quadratic functions are present. We propose a cutting surface procedure based on multiple diagonal perturbations to derive strong convex quadratic relaxations for nonconvex quadratic problem with separable constraints. Our resulting relaxation does not use significantly more variables than the original problem, in contrast to many other relaxations based on lifting. The corresponding sepa…
Quadratically constrained quadratic programs (QCQPs) are a highly expressive class of nonconvex optimization problems. While QCQPs are NP-hard in general, they admit a natural convex relaxation via the standard (Shor) semidefinite program (SDP) relaxation. Towards understanding when this relaxation is exact, we study general QCQPs and their (projected) SDP relaxations. We present sufficient (and in some cases, also necessary) conditions for objective value exactness (the condition that the objective values of the QCQP and its SDP relaxation coincide) and convex hull exactness (the condition th…
By applying the perturbation function approach, we propose the Lagrangian and the conjugate duals for minimization problems of the sum of two, generally nonconvex, functions. The main tools are the $Φ$-convexity theory and minimax theorems for $Φ$-convex functions. We provide conditions ensuring zero duality gap and introduce $Φ$-Karush-Kuhn-Tucker conditions that characterize solutions to primal and dual problems. We also discuss the relationship between the dual problems introduced in the present investigation and some conjugate-type duals existing in the literature.
In this paper, we propose a second-order continuous primal-dual dynamical system with time-dependent positive damping terms for a separable convex optimization problem with linear equality constraints. By the Lyapunov function approach, we investigate asymptotic properties of the proposed dynamical system as the time $t\to+\infty$. The convergence rates are derived for different choices of the damping coefficients. We also show that the obtained results are robust under external perturbations.
Using tools provided by the theory of abstract convexity, we extend conditions for zero duality gap to the context of nonconvex and nonsmooth optimization. Mimicking the classical setting, an abstract convex function is the upper envelope of a family of abstract affine functions (being conventional vertical translations of the abstract linear functions). We establish new conditions for zero duality gap under no topological assumptions on the space of abstract linear functions. In particular, we prove that the zero duality gap property can be fully characterized in terms of an inclusion involvi…
This paper addresses Online Convex Optimization (OCO) problems where the constraints have additive perturbations that (i) vary over time and (ii) are not known at the time to make a decision. Perturbations may not be i.i.d. generated and can be used to model a time-varying budget or commodity in resource allocation problems. The problem is to design a policy that obtains sublinear regret while ensuring that the constraints are satisfied on average. To solve this problem, we present a primal-dual proximal gradient algorithm that has $O(T^ε\vee T^{1-ε})$ regret and $O(T^ε)$ constraint violation,…
We consider maximizing a continuous convex function over the assignment polytope. Such problems arise in Graph Matching (the optimization version of Graph Isomorphism) and Quadratic Assignment problems. In the typical case of maximizing a convex function over a polytope the problem can be solved by using a simplicial algorithm such as Tuy's method or the Falk-Hoffman method, but these algorithms require that the underlying polytope be non-degenerate, which is not the case for the assignment polytope. In this note we show how a simple perturbation scheme can be used to create a "surrogate probl…
This paper analyzes the Lipschitz behavior of the feasible set in two parametric settings, associated with linear and convex systems in R^n. To start with, we deal with the parameter space of linear (finite/semi-infinite) systems identified with the corresponding sets of coefficient vectors, which are assumed to be closed subsets of R^(n+1). In this framework, where the Hausdorff distance is used to measure the size of perturbations, an explicit formula for computing the Lipschitz modulus of the feasible set mapping is provided. As direct antecedent, we appeal to its counterpart in the paramet…
This paper provides a theoretical and numerical comparison of classical first-order splitting methods for solving smooth convex optimization problems and cocoercive equations. From a theoretical point of view, we compare convergence rates of gradient descent, forward-backward, Peaceman-Rachford, and Douglas-Rachford algorithms for minimizing the sum of two smooth convex functions when one of them is strongly convex. A similar comparison is given in the more general cocoercive setting under the presence of strong monotonicity and we observe that the convergence rates in optimization are strictl…
In this article a family of second order ODEs associated to inertial gradient descend is studied. These ODEs are widely used to build trajectories converging to a minimizer $x^*$ of a function $F$, possibly convex. This family includes the continuous version of the Nesterov inertial scheme and the continuous heavy ball method. Several damping parameters, not necessarily vanishing, and a perturbation term $g$ are thus considered. The damping parameter is linked to the inertia of the associated inertial scheme and the perturbation term $g$ is linked to the error that can be done on the gradient …
We construct a two-sided discontinuous piecewise linear minimal valid function for the 1-row Gomory--Johnson model which is not extreme, but which is not a convex combination of other piecewise linear minimal valid functions. The new function only admits piecewise microperiodic perturbations. We present an algorithm for computations with a restricted class of such perturbations.
This article presents an optimization model for tertiary control in three-phase unbalanced microgrids. This model considers 24h operation and includes renewable energy sources, energy storage devices, and grid code limitations. Power flow equations are simplified using a recently developed approximation based on Wirtinger's calculus. The proposed model is evaluated both theoretically and practically. From the theoretical point of view, the model is suitable for tertiary control since it is convex; hence, global optimum, uniqueness of the solution, and convergence of the interior point method a…
The primal problem of multinomial likelihood maximization restricted to a convex closed subset of the probability simplex is studied. Contrary to widely held belief, a solution of this problem may assign a positive mass to an outcome with zero count. Related flaws in the simplified Lagrange and Fenchel dual problems, which arise because the recession directions are ignored, are identified and corrected. A solution of the primal problem can be obtained by the PP (perturbed primal) algorithm, that is, as the limit of a sequence of solutions of perturbed primal problems. The PP algorithm may be i…
Online Non-convex Optimization with Long-term Non-convex Constraints
Department of Electrical and Computer Engineering（电气与计算机工程系）；Johns Hopkins University（约翰霍普金斯大学）；International Business School（国际商务学院）；Suzhou Xi’an Jiaotong-Liverpool University（苏州市西安交通大学利物浦大学）；Department of Data and Systems Engineering（数据与系统工程系）；HKU Musketeers Foundation（HKU Musketeers基金会）；Institute of Data Science（数据科学研究所）；The University of Hong Kong（香港大学）
Convex-Analytic View of Query Selection for Safety Verification of Markov Decision Processes
The Institute of Statistical Mathematics（统计数学研究所）；Tohoku University（东北大学）；RIKEN AIP（理化学研究所人工智能中心）；CyberAgent；National Institute of Informatics（信息学国立研究所）
IMT School for Advanced Studies, Lucca, Italy（IMT卢卡高等研究院）；University of Genoa, Italy（热那亚大学）
In this short paper, a matrix perturbation bound on the eigenvalues found by principal component analysis is investigated, for the case in which the data matrix on which principal component analysis is performed is a convex combination of two data matrices. The application of the theoretical analysis to multi-objective optimization problems (e.g., those arising in the design of acoustic metamaterial filters) is briefly discussed, together with possible extensions.
When equipped with efficient optimization algorithms, the over-parameterized neural networks have demonstrated high level of performance even though the loss function is non-convex and non-smooth. While many works have been focusing on understanding the loss dynamics by training neural networks with the gradient descent (GD), in this work, we consider a broad class of optimization algorithms that are commonly used in practice. For example, we show from a dynamical system perspective that the Heavy Ball (HB) method can converge to global minimum on mean squared error (MSE) at a linear rate (sim…
