In this paper, we provide an elementary, geometric, and unified framework to analyze conic programs that we call the strict complementarity approach. This framework allows us to establish error bounds and quantify the sensitivity of the solution. The framework uses three classical ideas from convex geometry and linear algebra: linear regularity of convex sets, facial reduction, and orthogonal decomposition. We show how to use this framework to derive error bounds for linear programming (LP), second order cone programming (SOCP), and semidefinite programming (SDP).
Although the Karush-Kuhn-Tucker conditions suggest a connection between a conic optimization problem and a complementarity problem, it is difficult to find an accessible explicit form of this relationship in the literature. This note will present such a relationship.
Consider a homogeneous multifold convex conic system $$ Ax = 0, \; x\in K_1\times...\times K_r $$ and its alternative system $$ A\transp y \in K_1^*\times...\times K_r^*, $$ where $K_1,..., K_r$ are regular closed convex cones. We show that there is canonical partition of the index set ${1,...,r}$ determined by certain complementarity sets associated to the most interior solutions to the two systems. Our results are inspired by and extend the Goldman-Tucker Theorem for linear programming.
In this paper, we prove that all H$^+$(Z$^+$)-eigenvalues of each principal sub-tensor of a strictly semi-positive tensor are positive. We define two new constants associated with H$^+$(Z$^+$)eigenvalues of a strictly semi-positive tensor. With the help of these two constants, we establish upper bounds of an important quantity whose positivity is a necessary and sufficient condition for a general tensor to be a strictly semi-positive tensor. The monotonicity and boundedness of such a quantity are established too. Furthermore, we present global error bound analysis for a class of the nonlinear …
The MaxCut SDP is one of the most well-known semidefinite programs, and it has many favorable properties. One of its nicest geometric/duality properties is the fact that the vertices of its feasible region correspond exactly to the cuts of a graph, as proved by Laurent and Poljak in 1995. Recall that a boundary point $x$ of a convex set $C$ is called a vertex of $C$ if the normal cone of $C$ at $x$ is full-dimensional. We study how often strict complementarity holds or fails for the MaxCut SDP when a vertex of the feasible region is optimal, i.e., when the SDP relaxation is tight. While strict…
In this paper, we accomplish a unified convergence analysis of a second-order method of multipliers (i.e., a second-order augmented Lagrangian method) for solving the conventional nonlinear conic optimization problems.Specifically, the algorithm that we investigated incorporates a specially designed nonsmooth (generalized) Newton step to furnish a second-order update rule for the multipliers.We first show in a unified fashion that under a few abstract assumptions, the proposed method is locally convergent and possesses a (nonasymptotic) superlinear convergence rate, even though the penalty par…
We develop a novel variant of the classical Frank-Wolfe algorithm, which we call spectral Frank-Wolfe, for convex optimization over a spectrahedron. The spectral Frank-Wolfe algorithm has a novel ingredient: it computes a few eigenvectors of the gradient and solves a small-scale SDP in each iteration. Such procedure overcomes slow convergence of the classical Frank-Wolfe algorithm due to ignoring eigenvalue coalescence. We demonstrate that strict complementarity of the optimization problem is key to proving linear convergence of various algorithms, such as the spectral Frank-Wolfe algorithm as…
In this paper, using an optimal partition approach, we study the parametric analysis of a second-order conic optimization problem, where the objective function is perturbed along a fixed direction. We characterize the notions of so-called invariancy set and nonlinearity interval, which serve as stability regions of the optimal partition. We then propose, under the strict complementarity condition, an iterative procedure to compute a nonlinearity interval of the optimal partition. Furthermore, under primal and dual nondegeneracy conditions, we show that a boundary point of a nonlinearity interv…
We address the inverse problem of Lagrangian identification based on trajecto-ries in the context of nonlinear optimal control. We propose a general formulation of the inverse problem based on occupation measures and complementarity in linear programming. The use of occupation measures in this context offers several advan-tages from the theoretical, numerical and statistical points of view. We propose an approximation procedure for which strong theoretical guarantees are available. Finally, the relevance of the method is illustrated on academic examples.
Slater's condition -- existence of a "strictly feasible solution" -- is a common assumption in conic optimization. Without strict feasibility, first-order optimality conditions may be meaningless, the dual problem may yield little information about the primal, and small changes in the data may render the problem infeasible. Hence, failure of strict feasibility can negatively impact off-the-shelf numerical methods, such as primal-dual interior point methods, in particular. New optimization modelling techniques and convex relaxations for hard nonconvex problems have shown that the loss of strict…
In this paper we give a unified treatment of two different definitions of complementarity partition of multifold conic programs introduced independently in [J. F. Bonnans and H. Ramírez C., Math. Program. 104 (2005), no. 2-3, Ser. B, 205--227] for conic optimization problems, and in [J. Peña and V. Roshchina, Math. Program. 142 (2013), no 1-2, Ser. A, 579--589] for homogeneous feasibility problems. We show that both can be treated within the same unified geometric framework, and extend the latter notion to optimization problems. We also show that the two partitions do not coincide, and their i…
In this paper, we show that the subadditive dual of a feasible conic mixed-integer program (MIP) is a strong dual whenever it is feasible. Moreover, we show that this dual feasibility condition is equivalent to feasibility of the conic dual of the continuous relaxation of the conic MIP. In addition, we prove that all known conditions and other 'natural' conditions for strong duality, such as strict mixed-integer feasibility, boundedness of the feasible set or essentially strict feasibility imply that the subadditive dual is feasible. As an intermediate result, we extend the so-called 'finitene…
In this paper, one of our main purposes is to prove the boundedness of solution set of tensor complementarity problem with B tensor such that the specific bounds only depend on the structural properties of tensor. To achieve this purpose, firstly, we present that each B tensor is strictly semi-positive and each B$_0$ tensor is semi-positive. Subsequencely, the strictly lower and upper bounds of different operator norms are given for two positively homogeneous operators defined by B tensor. Finally, with the help of the upper bounds of different operator norms, we show the strcitly lower bound …
We consider the conic linear program given by a closed convex cone in an Euclidean space and a matrix, where vector on the right-hand-side of the constraint system and the vector defining the objective function are subject to change. Using the strict feasibility condition, we prove the locally Lipschitz continuity and obtain some differentiability properties of the optimal value function of the problem under right-hand-side perturbations. For the optimal value function under linear perturbations of the objective function, similar differentiability properties are obtained under the assumption s…
In this paper, we first introduce R0-W and SSM-W properties for the set of matrices which is a generalization of R0 and the strictly semimonotone matrix. We then prove some existence results for the extended horizontal linear complementarity problem when the involved matrices have these properties. With an additional condition on the set of matrices, we prove that the SSM-W property is equivalent to the unique solution for the corresponding extended horizontal linear complementarity problems. Finally, we give a necessary and sufficient condition for the connectedness of the solution set of the…
Second-order necessary optimality conditions for nonlinear conic programming problems that depend on a single Lagrange multiplier are usually built under nondegeneracy and strict complementarity. In this paper we establish a condition of such type for two classes of nonlinear conic problems, namely semidefinite and second-order cone programming, assuming Robinson's constraint qualification and a weak constant rank-type property which are, together, strictly weaker than nondegeneracy. Our approach is done via a penalty-based strategy, which is aimed at providing strong global convergence result…
The tensor complementarity problem is a specially structured nonlinear complementarity problem, then it has its particular and nice properties other than ones of the classical nonlinear complementarity problem. In this paper, it is proved that a tensor is an S-tensor if and only if the tensor complementarity problem is feasible, and each Q-tensor is an S-tensor. Furthermore, the boundedness of solution set of the tensor complementarity problem is equivalent to the uniqueness of solution for such a problem with zero vector. For the tensor complementarity problem with a strictly semi-positive te…
On the Local Linear Convergence of Operator Splitting Methods for Conic Programming
University of California San Diego（加州大学圣地亚哥分校）；MIT, Sloan School of Management（麻省理工学院斯隆管理学院）；University of Chicago, Department of Statistics（芝加哥大学统计学系）
Technion - Israel Institute of Technology（以色列理工学院）
In recent years it was proved that simple modifications of the classical Frank-Wolfe algorithm (aka conditional gradient algorithm) for smooth convex minimization over convex and compact polytopes, converge with linear rate, assuming the objective function has the quadratic growth property. However, the rate of these methods depends explicitly on the dimension of the problem which cannot explain their empirical success for large scale problems. In this paper we first demonstrate that already for very simple problems and even when the optimal solution lies on a low-dimensional face of the polyt…
Augmented Lagrangian methods for nonlinear semidefinite programming with complementarity constraints
Federal University of São Paulo（圣保罗联邦大学）；State University of Maringá（马林加州立大学）；Federal University of Paraná（巴拉那联邦大学）；Instituto Federal de Santa Catarina (IFSC)（圣卡塔琳娜联邦学院）
This paper is devoted to studying the robust isolated calmness of the Karush-Kuhn-Tucker (KKT) solution mapping for a large class of interesting conic programming problems (including most commonly known ones arising from applications) at a locally optimal solution. Under the Robinson constraint qualification, we show that the KKT solution mapping is robustly isolated calm if and only if both the strict Robinson constraint qualification and the second order sufficient condition hold. This implies, among others, that at a locally optimal solution the second order sufficient condition is needed f…
Technion - Israel Institute of Technology（以色列理工学院）
We consider optimization problems in which the goal is find a $k$-dimensional subspace of $\mathbb{R}^n$, $k<<n$, which minimizes a convex and smooth loss. Such problems generalize the fundamental task of principal component analysis (PCA) to include robust and sparse counterparts, and logistic PCA for binary data, among others. This problem could be approached either via nonconvex gradient methods with highly-efficient iterations, but for which arguing about fast convergence to a global minimizer is difficult or, via a convex relaxation for which arguing about convergence to a global minimize…
We present a short step interior point method for solving a class of nonlinear programming problems with quadratic objective function. Convex quadratic programming problems can be reformulated as problems in this class. The method is shown to have weak polynomial time complexity. A complete proof of the numerical stability of the method is provided. No requirements on feasibility, row-rank of the constraint Jacobian, strict complementarity, or conditioning of the problem are made. Infeasible problems are solved to an optimal interior least-squares solution.
A primal--dual interior-point method for nonsymmetric conic optimization with conjugate-free scaling
Nankai University（南开大学）；Suzhou University of Science and Technology（苏州科技大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）
We consider the minimization of a continuous function over the intersection of a regular cone with an affine set via a new class of adaptive first- and second-order optimization methods, building on the Hessian-barrier techniques introduced in [Bomze, Mertikopoulos, Schachinger, and Staudigl, Hessian barrier algorithms for linearly constrained optimization problems, SIAM Journal on Optimization, 2019]. Our approach is based on a potential-reduction mechanism and attains a suitably defined class of approximate first- or second-order KKT points with the optimal worst-case iteration complexity $O…
Relaxed constant positive linear dependence constraint qualification (RCPLD) for a system of smooth equalities and inequalities is a constraint qualification that is weaker than the usual constraint qualifications such as Mangasarian Fromovitz constraint qualification and the linear constraint qualification. Moreover RCPLD is known to induce an error bound property. In this paper we extend RCPLD to a very general feasibility system which may include Lipschitz continuous inequality constraints, complementarity constraints and abstract constraints. We show that this RCPLD for the general system …
In order to express a polyhedron as the (Minkowski) sum of a polytope and a polyhedral cone, Motzkin (1936) made a transition from the polyhedron to a polyhedral cone. Based on his excellent idea, we represent a set by a characteristic cone. By using this representation, we then reach four main results: (i) expressing a closed convex set containing no line as the direct sum of the convex hull of its extreme points and conical hull of its extreme directions, (ii) establishing a convex programming (CP) based framework for determining a maximal element-an element with the maximum number of positi…
Linear complementarity problems are a powerful tool for modeling many practically relevant situations such as market equilibria. They also connect many sub-areas of mathematics like game theory, optimization, and matrix theory. Despite their close relation to optimization, the protection of LCPs against uncertainties -- especially in the sense of robust optimization -- is still in its infancy. During the last years, robust LCPs have only been studied using the notions of strict and $Γ$-robustness. Unfortunately, both concepts lead to the problem that the existence of robust solutions cannot be…
Currently, the simplex method and the interior point method are indisputably the most popular algorithms for solving linear programs, LPs. Unlike general conic programs, LPs with a finite optimal value do not require strict feasibility in order to establish strong duality. Hence strict feasibility is seldom a concern, even though strict feasibility is equivalent to stability and a compact dual optimal set. This lack of concern is also true for other types of degeneracy of basic feasible solutions in LP. In this paper we discuss that the specific degeneracy that arises from lack of strict feasi…
De Klerk and Pasechnik (2002) introduced the bounds $\vartheta^{(r)}(G)$ ($r\in \mathbb{N}$) for the stability number $α(G)$ of a graph $G$ and conjectured exactness at order $α(G)-1$: $\vartheta^{(α(G)-1)}(G)=α(G)$. These bounds rely on the conic approximations $\mathcal{K}_n^{(r)}$ by Parrilo (2000) for the copositive cone $\text{COP}_n$. A difficulty in the convergence analysis of $\vartheta^{(r)}$ is the bad behaviour of the cones $\mathcal{K}_n^{(r)}$ under adding a zero row/column: when applied to a matrix not in $\mathcal{K}^{(0)}_n$ this gives a matrix not in any ${\mathcal{K}}^{(r)}_{…
In this paper, we aim to provide a comprehensive analysis on the linear rate convergence of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex composite optimization problems. Under a certain error bound condition, we establish the global linear rate of convergence for a more general semi-proximal ADMM with the dual steplength being restricted to be in the open interval $(0, (1+\sqrt{5})/2)$. In our analysis, we assume neither the strong convexity nor the strict complementarity except an error bound condition, which holds automatically for convex com…
In this paper, a class of optimization problems with nonlinear inequality constraints is discussed. Based on the ideas of sequential quadratic programming algorithm and the method of strongly sub-feasible directions, a new superlinearly convergent algorithm is proposed. The initial iteration point can be chosen arbitrarily for the algorithm. At each iteration, the new algorithm solves one quadratic programming subproblem which is always feasible, and one or two systems of linear equations with a common coefficient matrix. Moreover, the coefficient matrix is uniformly nonsingular. After finite …
We propose and analyse primal-dual interior-point algorithms for convex optimization problems in conic form. The families of algorithms we analyse are so-called short-step algorithms and they match the current best iteration complexity bounds for primal-dual symmetric interior-point algorithm of Nesterov and Todd, for symmetric cone programming problems with given self-scaled barriers. Our results apply to any self-concordant barrier for any convex cone. We also prove that certain specializations of our algorithms to hyperbolic cone programming problems (which lie strictly between symmetric co…
The optimization problems with simple bounds are an important class of problems. To facilitate the computation of such problems, an unconstrained-like dynamic method, motivated by the Lyapunov control principle, is proposed. This method employs the anti-windup limited integrator to address the bounds of parameters upon the dynamics for unconstrained problem, and then solves the transformed Initial-value Problems (IVPs) with mature Ordinary Differential Equation (ODE) integration methods. It is proved that when the gain matrix is diagonal, the result is equivalent to that of the general dynamic…
