It is an open question to determine if the theory of self-concordant barriers can provide an interior point method with strongly polynomial complexity in linear programming. In the special case of the logarithmic barrier, it was shown in [Allamigeon, Benchimol, Gaubert and Joswig, SIAM J. on Applied Algebra and Geometry, 2018] that the answer is negative. In this paper, we show that none of the self-concordant barrier interior point methods is strongly polynomial. This result is obtained by establishing that, on parametric families of convex optimization problems, the log-limit of the central path degenerates to a piecewise linear curve, independently of the choice of the barrier function. We provide an explicit linear program that falls in the same class as the Klee-Minty counterexample, i.e., in dimension $n$ with $2n$ constraints, in which the number of iterations is $Ω(2^n)$.
Massachusetts Institute of Technology (MIT)（麻省理工学院）
For any convex body $K \subseteq \mathbb R^n$, S. Bubeck and R. Eldan introduced the entropic barrier on $K$ and showed that it is a $(1+o(1)) \, n$-self-concordant barrier. In this note, we observe that the optimal bound of $n$ on the self-concordance parameter holds as a consequence of the dimensional Brascamp-Lieb inequality.
Microsoft Research（微软研究院）；Princeton University（普林斯顿大学）；Weizmann Institute of Science（魏茨曼科学研究所）
We prove that the Cramér transform of the uniform measure on a convex body in $\mathbb{R}^n$ is a $(1+o(1)) n$-self-concordant barrier, improving a seminal result of Nesterov and Nemirovski. This gives the first explicit construction of a universal barrier for convex bodies with optimal self-concordance parameter. The proof is based on basic geometry of log-concave distributions, and elementary duality in exponential families.
This paper shows that the self-concordance parameter of the universal barrier on any $n$-dimensional proper convex domain is upper bounded by $n$. This bound is tight and improves the previous $O(n)$ bound by Nesterov and Nemirovski. The key to our main result is a pair of new, sharp moment inequalities for $s$-concave distributions, which could be of independent interest.
Self-concordance is the most important property required for barriers in convex programming. It is intrinsically linked to the affine structure of the underlying space. Here we introduce an alternative notion of self-concordance which is linked to the projective structure. A function on a set $X \subset A^n$ in an $n$-dimensional affine space is projectively self-concordant if and only if it can be extended to an affinely self-concordant logarithmically homogeneous function on the conic extension $K \subset V^{n+1}$ of $X$ in the $(n+1)$-dimensional vector space obtained by homogenization of $…
Many problems in statistical learning, imaging, and computer vision involve the optimization of a non-convex objective function with singularities at the boundary of the feasible set. For such challenging instances, we develop a new interior-point technique building on the Hessian-barrier algorithm recently introduced in Bomze, Mertikopoulos, Schachinger and Staudigl, [SIAM J. Opt. 2019 29(3), pp. 2100-2127], where the Riemannian metric is induced by a generalized self-concordant function. This class of functions is sufficiently general to include most of the commonly used barrier functions in…
Quantum relative entropies are jointly convex functions of two positive definite matrices that generalize the Kullback-Leibler divergence and arise naturally in quantum information theory. In this paper, we prove self-concordance of natural barrier functions for the epigraphs of various quantum relative entropies and divergences. Furthermore we show that these barriers have optimal barrier parameter. These barriers allow convex optimization problems involving quantum relative entropies to be directly solved using interior point methods for non-symmetric cones, avoiding the approximations and l…
In this paper, we propose a distributed algorithm for solving large-scale separable convex problems using Lagrangian dual decomposition and the interior-point framework. By adding self-concordant barrier terms to the ordinary Lagrangian, we prove under mild assumptions that the corresponding family of augmented dual functions is self-concordant. This makes it possible to efficiently use the Newton method for tracing the central path. We show that the new algorithm is globally convergent and highly parallelizable and thus it is suitable for solving large-scale separable convex problems.
We study a class of monotone inclusions called "self-concordant inclusion" which covers three fundamental convex optimization formulations as special cases. We develop a new generalized Newton-type framework to solve this inclusion. Our framework subsumes three schemes: full-step, damped-step and path-following methods as specific instances, while allows one to use inexact computation to form generalized Newton directions. We prove a local quadratic convergence of both the full-step and damped-step algorithms. Then, we propose a new two-phase inexact path-following scheme for solving this mono…
The entropic barrier, studied by Bubeck and Eldan (Proc. Mach. Learn. Research, 2015), is a self-concordant barrier with asymptotically optimal self-concordance parameter. In this paper, we study the tropicalization of the central path associated with the entropic barrier, i.e., the logarithmic limit of this central path for a parametric family of linear programs defined over the field of Puiseux series. Our main result is that the tropicalization of the entropic central path is a piecewise linear curve which coincides with the tropicalization of the logarithmic central path studied by Allamig…
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self-concordant barrier. We provide a new joint treatment of proximal and self-concordant barrier concepts and illustrate that such problems can be efficiently solved, without the need of lifting the problem dimensions, as in disciplined convex optimization approach. We propose an in…
We prove that primal-dual log-barrier interior point methods are not strongly polynomial, by constructing a family of linear programs with $3r+1$ inequalities in dimension $2r$ for which the number of iterations performed is in $Ω(2^r)$. The total curvature of the central path of these linear programs is also exponential in $r$, disproving a continuous analogue of the Hirsch conjecture proposed by Deza, Terlaky and Zinchenko. Our method is to tropicalize the central path in linear programming. The tropical central path is the piecewise-linear limit of the central paths of parameterized familie…
We propose and analyse primal-dual interior-point algorithms for convex optimization problems in conic form. The families of algorithms we analyse are so-called short-step algorithms and they match the current best iteration complexity bounds for primal-dual symmetric interior-point algorithm of Nesterov and Todd, for symmetric cone programming problems with given self-scaled barriers. Our results apply to any self-concordant barrier for any convex cone. We also prove that certain specializations of our algorithms to hyperbolic cone programming problems (which lie strictly between symmetric co…
Short-step methods are an important class of algorithms for solving convex constrained optimization problems. In this short paper, we show that under very mild assumptions on the self-concordant barrier and the width of the $\ell_2$-neighbourhood, any short-step interior-point method is not strongly polynomial-time.
In this paper, we establish the local superlinear convergence property of some polynomial-time interior-point methods for an important family of conic optimization problems. The main structural property used in our analysis is the logarithmic homogeneity of self-concordant barrier function, which must have {\em negative curvature}. We propose a new path-following predictor-corrector scheme, which work only in the dual space. It is based on an easily computable gradient proximity measure, which ensures an automatic transformation of the global linear rate of convergence to the local superlinear…
We develop a short-step interior point method to optimize a linear function over a convex body assuming that one only knows a membership oracle for this body. The approach is based on Abernethy and Hazan's sketch of a universal interior point method using the so-called entropic barrier [arXiv 1507.02528v2, 2015]. It is well-known that the gradient and Hessian of the entropic barrier can be approximated by sampling from Boltzmann-Gibbs distributions, and the entropic barrier was shown to be self-concordant by Bubeck and Eldan [arXiv 1412.1587v3, 2015]. The analysis of our algorithm uses propert…
In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function $f$ and a (possibly nonsmooth) proper closed convex function $g$. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of convex optimization problems. It has also found numerous applications in machine learning. The proximal damped Newton (PDN) methods have been well studied in the literature for solving this problem that enjoy a nice iteration complexity. Given that at each iteration these methods t…
We develop a new inexact interior-point Lagrangian decomposition method to solve a wide range class of constrained composite convex optimization problems. Our method relies on four techniques: Lagrangian dual decomposition, self-concordant barrier smoothing, path-following, and proximal-Newton technique. It also allows one to approximately compute the solution of the primal subproblems (called the slave problems), which leads to inexact oracles (i.e., inexact gradients and Hessians) of the smoothed dual problem (called the master problem). The smoothed dual problem is nonsmooth, we propose to …
We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, with convergence and complexity guarantees that are independent of condition numbers and related problem-dependent quantities. Given a suitable initialization, similar guarantees also hold for strongly convex and smooth objectives without self-concordance. When implemented using ra…
We prove that the classic logarithmic barrier problem is equivalent to a particular logarithmic barrier positive relaxation problem with barrier and scaling parameters. Based on the equivalence, a line-search primal-dual interior-point relaxation method for nonlinear programs is presented. Our method does not require any primal or dual iterates to be interior-points, which is prominently different from the existing interior-point methods in the literature. A new logarithmic barrier penalty function dependent on both primal and dual variables is used to prompt the global convergence of the meth…
University of Michigan（密歇根大学）；Princeton University（普林斯顿大学）
This paper explores a surprising equivalence between two seemingly-distinct convex optimization methods. We show that simulated annealing, a well-studied random walk algorithms, is directly equivalent, in a certain sense, to the central path interior point algorithm for the the entropic universal barrier function. This connection exhibits several benefits. First, we are able improve the state of the art time complexity for convex optimization under the membership oracle model. We improve the analysis of the randomized algorithm of Kalai and Vempala by utilizing tools developed by Nesterov and …
We use the concept of barrier-based smoothing approximations introduced in [ C. B. Chua and Z. Li, A barrier-based smoothing proximal point algorithm for NCPs over closed convex cones, SIOPT 23(2), 2010] to extend the non-interior continuation method proposed in [B. Chen and N. Xiu, A global linear and local quadratic noninterior continuation method for nonlinear complementarity problems based on Chen-Mangasarian smoothing functions, SIOPT 9(3), 1999] to an inexact non-interior continuation method for variational inequalities over general closed convex sets. Newton equations involved in the me…
A Riemannian Variational and Spectral Framework for High-Dimensional Sphere Packing: Barrier-Dynamics Reconciliation, Periodic Rigidity, and Discrete-Time Guarantees
Applying an interior-point method to the central-path conditions is a widely used approach for solving quadratic programs. Reformulating these conditions in the log-domain is a natural variation on this approach that to our knowledge is previously unstudied. In this paper, we analyze log-domain interior-point methods and prove their polynomial-time convergence. We also prove that they are approximated by classical barrier methods in a precise sense and provide simple computational experiments illustrating their superior performance.
In this paper we address a practical aspect of differential barrier penalty functions in linear programming. In this respect we propose an affine scaling interior point algorithm based on a large classe of differential barrier functions. The comparison of the algorithm with a vesion of the classical affine scaling algorithm shows that the algorithm is robust and efficient. We thus show that differential barrier functions open up new perspectives in linear optimization.
In this paper, we develop an interior-point method for solving a class of convex optimization problems with time-varying objective and constraint functions. Using log-barrier penalty functions, we propose a continuous-time dynamical system for tracking the (time-varying) optimal solution with an asymptotically vanishing error. This dynamical system is composed of two terms: (i) a correction term consisting of a continuous-time version of Newton's method, and (ii) a prediction term able to track the drift of the optimal solution by taking into account the time-varying nature of the objective an…
Based on solving an equivalent parametric equality constrained mini-max problem of the classic logarithmic-barrier subproblem, we present a novel primal-dual interior-point relaxation method for nonlinear programs with general equality and nonnegative constraints. In each iteration, our method approximately solves the KKT system of a parametric equality constrained mini-max subproblem, which avoids the requirement that any primal or dual iterate is an interior-point. The method has some similarities to the warmstarting interior-point methods in relaxing the interior-point requirement and is ea…
With the help of a logarithmic barrier augmented Lagrangian function, we can obtain closed-form solutions of slack variables of logarithmic-barrier problems of nonlinear programs. As a result, a two-parameter primal-dual nonlinear system is proposed, which corresponds to the Karush-Kuhn-Tucker point and the infeasible stationary point of nonlinear programs, respectively, as one of two parameters vanishes. Based on this distinctive system, we present a primal-dual interior-point method capable of rapidly detecting infeasibility of nonlinear programs. The method generates interior-point iterates…
We study preconditioned proximal point methods for a class of saddle point problems, where the preconditioner decouples the overall proximal point method into an alternating primal--dual method. This is akin to the Chambolle--Pock method or the ADMM. In our work, we replace the squared distance in the dual step by a barrier function on a symmetric cone, while using a standard (Euclidean) proximal step for the primal variable. We show that under non-degeneracy and simple linear constraints, such a hybrid primal--dual algorithm can achieve linear convergence on originally strongly convex problem…
A primal--dual interior-point method for nonsymmetric conic optimization with conjugate-free scaling
Nankai University（南开大学）；Suzhou University of Science and Technology（苏州科技大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）
In recent work, we provide computational arguments for expanding the class of proper cones recognized by conic optimization solvers, to permit simpler, smaller, more natural conic formulations. We define an exotic cone as a proper cone for which we can implement a small set of tractable (i.e. fast, numerically stable, analytic) oracles for a logarithmically homogeneous self-concordant barrier for the cone or for its dual cone. Our extensible, open source conic interior point solver, Hypatia, allows modeling and solving any conic optimization problem over a Cartesian product of exotic cones. In…
