Interior-point methods offer a highly versatile framework for convex optimization that is effective in theory and practice. A key notion in their theory is that of a self-concordant barrier. We give a suitable generalization of self-concordance to Riemannian manifolds and show that it gives the same structural results and guarantees as in the Euclidean setting, in particular local quadratic convergence of Newton's method. We analyze a path-following method for optimizing compatible objectives over a convex domain for which one has a self-concordant barrier, and obtain the standard complexity guarantees as in the Euclidean setting. We provide general constructions of barriers, and show that on the space of positive-definite matrices and other symmetric spaces, the squared distance to a point is self-concordant. To demonstrate the versatility of our framework, we give algorithms with state-of-the-art complexity guarantees for the general class of scaling and non-commutative optimization problems, which have been of much recent interest, and we provide the first algorithms for efficiently finding high-precision solutions for computing minimal enclosing balls and geometric medians in nonpositive curvature.
Linear programming is now included in algorithm undergraduate and postgraduate courses for computer science majors. We give a self-contained treatment of an interior-point method which is particularly tailored to the typical mathematical background of CS students. In particular, only limited knowledge of linear algebra and calculus is assumed.
It is known that one can solve semidefinite programs to within fixed accuracy in polynomial time using the ellipsoid method (under some assumptions). In this paper it is shown that the same holds true when one uses the short-step, primal interior point method. The main idea of the proof is to employ Diophantine approximation at each iteration to bound the intermediate bit-sizes of iterates.
Mehrotra's algorithm has been the most successful infeasible interior-point algorithm for linear programming since 1990. Most popular interior-point software packages for linear programming are based on Mehrotra's algorithm. This paper proposes an alternative algorithm, arc-search infeasible interior-point algorithm. We will demonstrate, by testing Netlib problems and comparing the test results obtained by arc-search infeasible interior-point algorithm and Mehrotra's algorithm, that the proposed arc-search infeasible interior-point algorithm is a more efficient algorithm than Mehrotra's algori…
In this paper, we propose an arc-search infeasible-interior-point algorithm. We show that this algorithm is polynomial and the polynomial bound is ${\cal O}(nL)$ which is at least as good as the best existing bound for infeasible-interior-point algorithms for linear programming.
Resource allocation problems are usually solved with specialized methods exploiting their general sparsity and problem-specific algebraic structure. We show that the sparsity structure alone yields a closed-form Newton search direction for the generic primal-dual interior point method. Computational tests show that the interior point method consistently outperforms the best specialized methods when no additional algebraic structure is available.
Applying an interior-point method to the central-path conditions is a widely used approach for solving quadratic programs. Reformulating these conditions in the log-domain is a natural variation on this approach that to our knowledge is previously unstudied. In this paper, we analyze log-domain interior-point methods and prove their polynomial-time convergence. We also prove that they are approximated by classical barrier methods in a precise sense and provide simple computational experiments illustrating their superior performance.
We study infeasible-start primal-dual interior-point methods for convex optimization problems given in a typically natural form we denote as Domain-Driven formulation. Our algorithms extend many advantages of primal-dual interior-point techniques available for conic formulations, such as the current best complexity bounds, and more robust certificates of approximate optimality, unboundedness, and infeasibility, to Domain-Driven formulations. The complexity results are new for the infeasible-start setup used, even in the case of linear programming. In addition to complexity results, our algorit…
Based on solving an equivalent parametric equality constrained mini-max problem of the classic logarithmic-barrier subproblem, we present a novel primal-dual interior-point relaxation method for nonlinear programs with general equality and nonnegative constraints. In each iteration, our method approximately solves the KKT system of a parametric equality constrained mini-max subproblem, which avoids the requirement that any primal or dual iterate is an interior-point. The method has some similarities to the warmstarting interior-point methods in relaxing the interior-point requirement and is ea…
We present a short step interior point method for solving a class of nonlinear programming problems with quadratic objective function. Convex quadratic programming problems can be reformulated as problems in this class. The method is shown to have weak polynomial time complexity. A complete proof of the numerical stability of the method is provided. No requirements on feasibility, row-rank of the constraint Jacobian, strict complementarity, or conditioning of the problem are made. Infeasible problems are solved to an optimal interior least-squares solution.
Since more than three decades, interior-point methods proved very useful for optimization, from linear over semidefinite to conic (and partly beyond non-convex) programming; despite the fact that already in the semidefinite case (even when strong duality holds) "hard" problems are known. We shade a light on a rather surprising restriction in the non-commutative world (of semidefinite programming), namely "commutative" paths and propose a new family of solvers that is able to use the full richness of "non-commutative" search directions: (primal) feasible-interior-point methods. Beside a detaile…
In this paper, we propose an infeasible arc-search interior-point algorithm for solving nonlinear programming problems. Most algorithms based on interior-point methods are categorized as line search, since they compute a next iterate on a straight line determined by a search direction which approximates the central path.The proposed arc-search interior-point algorithm uses an arc for the approximation.We discuss convergence properties of the proposed algorithm.We also conduct numerical experiments on the CUTEst benchmark problems and compare the performance of the proposed arc-search algorithm…
In this paper, we proposed an interior point method for constrained optimization, which is characterized by the using of quasi-tangential subproblem. This algorithm follows the main ideas of primal dual interior point methods and Byrd-Omojokun's step decomposition strategy. The quasi-tangential subproblem is obtained by penalizing the null space constraint in the tangential subproblem. The resulted quasi-tangential step is not strictly lying in the null space of the gradients of constraints. We also use a line search trust-funnel-like strategy, instead of penalty function or filter technology,…
Point sets matching problems can be handled by optimal transport. The mechanism behind it is that optimal transport recovers the point-to-point correspondence associated with the least curl deformation. Optimal transport is a special form of linear programming with dense constraints. Linear programming can be handled by interior point methods, provided that the involved ill-conditioned Hessians can be computed accurately. During the decade, matrix balancing has been employed to compute optimal transport under entropy regularization approaches. The solution quality in the interior point method …
This paper introduces the algorithmic design and implementation of Tulip, an open-source interior-point solver for linear optimization. It implements a regularized homogeneous interior-point algorithm with multiple centrality corrections, and therefore handles unbounded and infeasible problems. The solver is written in Julia, thus allowing for a flexible and efficient implementation: Tulip's algorithmic framework is fully disentangled from linear algebra implementations and from a model's arithmetic. In particular, this allows to seamlessly integrate specialized routines for structured problem…
Recent advances in open source interior-point optimization methods and power system related software have provided researchers and educators with the necessary platform for simulating and optimizing power networks with unprecedented convenience. Within the Matpower software platform a combination of several different interior point optimization methods are provided and four different optimal power flow (OPF) formulations are recently available: the Polar-Power, Polar-Current, Cartesian-Power, and Cartesian-Current. The robustness and reliability of interior-point methods for different OPF form…
In this paper, we study an infeasible interior-point method for linear optimization with full-Newton step. The introduced method uses an algebraic equivalent transformation on the centering equation of the system which defines the central path. We prove that the method finds an $\varepsilon$-optimal solution of the underlying problem in polynomial time.
For each dimension $d$, $d$-dimensional integral simplices with exactly one interior integral point have bounded volume. This was first shown by Hensley. Explicit volume bounds were determined by Hensley, Lagarias and Ziegler, Pikhurko, and Averkov. In this paper we determine the exact upper volume bound for such simplices and characterize the volume-maximizing simplices. We also determine the sharp upper bound on the coefficient of asymmetry of an integral polytope with a single interior integral point. This result confirms a conjecture of Hensley from 1983. Moreover, for an integral simplex …
Interior point methods are among the most popular techniques for large scale nonlinear optimization, owing to their intrinsic ability of scaling to arbitrary large problem sizes. Their efficiency has attracted in recent years a lot of attention due to increasing demand for large scale optimization in industry and engineering. A parallel interior point method is discussed that exploits the intrinsic structure of large-scale nonlinear optimization problems so that the solution process can employ massively parallel high-performance computing infastructures. Since the overall performance of interi…
In this paper, we consider integral maximal lattice-free simplices. Such simplices have integer vertices and contain integer points in the relative interior of each of their facets, but no integer point is allowed in the full interior. In dimension three, we show that any integral maximal lattice-free simplex is equivalent to one of seven simplices up to unimodular transformation. For higher dimensions, we demonstrate that the set of integral maximal lattice-free simplices with vertices lying on the coordinate axes is finite. This gives rise to a conjecture that the total number of integral ma…
We give an explicit upper bound on the volume of lattice simplices with fixed positive number of interior lattice points. The bound differs from the conjectural sharp upper bound only by a linear factor in the dimension. This improves significantly upon the previously best results by Pikhurko from 2001.
In this paper, we introduce a new kernel function which differs from previous functions, and play an important role for generating a new design of primal-dual interior point algorithms for semidefinite linear complementarity problem. Its properties, allow us a great simplicity for the analysis of interior-point method, therefore the complexity of large-update primal-dual interior point is the best so far. Numerical tests have shown that the use of this function gave a big improvement in the results concerning the time and the number of iterations. so is well promising and perform well enough i…
In this paper we will discuss two variants of an inexact feasible interior point algorithm for convex quadratic programming. We will consider two different neighbourhoods: a (small) one induced by the use of the Euclidean norm which yields a short-step algorithm and a symmetric one induced by the use of the infinity norm which yields a (practical) long-step algorithm. Both algorithms allow for the Newton equation system to be solved inexactly. For both algorithms we will provide conditions for the level of error acceptable in the Newton equation and establish the worst-case complexity results.
In this paper we present general-purpose preconditioners for regularized augmented systems arising from optimization problems, and their corresponding normal equations. We discuss positive definite preconditioners, suitable for CG and MINRES. We consider "sparsifications" which avoid situations in which eigenvalues of the preconditioned matrix may become complex. Special attention is given to systems arising from the application of regularized interior point methods to linear or nonlinear convex programming problems.
We prove that the classic logarithmic barrier problem is equivalent to a particular logarithmic barrier positive relaxation problem with barrier and scaling parameters. Based on the equivalence, a line-search primal-dual interior-point relaxation method for nonlinear programs is presented. Our method does not require any primal or dual iterates to be interior-points, which is prominently different from the existing interior-point methods in the literature. A new logarithmic barrier penalty function dependent on both primal and dual variables is used to prompt the global convergence of the meth…
In this paper we propose an efficient distributed algorithm for solving loosely coupled convex optimization problems. The algorithm is based on a primal-dual interior-point method in which we use the alternating direction method of multipliers (ADMM) to compute the primal-dual directions at each iteration of the method. This enables us to join the exceptional convergence properties of primal-dual interior-point methods with the remarkable parallelizability of ADMM. The resulting algorithm has superior computational properties with respect to ADMM directly applied to our problem. The amount of …
In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in [An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming, Computational Optimization and Applications, 78, 307--351 (2021)] for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. In particular, we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM) and interpret the algorithm (IP-PMM) as a primal-dual regularized IPM, suitable for sol…
Regularization and interior point approaches offer valuable perspectives to address constrained nonlinear optimization problems in view of control applications. This paper discusses the interactions between these techniques and proposes an algorithm that synergistically combines them. Building a sequence of closely related subproblems and approximately solving each of them, this approach inherently exploits warm-starting, early termination, and the possibility to adopt subsolvers tailored to specific problem structures. Moreover, by relaxing the equality constraints with a proximal penalty, th…
This paper consists of four general parts: convex sets; convex functions; convex optimization; and the interior-point algorithm. I will start by introducing the definition of convex sets and give three common convex set examples which will be used later in this paper, then prove the significant separating and supporting hyperplane theorems. Stepping into convex functions, in addition to offering definitions, I will also prove the first order and second-order conditions for convexity of a function, and then introduce a couple of examples that will be used in a convex optimization problem later.…
We propose and analyse primal-dual interior-point algorithms for convex optimization problems in conic form. The families of algorithms we analyse are so-called short-step algorithms and they match the current best iteration complexity bounds for primal-dual symmetric interior-point algorithm of Nesterov and Todd, for symmetric cone programming problems with given self-scaled barriers. Our results apply to any self-concordant barrier for any convex cone. We also prove that certain specializations of our algorithms to hyperbolic cone programming problems (which lie strictly between symmetric co…
In this paper, we study reinforcement learning (RL) algorithms to solve real-world decision problems with the objective of maximizing the long-term reward as well as satisfying cumulative constraints. We propose a novel first-order policy optimization method, Interior-point Policy Optimization (IPO), which augments the objective with logarithmic barrier functions, inspired by the interior-point method. Our proposed method is easy to implement with performance guarantees and can handle general types of cumulative multiconstraint settings. We conduct extensive evaluations to compare our approach…
In this paper, we propose a distributed algorithm for solving large-scale separable convex problems using Lagrangian dual decomposition and the interior-point framework. By adding self-concordant barrier terms to the ordinary Lagrangian, we prove under mild assumptions that the corresponding family of augmented dual functions is self-concordant. This makes it possible to efficiently use the Newton method for tracing the central path. We show that the new algorithm is globally convergent and highly parallelizable and thus it is suitable for solving large-scale separable convex problems.
Auxiliary variables are often used to model a convex piecewise linear function in the framework of linear optimization. This work shows that such variables yield a block diagonal plus low rank structure in the reduced KKT system of the dual problem. We show how the structure can be detected efficiently, and derive the linear algebra formulas for an interior point method which exploit such structure. The structure is detected in 36% of the cases in Netlib. Numerical results on the inverse planning problem in radiation therapy show an order of magnitude speed-up compared to the state-of-the-art …
A new relaxed variant of interior point method for low-rank semidefinite programming problems is proposed in this paper. The method is a step outside of the usual interior point framework. In anticipation to converging to a low-rank primal solution, a special nearly low-rank form of all primal iterates is imposed. To accommodate such a (restrictive) structure, the first order optimality conditions have to be relaxed and are therefore approximated by solving an auxiliary least-squares problem. The relaxed interior point framework opens numerous possibilities how primal and dual approximated New…
Primal-dual interior-point multigrid method for topology optimization
University of Birmingham（伯明翰大学）；Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic（捷克科学院信息论与自动化研究所）；University of Kirkuk（基尔库克大学）
Large-scale optimization problems that seek sparse solutions have become ubiquitous. They are routinely solved with various specialized first-order methods. Although such methods are often fast, they usually struggle with not-so-well conditioned problems. In this paper, specialized variants of an interior point-proximal method of multipliers are proposed and analyzed for problems of this class. Computational experience on a variety of problems, namely, multi-period portfolio optimization, classification of data coming from functional Magnetic Resonance Imaging, restoration of images corrupted …
Computing the Wasserstein barycenter of a set of probability measures under the optimal transport metric can quickly become prohibitive for traditional second-order algorithms, such as interior-point methods, as the support size of the measures increases. In this paper, we overcome the difficulty by developing a new adapted interior-point method that fully exploits the problem's special matrix structure to reduce the iteration complexity and speed up the Newton procedure. Different from regularization approaches, our method achieves a well-balanced tradeoff between accuracy and speed. A numeri…
This paper considers the problem of Bayesian optimization for systems with safety-critical constraints, where both the objective function and the constraints are unknown, but can be observed by querying the system. In safety-critical applications, querying the system at an infeasible point can have catastrophic consequences. Such systems require a safe learning framework, such that the performance objective can be optimized while satisfying the safety-critical constraints with high probability. In this paper we propose a safe Bayesian optimization framework that ensures that the points queried…
