In this paper we will discuss two variants of an inexact feasible interior point algorithm for convex quadratic programming. We will consider two different neighbourhoods: a (small) one induced by the use of the Euclidean norm which yields a short-step algorithm and a symmetric one induced by the use of the infinity norm which yields a (practical) long-step algorithm. Both algorithms allow for the Newton equation system to be solved inexactly. For both algorithms we will provide conditions for the level of error acceptable in the Newton equation and establish the worst-case complexity results.
We develop a new inexact interior-point Lagrangian decomposition method to solve a wide range class of constrained composite convex optimization problems. Our method relies on four techniques: Lagrangian dual decomposition, self-concordant barrier smoothing, path-following, and proximal-Newton technique. It also allows one to approximately compute the solution of the primal subproblems (called the slave problems), which leads to inexact oracles (i.e., inexact gradients and Hessians) of the smoothed dual problem (called the master problem). The smoothed dual problem is nonsmooth, we propose to …
In this paper, we propose a new method that combines the inexact Newton method with a procedure to obtain a feasible inexact projection for solving constrained smooth and nonsmooth equations. The local convergence theorems are established under the assumption of smoothness or semismoothness of the function that defines the equation and its regularity at the solution. In particular, we show that a sequence generated by the method converges to a solution with linear, superlinear, or quadratic rate, under suitable conditions. Moreover, some numerical experiments are reported to illustrate the pra…
In this paper, we propose a new inexact version of the projected subgradient method to solve nondifferentiable constrained convex optimization problems. The method combine $ε$-subgradient method with a procedure to obtain a feasible inexact projection onto the constraint set. Asymptotic convergence results and iteration-complexity bounds for the sequence generated by the method employing the well known exogenous stepsizes, Polyak's stepsizes, and dynamic stepsizes are established.
Interior-point methods for linear programming problems require the repeated solution of a linear system of equations. Solving these linear systems is non-trivial due to the severe ill-conditioning of the matrices towards convergence. This issue can be alleviated by incorporating suitable regularization terms in the linear programming problem. Regularization also allows us to efficiently handle rank deficient constraint matrices. We provide a convergence analysis of a regularized inexact interior-point method. The term `inexact' refers to the fact that we do not need to compute the true solutio…
We use the concept of barrier-based smoothing approximations introduced in [ C. B. Chua and Z. Li, A barrier-based smoothing proximal point algorithm for NCPs over closed convex cones, SIOPT 23(2), 2010] to extend the non-interior continuation method proposed in [B. Chen and N. Xiu, A global linear and local quadratic noninterior continuation method for nonlinear complementarity problems based on Chen-Mangasarian smoothing functions, SIOPT 9(3), 1999] to an inexact non-interior continuation method for variational inequalities over general closed convex sets. Newton equations involved in the me…
A class of interior point methods using inexact directions is analysed. The linear system arising in interior point methods for linear programming is reformulated such that the solution is less sensitive to perturbations in the right-hand side. For the new system an implementable condition is formulated that controls the relative error in the solution. Based on this condition, a feasible and an infeasible potential reduction method are described which retain the convergence and complexity bounds known for exact directions.
We herein propose a variant of the projected inexact Levenberg--Marquardt method (ILMM) for solving constrained nonsmooth equations. Since the orthogonal projection onto the feasible set may be computationally expensive, we propose a local ILMM with feasible inexact projections. By using assumption of semi-smoothness and an error bound condition which is weaker than the standard full rank assumption, we establish its convergence results, including results on its rate. Finally, some computacional results are reported in order to illustrate the advantages of the new schemes.
Quantum linear system algorithms (QLSAs) have the potential to speed up algorithms that rely on solving linear systems. Interior Point Methods (IPMs) yield a fundamental family of polynomial-time algorithms for solving optimization problems. IPMs solve a Newton linear system at each iteration to find the search direction, and thus QLSAs can potentially speed up IPMs. Due to the noise in contemporary quantum computers, such quantum-assisted IPM (QIPM) only allows an inexact solution for the Newton linear system. Typically, an inexact search direction leads to an infeasible solution. In our work…
We provide new tools for worst-case performance analysis of the gradient (or steepest descent) method of Cauchy for smooth strongly convex functions, and Newton's method for self-concordant functions, including the case of inexact search directions. The analysis uses semidefinite programming performance estimation, as pioneered by Drori and Teboulle [Mathematical Programming, 145(1-2):451-482, 2014], and extends recent performance estimation results for the method of Cauchy by the authors [Optimization Letters, 11(7), 1185-1199, 2017]. To illustrate the applicability of the tools, we demonstra…
A gradient projection method with feasible inexact projections is proposed in the present paper. The inexact projection is performed using a relative error tolerance. Asymptotic convergence analysis and iteration-complexity bounds of the method employing constant and Armijo step sizes are presented. Numerical results are reported illustrating the potential advantages of considering inexact projections instead of exact ones in some medium scale instances of a least squares problem over the spectrohedron.
The purpose of this paper is to present an inexact version of the scaled gradient projection method on a convex set, which is inexact in two sense. First, an inexact projection on the feasible set is computed, allowing for an appropriate relative error tolerance. Second, an inexact non-monotone line search scheme is employed to compute a step size which defines the next iteration. It is shown that the proposed method has similar asymptotic convergence properties and iteration-complexity bounds as the usual scaled gradient projection method employing monotone line searches.
The primary focus of this paper is on designing an inexact first-order algorithm for solving constrained nonlinear optimization problems. By controlling the inexactness of the subproblem solution, we can significantly reduce the computational cost needed for each iteration. A penalty parameter updating strategy during the process of solving the subproblem enables the algorithm to automatically detect infeasibility. Global convergence for both feasible and infeasible cases are proved. Complexity analysis for the KKT residual is also derived under mild assumptions. Numerical experiments exhibit …
Convex relaxations of AC optimal power flow (AC-OPF) problems have attracted significant interest as in several instances they provably yield the global optimum to the original non-convex problem. If, however, the relaxation is inexact, the obtained solution is not AC-feasible. The quality of the obtained solution is essential for several practical applications of AC-OPF, but detailed analyses are lacking in existing literature. This paper aims to cover this gap. We provide an in-depth investigation of the solution characteristics when convex relaxations are inexact, we assess the most promisi…
In this paper, we develop new first-order method for composite non-convex minimization problems with simple constraints and inexact oracle. The objective function is given as a sum of "`hard"', possibly non-convex part, and "`simple"' convex part. Informally speaking, oracle inexactness means that, for the "`hard"' part, at any point we can approximately calculate the value of the function and construct a quadratic function, which approximately bounds this function from above. We give several examples of such inexactness: smooth non-convex functions with inexact Hölder-continuous gradient, fun…
We study a general convex optimization problem, which covers various classic problems in different areas and particularly includes many optimal transport related problems arising in recent years. To solve this problem, we revisit the classic Bregman proximal point algorithm (BPPA) and introduce a new inexact stopping condition for solving the subproblems, which can circumvent the underlying feasibility difficulty often appearing in existing inexact conditions when the problem has a complex feasible set. Our inexact condition also covers several existing inexact conditions as special cases and …
In this work, we consider a constrained convex problem with linear inequalities and provide an inexact penalty re-formulation of the problem. The novelty is in the choice of the penalty functions, which are smooth and can induce a non-zero penalty over some points in feasible region of the original constrained problem. The resulting unconstrained penalized problem is parametrized by two penalty parameters which control the slope and the curvature of the penalty function. With a suitable selection of these penalty parameters, we show that the solutions of the resulting penalized unconstrained p…
A framework is proposed for solving general convex quadratic programs (CQPs) from an infeasible starting point by invoking an existing feasible-start algorithm tailored for inequality-constrained CQPs. The central tool is an exact penalty function scheme equipped with a penalty-parameter updating rule. The feasible-start algorithm merely has to satisfy certain general requirements, and so is the updating rule. Under mild assumptions, the framework is proved to converge on CQPs with both inequality and equality constraints and, at a negligible additional cost per iteration, produces an infeasib…
Randomized Branch Methods with Inexact Subproblems for Bouligand Stationarity in Linear and Quadratic Programs with Complementarity Constraints
Southern University of Science and Technology（南方科技大学）；Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院计算数学与科学工程计算研究所）；University of Chinese Academy of Sciences（中国科学院大学）；National Center for Applied Mathematics Shenzhen（深圳应用数学国家研究中心）
Interior point methods (IPMs) are a common approach for solving linear programs (LPs) with strong theoretical guarantees and solid empirical performance. The time complexity of these methods is dominated by the cost of solving a linear system of equations at each iteration. In common applications of linear programming, particularly in machine learning and scientific computing, the size of this linear system can become prohibitively large, requiring the use of iterative solvers, which provide an approximate solution to the linear system. However, approximately solving the linear system at each …
In this paper we present the solver DuQuad specialized for solving general convex quadratic problems arising in many engineering applications. When it is difficult to project on the primal feasible set, we use the (augmented) Lagrangian relaxation to handle the complicated constraints and then, we apply dual first order algorithms based on inexact dual gradient information for solving the corresponding dual problem. The iteration complexity analysis is based on two types of approximate primal solutions: the primal last iterate and an average of primal iterates. We provide computational complex…
We develop a short-step interior point method to optimize a linear function over a convex body assuming that one only knows a membership oracle for this body. The approach is based on Abernethy and Hazan's sketch of a universal interior point method using the so-called entropic barrier [arXiv 1507.02528v2, 2015]. It is well-known that the gradient and Hessian of the entropic barrier can be approximated by sampling from Boltzmann-Gibbs distributions, and the entropic barrier was shown to be self-concordant by Bubeck and Eldan [arXiv 1412.1587v3, 2015]. The analysis of our algorithm uses propert…
In this paper, we consider a nonlinear semi-infinite program that minimizes a function including a log-determinant (logdet) function over positive definite matrix constraints and infinitely many convex inequality constraints, called SIPLOG for short. The main purpose of the paper is to develop an algorithm for computing a Karush-Kuhn-Tucker (KKT) point for the SIPLOG efficiently. More specifically, we propose an interior point sequential quadratic programming-type method that inexactly solves a sequence of semi-infinite quadratic programs approximating the SIPLOG. Furthermore, to generate a se…
In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in [An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming, Computational Optimization and Applications, 78, 307--351 (2021)] for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. In particular, we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM) and interpret the algorithm (IP-PMM) as a primal-dual regularized IPM, suitable for sol…
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self-concordant barrier. We provide a new joint treatment of proximal and self-concordant barrier concepts and illustrate that such problems can be efficiently solved, without the need of lifting the problem dimensions, as in disciplined convex optimization approach. We propose an in…
We prove that the classic logarithmic barrier problem is equivalent to a particular logarithmic barrier positive relaxation problem with barrier and scaling parameters. Based on the equivalence, a line-search primal-dual interior-point relaxation method for nonlinear programs is presented. Our method does not require any primal or dual iterates to be interior-points, which is prominently different from the existing interior-point methods in the literature. A new logarithmic barrier penalty function dependent on both primal and dual variables is used to prompt the global convergence of the meth…
The leaky ReLU network with a group sparse regularization term has been widely used in the recent years. However, training such a network yields a nonsmooth nonconvex optimization problem and there exists a lack of approaches to compute a stationary point deterministically. In this paper, we first resolve the multi-layer composite term in the original optimization problem by introducing auxiliary variables and additional constraints. We show the new model has a nonempty and bounded solution set and its feasible set satisfies the Mangasarian-Fromovitz constraint qualification. Moreover, we show…
The classical convex feasibility problem in a finite dimensional Euclidean space is studied in the present paper. We are interested in two cases. First, we assume to know how to compute an exact project onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method can be used for computing efficiently an inexact projection on it. Second, we assume that both sets involved are compact such that the CondG method can be used for computing efficiently inexact projections on them. We combine alternating projection method with CondG method to design a ne…
Primal-dual interior-point multigrid method for topology optimization
University of Birmingham（伯明翰大学）；Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic（捷克科学院信息论与自动化研究所）；University of Kirkuk（基尔库克大学）
We consider primal-dual pairs of semidefinite programs and assume that they are ill-posed, i.e., both primal and dual are either weakly feasible or weakly infeasible. Under such circumstances, strong duality may break down and the primal and dual might have a nonzero duality gap. Nevertheless, there are arbitrary small perturbations to the problem data which makes the perturbed primal-dual pair strongly feasible thus zeroing the duality gap. In this paper, we conduct an asymptotic analysis of the optimal value as the perturbation is driven to zero. Specifically, we fix two positive definite ma…
We study linear programming relaxations of nonconvex quadratic programs given by the reformulation-linearization technique (RLT), referred to as RLT relaxations. We investigate the relations between the polyhedral properties of the feasible regions of a quadratic program and its RLT relaxation. We establish various connections between recession directions, boundedness, and vertices of the two feasible regions. Using these properties, we present necessary and sufficient exactness conditions for RLT relaxations. We then give a thorough discussion of how our results can be converted into simple a…
Quantum computing has attracted significant interest in the optimization community because it potentially can solve classes of optimization problems faster than conventional supercomputers. Several researchers proposed quantum computing methods, especially Quantum Interior Point Methods (QIPMs), to solve convex optimization problems, such as Linear Optimization, Semidefinite Optimization, and Second-order Cone Optimization problems. Most of them have applied a Quantum Linear System Algorithm at each iteration to compute a Newton step. However, using quantum linear solvers in QIPMs comes with m…
Since more than three decades, interior-point methods proved very useful for optimization, from linear over semidefinite to conic (and partly beyond non-convex) programming; despite the fact that already in the semidefinite case (even when strong duality holds) "hard" problems are known. We shade a light on a rather surprising restriction in the non-commutative world (of semidefinite programming), namely "commutative" paths and propose a new family of solvers that is able to use the full richness of "non-commutative" search directions: (primal) feasible-interior-point methods. Beside a detaile…
With the help of a logarithmic barrier augmented Lagrangian function, we can obtain closed-form solutions of slack variables of logarithmic-barrier problems of nonlinear programs. As a result, a two-parameter primal-dual nonlinear system is proposed, which corresponds to the Karush-Kuhn-Tucker point and the infeasible stationary point of nonlinear programs, respectively, as one of two parameters vanishes. Based on this distinctive system, we present a primal-dual interior-point method capable of rapidly detecting infeasibility of nonlinear programs. The method generates interior-point iterates…
In this paper, we propose a new algorithm combining the Douglas-Rachford (DR) algorithm and the Frank-Wolfe algorithm, also known as the conditional gradient (CondG) method, for solving the classic convex feasibility problem. Within the algorithm, which will be named {\it Approximate Douglas-Rachford (ApDR) algorithm}, the CondG method is used as a subroutine to compute feasible inexact projections on the sets under consideration, and the ApDR iteration is defined based on the DR iteration. The ApDR algorithm generates two sequences, the main sequence, based on the DR iteration, and its corres…
This paper studies the worst case iteration complexity of an infeasible interior point method (IPM) for seconder order cone programming (SOCP), which is more convenient for warmstarting compared with feasible IPMs. The method studied bases on the homogeneous and self-dual model and the Monteiro-Zhang family of searching directions. Its worst case iteration complexity is $O\left(k^{1/2}\log\left(ε^{-1}\right)\right)$, to reduce the primal residual, dual residual, and complementarity gap by a factor of $ε$, where $k$ is the number of cone constraints. The result is the same as the best known res…
In this paper, an inexact proximal-point penalty method is studied for constrained optimization problems, where the objective function is non-convex, and the constraint functions can also be non-convex. The proposed method approximately solves a sequence of subproblems, each of which is formed by adding to the original objective function a proximal term and quadratic penalty terms associated to the constraint functions. Under a weak-convexity assumption, each subproblem is made strongly convex and can be solved effectively to a required accuracy by an optimal gradient-based method. The computa…
This paper considers a class of convex optimization problems where both, the objective function and the constraints, have a continuously varying dependence on time. Our goal is to develop an algorithm to track the optimal solution as it continuously changes over time inside or on the boundary of the dynamic feasible set. We develop an interior point method that asymptotically succeeds in tracking this optimal point in nonstationary settings. The method utilizes a time varying constraint slack and a prediction-correction structure that relies on time derivatives of functions and constraints and…
Regularization and interior point approaches offer valuable perspectives to address constrained nonlinear optimization problems in view of control applications. This paper discusses the interactions between these techniques and proposes an algorithm that synergistically combines them. Building a sequence of closely related subproblems and approximately solving each of them, this approach inherently exploits warm-starting, early termination, and the possibility to adopt subsolvers tailored to specific problem structures. Moreover, by relaxing the equality constraints with a proximal penalty, th…
