A sequential piecewise linear programming method is presented where bounded domains of non-convex functions are successively contracted about the solution of a piecewise linear program at each iteration of the algorithm. Although feasibility and optimality are not guaranteed, we show that the method is capable of obtaining convergent and optimal solutions on a number of Nonlinear Programming (NLP) and Mixed Integer Nonlinear Programming (MINLP) problems using only a small number of breakpoints and integer variables.
A computationally efficient method to solve non-convex programming problems with linear equality constraints is presented. The proposed method is based on a recursively feasible and descending sequential convex programming procedure proven to converge to a locally optimal solution. Assuming that the first convex problem in the sequence is feasible, these properties are obtained by convexifying the non-convex cost and inequality constraints with inner-convex approximations. Additionally, a computationally efficient method is introduced to obtain inner-convex approximations based on Taylor serie…
Many separable nonlinear optimization problems can be approximated by their nonlinear objective functions with piecewise linear functions. A natural question arising from applying this approach is how to break the interval of interest into subintervals (pieces) to achieve a good approximation. We present formulations to optimize the location of the knots. We apply a sequential quadratic programming method and a spectral projected gradient method to solve the problem. We report numerical experiments to show the effectiveness of the proposed approaches.
University of Tsukuba（筑波大学）；Senshu University（专修大学）；Tokyo University of Agriculture and Technology（东京农工大学）
This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between the logistic loss function and its quadratic approximation, their formulation may fail to find a good subset of features. To overcome this drawback, we apply a piecewise-linear approximation to the logistic loss function. Accordingly, we frame the feature subset selection problem…
We consider a control constrained parabolic optimal control problem and use variational discretization for its time semi-discretization. The state equation is treated with a Petrov-Galerkin scheme using a piecewise constant Ansatz for the state and piecewise linear, continuous test functions. This results in variants of the Crank-Nicolson scheme for the state and the adjoint state. Exploiting a superconvergence result we prove second order convergence in time of the error in the controls. Moreover, the piecewise linear and continuous parabolic projection of the discrete state on the dual time …
Our study is motivated by the solution of Mixed-Integer Non-Linear Programming (MINLP) problems with separable non-convex functions via the Sequential Convex MINLP technique, an iterative method whose main characteristic is that of solving, for bounding purposes, piecewise-convex MINLP relaxations obtained by identifying the intervals in which each univariate function is convex or concave and then relaxing the concave parts with piecewise-linear relaxations of increasing precision. This process requires the introduction of new binary variables for the activation of the intervals where the func…
Image segmentation is an important median level vision topic. Accurate and efficient multiphase segmentation for images with intensity inhomogeneity is still a great challenge. We present a new two-stage multiphase segmentation method trying to tackle this, where the key is to compute an inhomogeneity-free approximate image. For this, we propose to use a new non-Lipschitz variational decomposition model in the first stage. The minimization problem is solved by an iterative support shrinking algorithm, with a global convergence guarantee and a lower bound theory of the image gradient of the ite…
A popular class of algorithms to optimize the dual LP relaxation of the discrete energy minimization problem (a.k.a.\ MAP inference in graphical models or valued constraint satisfaction) are convergent message-passing algorithms, such as max-sum diffusion, TRW-S, MPLP and SRMP. These algorithms are successful in practice, despite the fact that they are a version of coordinate minimization applied to a convex piecewise-affine function, which is not guaranteed to converge to a global minimizer. These algorithms converge only to a local minimizer, characterized by local consistency known from con…
For optimization problems with nonlinear constraints, linearly constrained Lagrangian (LCL) methods sequentially minimize a Lagrangian function subject to linearized constraints. These methods converge rapidly near a solution but may not be reliable from arbitrary starting points. The well known example \MINOS\ has proven effective on many large problems. Its success motivates us to propose a globally convergent variant. Our stabilized LCL method possesses two important properties: the subproblems are always feasible, and they may be solved inexactly. These features are present in \MINOS only …
In this paper, a class of optimization problems with nonlinear inequality constraints is discussed. Based on the ideas of sequential quadratic programming algorithm and the method of strongly sub-feasible directions, a new superlinearly convergent algorithm is proposed. The initial iteration point can be chosen arbitrarily for the algorithm. At each iteration, the new algorithm solves one quadratic programming subproblem which is always feasible, and one or two systems of linear equations with a common coefficient matrix. Moreover, the coefficient matrix is uniformly nonsingular. After finite …
In this article, a globally convergent sequential quadratic programming (SQP) method is developed for multi-objective optimization problems with inequality type constraints. A feasible descent direction is obtained using a linear approximation of all objective functions as well as constraint functions. The sub-problem at every iteration of the sequence has feasible solution. A non-differentiable penalty function is used to deal with constraint violations. A descent sequence is generated which converges to a critical point under the Mangasarian-Fromovitz constraint qualification along with some…
Motivated by the increasing availability of high-performance parallel computing, we design a distributed parallel algorithm for linearly-coupled block-structured nonconvex constrained optimization problems. Our algorithm performs Jacobi-type proximal updates of the augmented Lagrangian function, requiring only local solutions of separable block nonlinear programming (NLP) problems. We provide a cheap and explicitly computable Lyapunov function that allows us to establish global and local sublinear convergence of our algorithm, its iteration complexity, as well as simple, practical and theoreti…
This paper mainly concerns with the primal superlinear convergence of the quasi-Newton sequential quadratic programming (SQP) method for piecewise linear-quadratic composite optimization problems. We show that the latter primal superlinear convergence can be justified under the noncriticality of Lagrange multipliers and a version of the Dennis-More condition. Furthermore, we show that if we replace the noncriticality condition with the second-order sufficient condition, this primal superlinear convergence is equivalent with an appropriate version of the Dennis-More condition. We also recover B…
Control invariant set is critical for guaranteeing safe control and the problem of computing control invariant set for linear discrete-time system is revisited in this paper by using a data-driven approach. Specifically, sample points on convergent trajectories of linear MPC are recorded, of which the convex hull formulates a control invariant set for the linear system. To approximate the convex hull of multiple sample points, a convex piecewise linear (PWL) fitting framework has been proposed, which yields a polyhedral approximation with predefined complexity. A descent algorithm for the conv…
ENS Paris-Saclay（巴黎萨克雷高等师范学校）；University of Genova（热那亚大学）；Université Toulouse 3 Paul Sabatier（图卢兹第三大学）
We prove that the iterates produced by, either the scalar step size variant, or the coordinatewise variant of AdaGrad algorithm, are convergent sequences when applied to convex objective functions with Lipschitz gradient. The key insight is to remark that such AdaGrad sequences satisfy a variable metric quasi-Fejér monotonicity property, which allows to prove convergence.
Superlinear and quadratic convergence of a stabilized sequential quadratic semidefinite programming method for nonlinear semidefinite programming problems
A Convergent Algorithm Based on Deterministic Approximation for a Large Class of Regime-Switching Generalized Stochastic Game-Theoretic Riccati Differential Equations
We introduce a contractive abstract dynamic programming framework and related policy iteration algorithms, specifically designed for sequential zero-sum games and minimax problems with a general structure. Aside from greater generality, the advantage of our algorithms over alternatives is that they resolve some long-standing convergence difficulties of the "natural" policy iteration algorithm, which have been known since the Pollatschek and Avi-Itzhak method [PoA69] for finite-state Markov games. Mathematically, this "natural" algorithm is a form of Newton's method for solving Bellman's equati…
This paper presents the SCvx algorithm, a successive convexification algorithm designed to solve non-convex constrained optimal control problems with global convergence and superlinear convergence-rate guarantees. The proposed algorithm can handle nonlinear dynamics and non-convex state and control constraints. It solves the original problem to optimality by successively linearizing non-convex dynamics and constraints about the solution of the previous iteration. The resulting convex subproblems are numerically tractable, and can be computed quickly and reliably using convex optimization solve…
Multistage stochastic programming deals with operational and planning problems that involve a sequence of decisions over time while responding to realizations that are uncertain. Algorithms designed to address multistage stochastic linear programming (MSLP) problems often rely upon scenario trees to represent the underlying stochastic process. When this process exhibits stagewise independence, sampling-based techniques, particularly the stochastic dual dynamic programming (SDDP) algorithm, have received wide acceptance. However, these sampling-based methods still operate with a deterministic r…
In this paper, the elliptic PDE-constrained optimization problem with box constraints on the control is studied. To numerically solve the problem, we apply the 'optimize-discretize-optimize' strategy. Specifically, the alternating direction method of multipliers (ADMM) algorithm is applied in function space first, then the standard piecewise linear finite element approach is employed to discretize the subproblems in each iteration. Finally, some efficient numerical methods are applied to solve the discretized subproblems based on their structures. Motivated by the idea of the multi-level strat…
We present adaptive sequential SAA (sample average approximation) algorithms to solve large-scale two-stage stochastic linear programs. The iterative algorithm framework we propose is organized into \emph{outer} and \emph{inner} iterations as follows: during each outer iteration, a sample-path problem is implicitly generated using a sample of observations or ``scenarios," and solved only \emph{imprecisely}, to within a tolerance that is chosen \emph{adaptively}, by balancing the estimated statistical error against solution error. The solutions from prior iterations serve as \emph{warm starts} …
