We present and analyze a central cutting surface algorithm for general semi-infinite convex optimization problems, and use it to develop a novel algorithm for distributionally robust optimization problems in which the uncertainty set consists of probability distributions with given bounds on their moments. Moments of arbitrary order, as well as non-polynomial moments can be included in the formulation. We show that this gives rise to a hierarchy of optimization problems with decreasing levels of risk-aversion, with classic robust optimization at one end of the spectrum, and stochastic programming at the other. Although our primary motivation is to solve distributionally robust optimization problems with moment uncertainty, the cutting surface method for general semi-infinite convex programs is also of independent interest. The proposed method is applicable to problems with non-differentiable semi-infinite constraints indexed by an infinite-dimensional index set. Examples comparing the cutting surface algorithm to the central cutting plane algorithm of Kortanek and No demonstrate the potential of our algorithm even in the solution of traditional semi-infinite convex programming problems whose constraints are differentiable and are indexed by an index set of low dimension. After the rate of convergence analysis of the cutting surface algorithm, we extend the authors' moment matching scenario generation algorithm to a probabilistic algorithm that finds optimal probability distributions subject to moment constraints. The combination of this distribution optimization method and the central cutting surface algorithm yields a solution to a family of distributionally robust optimization problems that are considerably more general than the ones proposed to date.
An analytic center cutting plane method is an iterative algorithm based on the computation of analytic centers. In this paper, we propose some analytic center cutting plane methods for solving quasimonotone or pseudomonotone variational inequalities whose domains are bounded or unbounded convex bodies.
We introduce StoDCuP (Stochastic Dynamic Cutting Plane), an extension of the Stochastic Dual Dynamic Programming (SDDP) algorithm to solve multistage stochastic convex optimization problems. At each iteration, the algorithm builds lower affine functions not only for the cost-to-go functions, as SDDP does, but also for some or all nonlinear cost and constraint functions. We show the almost sure convergence of StoDCuP. We also introduce an inexact variant of StoDCuP where all subproblems are solved approximately (with bounded errors) and show the almost sure convergence of this variant for vanis…
We consider the integer points in a unimodular cone K ordered by a lexicographic rule defined by a lattice basis. To each integer point x in K we associate a family of inequalities (lex-cuts) that defines the convex hull of the integer points in K that are not lexicographically smaller than x. The family of lex-cuts contains the Chvatal-Gomory cuts, but does not contain and is not contained in the family of split cuts. This provides a finite cutting plane method to solve the integer program min{cx : x \in S \cap Z^n }, where S \subset R^n is a compact set and c \in Z^n . We analyze the number …
Many problems of interest for cyber-physical network systems can be formulated as Mixed Integer Linear Programs in which the constraints are distributed among the agents. In this paper we propose a distributed algorithm to solve this class of optimization problems in a peer-to-peer network with no coordinator and with limited computation and communication capabilities. In the proposed algorithm, at each communication round, agents solve locally a small LP, generate suitable cutting planes, namely intersection cuts and cost-based cuts, and communicate a fixed number of active constraints, i.e.,…
Recently, Kronqvist et al.~\cite{KronqvistLundellWesterlund2016} rediscovered the supporting hyperplane algorithm of Veinott~\cite{Veinott1967} and demonstrated its computational benefits for solving convex mixed-integer nonlinear programs. In this paper we derive the algorithm from a geometric point of view. This enables us to show that the supporting hyperplane algorithm is equivalent to Kelley's cutting plane algorithm~\cite{J.E.Kelley1960} applied to a particular reformulation of the problem. As a result, we extend the applicability of the supporting hyperplane algorithm to convex problems…
On Parallel and Batch-Cutting Strategies for Norm-Minimization-Based Convex Vector Optimization
King Fahd University of Petroleum & Minerals（法赫德国王石油与矿产大学）；Interdisciplinary Research Center for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals（法赫德国王石油与矿产大学智能交通与物流交叉研究中心）
We propose a new variant of Kelley's cutting-plane method for minimizing a nonsmooth convex Lipschitz-continuous function over the Euclidean space. We derive the method through a constructive approach and prove that it attains the optimal rate of convergence for this class of problems.
In this paper, we present a novel approach for solving a maintenance scheduling problem from the French electricity grid company RTE, as presented in the ROADEF 2020 challenge. Our approach combines constraint generation with a new family of cutting planes to address the nonconvexity of the problem. We provide mathematical proofs and separation algorithms for the cutting planes and study the practical impact of our approach on the challenge instances. This method is applicable to a wide range of problems involving the modeling of uncertainty, and is shown to improve results significantly on re…
In this paper, we present an analysis of the strength of sparse cutting-planes for mixed integer linear programs (MILP) with sparse formulations. We examine three kinds of problems: packing problems, covering problems, and more general MILPs with the only assumption that the objective function is non-negative. Given a MILP instance of one of these three types, assume that we decide on the support of cutting-planes to be used and the strongest inequalities on these supports are added to the linear programming relaxation. Call the optimal objective function value of the linear programming relaxa…
In this paper, we propose a cutting plane algorithm based on DC (Difference-of-Convex) programming and DC cut for globally solving Mixed-Binary Linear Program (MBLP). We first use a classical DC programming formulation via the exact penalization to formulate MBLP as a DC program, which can be solved by DCA algorithm. Then, we focus on the construction of DC cuts, which serves either as a local cut (namely type-I DC cut) at feasible local minimizer of MBLP, or as a global cut (namely type-II DC cut) at infeasible local minimizer of MBLP if some particular assumptions are verified. Otherwise, th…
We propose a successive generation of cutting inequalities for binary quadratic optimization problems. Multiple cutting inequalities are successively generated for the convex hull of the set of the optimal solutions $\subset \{0, 1\}^n$, while the standard cutting inequalities are used for the convex hull of the feasible region. An arbitrary linear inequality with integer coefficients and the right-hand side value in integer is considered as a candidate for a valid inequality. The validity of the linear inequality is determined by solving a conic relaxation of a subproblem such as the doubly n…
The incorporation of cutting planes within the branch-and-bound algorithm, known as branch-and-cut, forms the backbone of modern integer programming solvers. These solvers are the foremost method for solving discrete optimization problems and thus have a vast array of applications in machine learning, operations research, and many other fields. Choosing cutting planes effectively is a major research topic in the theory and practice of integer programming. We conduct a novel structural analysis of branch-and-cut that pins down how every step of the algorithm is affected by changes in the parame…
Integer programming (IP) is a general optimization framework widely applicable to a variety of unstructured and structured problems arising in, e.g., scheduling, production planning, and graph optimization. As IP models many provably hard to solve problems, modern IP solvers rely on many heuristics. These heuristics are usually human-designed, and naturally prone to suboptimality. The goal of this work is to show that the performance of those solvers can be greatly enhanced using reinforcement learning (RL). In particular, we investigate a specific methodology for solving IPs, known as the Cut…
For convex sets $K$ and $L$ in ${\mathbb{R}}^d$ we define $R_L(K)$ to be the convex hull of all points belonging to $K$ but not to the interior of $L$. Cutting-plane methods from integer and mixed-integer optimization can be expressed in geometric terms using functionals $R_L$ with appropriately chosen sets $L$. We describe the geometric properties of $R_L(K)$ and characterize those $L$ for which $R_L$ maps polyhedra to polyhedra. For certain natural classes ${\mathcal{L}}$ of convex sets in ${\mathbb{R}}^d$ we consider the functional $R_{\mathcal{L}}$ given by $R_{\mathcal{L}}(K):= \bigcap_{L…
This paper proposes a general duality framework for the problem of minimizing a convex integral functional over a space of stochastic processes adapted to a given filtration. The framework unifies many well-known duality frameworks from operations research and mathematical finance. The unification allows the extension of some useful techniques from these two fields to a much wider class of problems. In particular, combining certain finite-dimensional techniques from convex analysis with measure theoretic techniques from mathematical finance, we are able to close the duality gap in some situati…
We show that it is possible to obtain an $O(ε^{-4/3})$ expected runtime --- including computational cost --- for finding $ε$-stationary points of smooth nonconvex functions using cutting plane methods. This improves on the best-known epsilon dependence achieved by cubic regularized Newton of $O(ε^{-3/2})$ as proved by Nesterov and Polyak (2006). Our techniques utilize the convex until proven guilty principle proposed by Carmon, Duchi, Hinder, and Sidford (2017).
The robust truss topology optimization against the uncertain static external load can be formulated as mixed-integer semidefinite programming. Although a global optimal solution can be computed with a branch-and-bound method, it is very time-consuming. This paper presents an alternative formulation, semidefinite programming with complementarity constraints, and proposes an efficient heuristic. The proposed method is based upon the convex-concave procedure for DC (difference-of-convex) programming. It is shown that the method can often find a practically reasonable truss design within the compu…
Current state-of-the-art methods for solving discrete optimization problems are usually restricted to convex settings. In this paper, we propose a general approach based on cutting planes for solving nonlinear, possibly nonconvex, binary optimization problems. We provide a rigorous convergence analysis that quantifies the number of iterations required under different conditions. This is different to most other work in discrete optimization where only finite convergence is proved. Moreover, using tools from variational analysis, we provide necessary and sufficient dual optimality conditions.
In this paper we introduce disciplined convex-concave programming (DCCP), which combines the ideas of disciplined convex programming (DCP) with convex-concave programming (CCP). Convex-concave programming is an organized heuristic for solving nonconvex problems that involve objective and constraint functions that are a sum of a convex and a concave term. DCP is a structured way to define convex optimization problems, based on a family of basic convex and concave functions and a few rules for combining them. Problems expressed using DCP can be automatically converted to standard form and solved…
We study a mixed integer linear program with m integer variables and k non-negative continuous variables in the form of the relaxation of the corner polyhedron that was introduced by Andersen, Louveaux, Weismantel and Wolsey [Inequalities from two rows of a simplex tableau, Proc. IPCO 2007, LNCS, vol. 4513, Springer, pp. 1--15]. We describe the facets of this mixed integer linear program via the extreme points of a well-defined polyhedron. We then utilize this description to give polynomial time algorithms to derive valid inequalities with optimal l_p norm for arbitrary, but fixed m. For the c…
We give a simple and natural method for computing approximately optimal solutions for minimizing a convex function $f$ over a convex set $K$ given by a separation oracle. Our method utilizes the Frank--Wolfe algorithm over the cone of valid inequalities of $K$ and subgradients of $f$. Under the assumption that $f$ is $L$-Lipschitz and that $K$ contains a ball of radius $r$ and is contained inside the origin centered ball of radius $R$, using $O(\frac{(RL)^2}{\varepsilon^2} \cdot \frac{R^2}{r^2})$ iterations and calls to the oracle, our main method outputs a point $x \in K$ satisfying $f(x) \le…
This paper is devoted to some approaches for convex min-min problems with smoothness and strong convexity in only one of the two variable groups. It is shown that the proposed approaches, based on Vaidya's cutting plane method and Nesterov's fast gradient method, achieve the linear convergence. The outer minimization problem is solved using Vaidya's cutting plane method, and the inner problem (smooth and strongly convex) is solved using the fast gradient method. Due to the importance of machine learning applications, we also consider the case when the objective function is a sum of a large num…
We present an efficient algorithm for solving fractional programming problems whose objective functions are the ratio of a low-rank quadratic to a positive definite quadratic with convex constraints. The proposed algorithm for these convex-convex problems is based on the Shen-Yu Quadratic Transform which finds stationary points of concave-convex sum-of-ratios problems. We further use elements of the algorithm proposed in [arXiv:1802.10192] and the classic Dinkelbach approach to ensure convergence. We show that our algorithm performs better than previous algorithms for low-rank problems.
Technische Universität Berlin（柏林工业大学）；Zuse Institute Berlin（柏林祖斯研究所）；Fair Isaac Deutschland GmbH（费埃哲德国有限公司）
Cutting planes are a crucial component of state-of-the-art mixed-integer programming solvers, with the choice of which subset of cuts to add being vital for solver performance. We propose new distance-based measures to qualify the value of a cut by quantifying the extent to which it separates relevant parts of the relaxed feasible set. For this purpose, we use the analytic centers of the relaxation polytope or of its optimal face, as well as alternative optimal solutions of the linear programming relaxation. We assess the impact of the choice of distance measure on root node performance and th…
We extend the notion of some energy-type expressions based on two sets, developed in the abstract potential theory. We also give the discretized version of the quantities defined, similar to Chebyshev constant. This extension allows to apply the potential-theoretic results to infinite quadratic programming problems. Together with a cutting plane algorithm, the Chebyshev-constant method ensures that under certain conditions, the infinite problem can be reduced to semi-infinite or to finite problems.
A computationally efficient method to solve non-convex programming problems with linear equality constraints is presented. The proposed method is based on a recursively feasible and descending sequential convex programming procedure proven to converge to a locally optimal solution. Assuming that the first convex problem in the sequence is feasible, these properties are obtained by convexifying the non-convex cost and inequality constraints with inner-convex approximations. Additionally, a computationally efficient method is introduced to obtain inner-convex approximations based on Taylor serie…
We exhibit a class of classical or tropical posynomial systems which can be solved by reduction to linear or convex programming problems. This relies on a notion of colorful vectors with respect to a collection of Newton polytopes. This extends the convex programming approach of one player stochastic games.
We present Epsilon, a system for general convex programming using fast linear and proximal operators. As with existing convex programming frameworks, users specify convex optimization problems using a natural grammar for mathematical expressions, composing functions in a way that is guaranteed to be convex by the rules of disciplined convex programming. Given such an input, the Epsilon compiler transforms the optimization problem into a mathematically equivalent form consisting only of functions with efficient proximal operators---an intermediate representation we refer to as prox-affine form.…
Recently some specific classes of non-smooth and non-Lipschitz convex optimization problems were selected by Yu.~Nesterov along with H.~Lu. We consider convex programming problems with similar smoothness conditions for the objective function and functional constraints. We introduce a new concept of an inexact model and propose some analogues of switching subgradient schemes for convex programming problems for the relatively Lipschitz-continuous objective function and functional constraints. Some class of online convex optimization problems is considered. The proposed methods are optimal in the…
A sequential piecewise linear programming method is presented where bounded domains of non-convex functions are successively contracted about the solution of a piecewise linear program at each iteration of the algorithm. Although feasibility and optimality are not guaranteed, we show that the method is capable of obtaining convergent and optimal solutions on a number of Nonlinear Programming (NLP) and Mixed Integer Nonlinear Programming (MINLP) problems using only a small number of breakpoints and integer variables.
This paper studies the dynamic programming principle for general convex stochastic optimization problems introduced by Rockafellar and Wets in [30]. We extend the applicability of the theory by relaxing compactness and boundedness assumptions. In the context of financial mathematics, the relaxed assumption are satisfied under the well-known no-arbitrage condition and the reasonable asymptotic elasticity condition of the utility function. Besides financial mathematics, we obtain several new results in linear and nonlinear stochastic programming and stochastic optimal control.
Kuhn-Tucker conditions for mathematical programming problems in Banach spaces partially ordered by cone with empty interior are obtained under strong simultaneity condition. If partial ordered cone has interior point, it is proved that Slater and strong simultaneity conditions are equivalent.
The purpose of this paper is to study a non-convex fuzzy multi-objective quadratic programming problem, in which both the technological coefficients and resources are fuzzy with nonlinear membership function. A computational procedure to find a fuzzy efficient solution of this problem is developed. A numerical example is given to illustrate the procedure.
In this paper, we analyze in depth a simplicial decomposition like algorithmic framework for large scale convex quadratic programming. In particular, we first propose two tailored strategies for handling the master problem. Then, we describe a few techniques for speeding up the solution of the pricing problem. We report extensive numerical experiments on both real portfolio optimization and general quadratic programming problems, showing the efficiency and robustness of the method when compared to Cplex.
A multi-convex optimization problem is one in which the variables can be partitioned into sets over which the problem is convex when the other variables are fixed. Multi-convex problems are generally solved approximately using variations on alternating or cyclic minimization. Multi-convex problems arise in many applications, such as nonnegative matrix factorization, generalized low rank models, and structured control synthesis, to name just a few. In most applications to date the multi-convexity is simple to verify by hand. In this paper we study the automatic detection and verification of mul…
We study the generalization of split, k-branch split, and intersection cuts from Mixed Integer Linear Programming to the realm of Mixed Integer Nonlinear Programming. Constructing such cuts requires calculating the convex hull of the difference between a convex set and an open set with a simple geometric structure. We introduce two techniques to give precise characterizations of such convex hulls and use them to construct split, k-branch split, and intersection cuts for several classes of non-polyhedral sets. In particular, we give simple formulas for split cuts for essentially all convex sets…
A lift-and-permute scheme of alternating direction method of multipliers (ADMM) is proposed for linearly constrained convex programming. It contains not only the newly developed balanced augmented Lagrangian method and its dual-primal variation, but also the proximal ADMM and Douglas-Rachford splitting algorithm. It helps to propose accelerated algorithms with worst-case $O(1/k^2)$ convergence rates in the case that the objective function to be minimized is strongly convex.
Optimization problems involving sequential decisions in a stochastic environment were studied in Stochastic Programming (SP), Stochastic Optimal Control (SOC) and Markov Decision Processes (MDP). In this paper we mainly concentrate on SP and SOC modelling approaches. In these frameworks there are natural situations when the considered problems are convex. Classical approach to sequential optimization is based on dynamic programming. It has the problem of the so-called ``Curse of Dimensionality", in that its computational complexity increases exponentially with increase of dimension of state va…
