A Branch and Bound Based on NSGAII Algorithm for Multi-Objective Mixed Integer Non Linear Optimization Problems
Ahmed Jaber, Pascal Lafon, Rafic Younes
备注：
This article has been submitted for publication in Engineering Optimization Journal, published by Taylor & Francis. The article contains 28 pages, 13 figures, and 5 tables
Multi-Objective Mixed-Integer Non-Linear Programming problems (MO-MINLPs) appear in several real-world applications, especially in the mechanical engineering field. To determine a good approximated Pareto front for this type of problems, we propose a general hybrid approach based on a Multi-Criteria Branch-and-Bound (MCBB) and Non-dominated Sorting Genetic Algorithm 2 (NSGAII). We present a computational experiment based on a statistical assessment to compare the performance of the proposed algorithm (BnB-NSGAII) with NSGAII using well-known metrics from literature. We propose a new metric, Investment Ratio (IR), that relate the quality of the solution to the consumed effort. We consider five real-world mechanical engineering problems and two mathematical ones to be used as test problems in this experiment. Experimental results indicate that BnB-NSGAII could be a competitive alternative for solving MO-MINLPs.
The 3 stage reducer problem is a point of interest for many researchers. In this paper, this problem is reformulated to a bi-objective problem with additional constraints to meet the ISO mechanical standards. Those additional constraints increase the complexity of the problem, such that, NSGAII performance is not sufficient. To overcome this, we propose to use BnB-NSGAII method - a hybrid multi-criteria branch and bound with NSGAII - to initialize NSGAII before solving the problem, seeking for a better initial population. A new feature is also proposed to enhance BnB-NSGAII method, called the …
Dominance move (DoM) is a binary quality indicator that can be used in multiobjective optimization. It can compare solution sets while representing some important features such as convergence, spread, uniformity, and cardinality. DoM has an intuitive concept and considers the minimum move of one set needed to weakly Pareto dominate the other set. Despite the aforementioned properties, DoM is hard to calculate. The original formulation presents an efficient and exact method to calculate it in a biobjective case only. This work presents a new approach to calculate and extend DoM to deal with thr…
An important problem in optimization is the construction of mixed-integer programming (MIP) formulations of disjunctive constraints that are both strong and small. Motivated by lower bounds on the number of integer variables that are required by traditional MIP formulations, we present a more general mixed-integer branching formulation framework. Our approach maintains favorable algorithmic properties of traditional MIP formulations: in particular, amenability to branch-and-bound and branch-and-cut algorithms. Our main technical result gives an explicit linear inequality description for both t…
We present a generic branch-and-bound algorithm for finding all the Pareto solutions of a biobjective mixed-integer linear program. The main contributions are new algorithms for obtaining dual bounds at a node, for checking node fathoming, presolve and duality gap measurement. Our branch-and-bound is predominantly a decision space search method since the branching is performed on the decision variables, akin to single objective problems, although we also sometimes split gaps and branch in the objective space. The various algorithms are implemented using a data structure for storing Pareto sets…
We investigate how the numerical properties of the LP relaxations evolve throughout the solution procedure in a solver employing the branch-and-cut algorithm. The long-term goal of this work is to determine whether the effect on the numerical conditioning of the LP relaxations resulting from the branching and cutting operations can be effectively predicted and whether such predictions can be used to make better algorithmic choices. In a first step towards this goal, we discuss here the numerical behavior of an existing solver in order to determine whether our intuitive understanding of this be…
This work attempts to combine the strengths of two major technologies that have matured over the last three decades: global mixed-integer nonlinear optimization and branch-and-price. We consider a class of generally nonconvex mixed-integer nonlinear programs (MINLPs) with linear complicating constraints and integer linking variables. If the complicating constraints are removed, the problem becomes easy to solve, e.g. due to decomposable structure. Integrality of the linking variables allows us to apply a discretization approach to derive a Dantzig-Wolfe reformulation and solve the problem to g…
We consider the global optimization of nonconvex quadratic programs and mixed-integer quadratic programs. We present a family of convex quadratic relaxations which are derived by convexifying nonconvex quadratic functions through perturbations of the quadratic matrix. We investigate the theoretical properties of these quadratic relaxations and show that they are equivalent to some particular semidefinite programs. We also introduce novel branching variable selection strategies which can be used in conjunction with the quadratic relaxations investigated in this paper. We integrate the proposed …
A key ingredient in branch and bound (B&B) solvers for mixed-integer programming (MIP) is the selection of branching variables since poor or arbitrary selection can affect the size of the resulting search trees by orders of magnitude. A recent article by Le Bodic and Nemhauser [Mathematical Programming, (2017)] investigated variable selection rules by developing a theoretical model of B&B trees from which they developed some new, effective scoring functions for MIP solvers. In their work, Le Bodic and Nemhauser left several open theoretical problems, solutions to which could guide the future d…
We propose a mixed-integer quadratic programming (QP) solver that is suitable for use in embedded applications, for example, hybrid model predictive control (MPC). The solver is based on the branch-and-bound method, and uses a recently proposed dual active-set solver for solving the resulting QP relaxations. Moreover, we tailor the search of the branch-and-bound tree to be suitable for embedded applications on limited hardware; we show, for example, how a node in the branch-and-bound tree can be represented by only two integers. The embeddability of the solver is shown by successfully running …
We propose a feasible active set method for convex quadratic programming problems with non-negativity constraints. This method is specifically designed to be embedded into a branch-and-bound algorithm for convex quadratic mixed integer programming problems. The branch-and-bound algorithm generalizes the approach for unconstrained convex quadratic integer programming proposed by Buchheim, Caprara and Lodi to the presence of linear constraints. The main feature of the latter approach consists in a sophisticated preprocessing phase, leading to a fast enumeration of the branch-and-bound nodes. Mor…
Transmission network expansion planning is a mixed-integer optimization problem, whose solution is used to guide future investment in transmission equipment. An approach is presented to find the global solution of the transmission planning problem using an AC network model. The approach builds on the semidefinite relaxation of the AC optimal power flow problem (ACOPF); its computational engine is a new specialized branch-and-cut algorithm for transmission expansion planning to deal with the underlying mixed-integer ACOPF problem. Valid inequalities that are based on specific knowledge of the e…
It is a crucial problem how to heat oil and save running cost for crude oil transport. This paper strictly formulates such a heated oil pipeline problem as a mixed integer nonlinear programming model. Nonconvex and convex continuous relaxations of the model are proposed, which are proved to be equivalent under some suitable conditions. Meanwhile, we provide a preprocessing procedure to guarantee these conditions. Therefore we are able to design a branch-and-bound algorithm for solving the mixed integer nonlinear programming model to global optimality. To make the branch-and-bound algorithm mor…
The feeder reconfiguration problem chooses the on/off status of the switches in a distribution network in order to minimize a certain cost such as power loss. It is a mixed integer nonlinear program and hence hard to solve. A popular heuristic search consists of repeated application of branch exchange, where some loads are transferred from one feeder to another feeder while maintaining the radial structure of the network, until no load transfer can further reduce the cost. Optimizing each branch exchange step is itself a mixed integer nonlinear program. In this paper we propose an efficient al…
We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is $O(n)$ for each termination check assuming a bounded problem domain. We show that this domain restriction can be relaxed for problems whose data satisfies a simple rank condition, in which case each check requires an $O(n^2)$ s…
We study the complexity of cutting planes and branching schemes from a theoretical point of view. We give some rigorous underpinnings to the empirically observed phenomenon that combining cutting planes and branching into a branch-and-cut framework can be orders of magnitude more efficient than employing these tools on their own. In particular, we give general conditions under which a cutting plane strategy and a branching scheme give a provably exponential advantage in efficiency when combined into branch-and-cut. The efficiency of these algorithms is evaluated using two concrete measures: nu…
In this paper, we describe a comprehensive algorithmic framework for solving mixed integer bilevel linear optimization problems (MIBLPs) using a generalized branch-and-cut approach. The framework presented merges features from existing algorithms (for both traditional mixed integer linear optimization and MIBLPs) with new techniques to produce a flexible and robust framework capable of solving a wide range of bilevel optimization problems. The framework has been fully implemented in the open-source solver MibS. The paper describes the algorithmic options offered by MibS and presents computatio…
In this paper we address game theory problems arising in the context of network security. In traditional game theory problems, given a defender and an attacker, one searches for mixed strategies which minimize a linear payoff functional. In the problems addressed in this paper an additional quadratic term is added to the minimization problem. Such term represents switching costs, i.e., the costs for the defender of switching from a given strategy to another one at successive rounds of a Nash game. The resulting problems are nonconvex QP ones with linear constraints and turn out to be very chal…
We study the generalization of split, k-branch split, and intersection cuts from Mixed Integer Linear Programming to the realm of Mixed Integer Nonlinear Programming. Constructing such cuts requires calculating the convex hull of the difference between a convex set and an open set with a simple geometric structure. We introduce two techniques to give precise characterizations of such convex hulls and use them to construct split, k-branch split, and intersection cuts for several classes of non-polyhedral sets. In particular, we give simple formulas for split cuts for essentially all convex sets…
We apply a branch-and-bound (B\&B) algorithm to the D-optimality problem based on a convex mixed-integer nonlinear formulation. We discuss possible methodologies to accelerate the convergence of the B\&B algorithm, by combining the use of different upper bounds, variable-bound tightening inequalities, and local-search procedures. Different methodologies to compute the determinant of a matrix after a rank-one update are investigated to accelerate the local-searches. We discuss our findings through numerical experiments with randomly generated test problem.
Mixed integer nonlinear programs (MINLPs) are arguably among the hardest optimization problems, with a wide range of applications. MINLP solvers that are based on linear relaxations and spatial branching work similar as mixed integer programming (MIP) solvers in the sense that they are based on a branch-and-cut algorithm, enhanced by various heuristics, domain propagation, and presolving techniques. However, the analysis of infeasible subproblems, which is an important component of most major MIP solvers, has been hardly studied in the context of MINLPs. There are two main approaches for infea…
The selection of branching variables is a key component of branch-and-bound algorithms for solving Mixed-Integer Programming (MIP) problems since the quality of the selection procedure is likely to have a significant effect on the size of the enumeration tree. State-of-the-art procedures base the selection of variables on their "LP gains", which is the dual bound improvement obtained after branching on a variable. There are various ways of selecting variables depending on their LP gains. However, all methods are evaluated empirically. In this paper we present a theoretical model for the select…
Data-driven Mixed Integer Optimization through Probabilistic Multi-variable Branching
Shanghai University of Finance and Economics（上海财经大学）；Stanford University（斯坦福大学）；Antai College of Economics and Management, Shanghai Jiao Tong University（上海交通大学安泰经济与管理学院）
We present a new branch-and-bound type search method for mixed integer linear optimization problems based on the concept of offshoots (introduced in this paper). While similar to a classic branch-and-bound method, it allows for changing the order of the variables in a dive (shaping) and removing unnecessary branching variables from a dive (trimming). The regular branch-and-bound algorithm can be seen as a special case of our new method. We also discuss extensions to our new method such as choosing to branch from the top or the bottom of an offshoot. We present several numerical experiments to …
Speakman and Lee (2017) gave a formula for the volume of the convex hull of the graph of a trilinear monomial, $y=x_1x_2x_3$, over a box in the nonnegative orthant, in terms of the upper and lower bounds on the variables. This was done in the context of using volume as a measure for comparing alternative convexifications to guide the implementation of spatial branch-and-bound for mixed integer nonlinear optimization problems. Here, we introduce an alternative method for computing this volume, making use of the rich theory of mixed volumes. This new method may lead to a natural approach for con…
We propose an improved successive branch reduction (SBR) method to solve stochastic distribution network reconfiguration (SDNR), a mixed-integer program that is known to be computationally challenging. First, for a special distribution network with a single redundant branch, we propose an improved design for a one-stage SBR algorithm in the literature to incorporate uncertain renewable generations and loads. Based on solving stochastic optimal power flow, the improved algorithm identifies and searches through a small set of candidate branches, from which it determines the optimal branch to ope…
Speeding Up Mixed-Integer Programming Solvers with Sparse Learning for Branching
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology（佐治亚理工学院 H. Milton Stewart 工业与系统工程系）；School of Chemical & Biomolecular Engineering, Georgia Institute of Technology（佐治亚理工学院化学与生物分子工程学院）
We present an exact algorithm for mean-risk optimization subject to a budget constraint, where decision variables may be continuous or integer. The risk is measured by the covariance matrix and weighted by an arbitrary monotone function, which allows to model risk-aversion in a very individual way. We address this class of convex mixed-integer minimization problems by designing a branch-and-bound algorithm, where at each node, the continuous relaxation is solved by a non-monotone Frank-Wolfe type algorithm with away-steps. Experimental results on portfolio optimization problems show that our a…
Nonconvex mixed-integer nonlinear programs (MINLPs) represent a challenging class of optimization problems that often arise in engineering and scientific applications. Because of nonconvexities, these programs are typically solved with global optimization algorithms, which have limited scalability. However, nonlinear branch-and-bound has recently been shown to be an effective heuristic for quickly finding high-quality solutions to large-scale nonconvex MINLPs, such as those arising in infrastructure network optimization. This work proposes Juniper, a Julia-based open-source solver for nonlinea…
In this paper, we present the first outer approximation algorithm for multi-objective mixed-integer linear programming problems with any number of objectives. The algorithm also works for certain classes of non-linear programming problems. It produces the non-dominated extreme points as well as the facets of the convex hull of these points. The algorithm relies on an oracle which solves single-objective weighted-sum problems and we show that the required number of oracle calls is polynomial in the number of facets of the convex hull of the non-dominated extreme points in the case of multiobjec…
We study two-stage robust optimization problems with mixed discrete-continuous decisions in both stages. Despite their broad range of applications, these problems pose two fundamental challenges: (i) they constitute infinite-dimensional problems that require a finite-dimensional approximation, and (ii) the presence of discrete recourse decisions typically prohibits duality-based solution schemes. We address the first challenge by studying a $K$-adaptability formulation that selects $K$ candidate recourse policies before observing the realization of the uncertain parameters and that implements …
We investigate the theoretical complexity of branch-and-bound (BB) and cutting plane (CP) algorithms for mixed-integer optimization. In particular, we study the relative efficiency of BB and CP, when both are based on the same family of disjunctions. We extend a result of Dash to the nonlinear setting which shows that for convex 0/1 problems, CP does at least as well as BB, with variable disjunctions. We sharpen this by giving instances of the stable set problem where we can provably establish that CP does exponentially better than BB. We further show that if one moves away from 0/1 sets, this…
The incorporation of cutting planes within the branch-and-bound algorithm, known as branch-and-cut, forms the backbone of modern integer programming solvers. These solvers are the foremost method for solving discrete optimization problems and thus have a vast array of applications in machine learning, operations research, and many other fields. Choosing cutting planes effectively is a major research topic in the theory and practice of integer programming. We conduct a novel structural analysis of branch-and-cut that pins down how every step of the algorithm is affected by changes in the parame…
This paper proposes an open-source distributed solver for solving Sparse Convex Optimization (SCO) problems over computational networks. Motivated by past algorithmic advances in mixed-integer optimization, the Sparse Convex Optimization Toolkit (SCOT) adopts a mixed-integer approach to find exact solutions to SCO problems. In particular, SCOT brings together various techniques to transform the original SCO problem into an equivalent convex Mixed-Integer Nonlinear Programming (MINLP) problem that can benefit from high-performance and parallel computing platforms. To solve the equivalent mixed-…
We develop a decomposition algorithm for distributionally-robust two-stage stochastic mixed-integer convex cone programs, and its important special case of distributionally-robust two-stage stochastic mixed-integer second order cone programs. This generalizes the algorithm proposed by Sen and Sherali~[Mathematical Programming 106(2): 203-223, 2006]. We show that the proposed algorithm is finitely convergent if the second-stage problems are solved to optimality at incumbent first stage solutions, and solution to an optimization problem to identify worst-case probability distribution is availabl…
Shanghai Jiao Tong University（上海交通大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；University College London（伦敦大学学院）
Deriving a good variable selection strategy in branch-and-bound is essential for the efficiency of modern mixed-integer programming (MIP) solvers. With MIP branching data collected during the previous solution process, learning to branch methods have recently become superior over heuristics. As branch-and-bound is naturally a sequential decision making task, one should learn to optimize the utility of the whole MIP solving process instead of being myopic on each step. In this work, we formulate learning to branch as an offline reinforcement learning (RL) problem, and propose a long-sighted hyb…
In line with the growing trend of using machine learning to help solve combinatorial optimisation problems, one promising idea is to improve node selection within a mixed integer programming (MIP) branch-and-bound tree by using a learned policy. Previous work using imitation learning indicates the feasibility of acquiring a node selection policy, by learning an adaptive node searching order. In contrast, our imitation learning policy is focused solely on learning which of a node's children to select. We present an offline method to learn such a policy in two settings: one that comprises a heur…
Akaike's information criterion (AIC) is a measure of the quality of a statistical model for a given set of data. We can determine the best statistical model for a particular data set by the minimization of the AIC. Since we need to evaluate exponentially many candidates of the model by the minimization of the AIC, the minimization is unreasonable. Instead, stepwise methods, which are local search algorithms, are commonly used to find a better statistical model though it may not be the best. We propose a branch and bound search algorithm for a mixed integer nonlinear programming formulation of …
Machine Learning Algorithms for Improving Exact Classical Solvers in Mixed Integer Continuous Optimization
Fakultät für Mathematik, Universität Wien（维也纳大学数学系）；Department of Computer Science, Czech Technical University（捷克技术大学计算机科学系）；CEO Olicheza Limited（Olicheza有限公司）
Cutting planes are a key ingredient to successfully solve mixed-integer linear programs. For specific problems, their strength is often theoretically assessed by showing that they are facet-defining for the corresponding mixed-integer hull. In this paper we experimentally investigate the dimensions of faces induced by general-purpose cutting planes generated by a state-of-the-art solver. Therefore, we relate the dimension of each cutting plane to its impact in a branch-and-bound algorithm.
In this paper, we surveyed the existing literature studying different approaches and algorithms for the four critical components in the general branch and bound (B&B) algorithm, namely, branching variable selection, node selection, node pruning, and cutting-plane selection. However, the complexity of the B&B algorithm always grows exponentially with respect to the increase of the decision variable dimensions. In order to improve the speed of B&B algorithms, learning techniques have been introduced in this algorithm recently. We further surveyed how machine learning can be used to improve the f…
