This paper studies a strategy for data-driven algorithm design for large-scale combinatorial optimization problems that can leverage existing state-of-the-art solvers in general purpose ways. The goal is to arrive at new approaches that can reliably outperform existing solvers in wall-clock time. We focus on solving integer programs, and ground our approach in the large neighborhood search (LNS) paradigm, which iteratively chooses a subset of variables to optimize while leaving the remainder fixed. The appeal of LNS is that it can easily use any existing solver as a subroutine, and thus can inherit the benefits of carefully engineered heuristic or complete approaches and their software implementations. We show that one can learn a good neighborhood selector using imitation and reinforcement learning techniques. Through an extensive empirical validation in bounded-time optimization, we demonstrate that our LNS framework can significantly outperform compared to state-of-the-art commercial solvers such as Gurobi.
Large Neighborhood Search (LNS) is a combinatorial optimization heuristic that starts with an assignment of values for the variables to be optimized, and iteratively improves it by searching a large neighborhood around the current assignment. In this paper we consider a learning-based LNS approach for mixed integer programs (MIPs). We train a Neural Diving model to represent a probability distribution over assignments, which, together with an off-the-shelf MIP solver, generates an initial assignment. Formulating the subsequent search steps as a Markov Decision Process, we train a Neural Neighb…
Nanyang Technological University（南洋理工大学）；Shandong University（山东大学）；Singapore Institute of Manufacturing Technology（新加坡制造技术研究院）
We propose a deep reinforcement learning (RL) method to learn large neighborhood search (LNS) policy for integer programming (IP). The RL policy is trained as the destroy operator to select a subset of variables at each step, which is reoptimized by an IP solver as the repair operator. However, the combinatorial number of variable subsets prevents direct application of typical RL algorithms. To tackle this challenge, we represent all subsets by factorizing them into binary decisions on each variable. We then design a neural network to learn policies for each variable in parallel, trained by a …
Balans: Multi-Armed Bandits-based Adaptive Large Neighborhood Search for Mixed-Integer Programming Problem
Department of Computer Science, University of Southern California（南加州大学计算机科学系）；AI Center of Excellence, Fidelity Investments（富达投资人工智能卓越中心）；Department of Computer Science, Brown University（布朗大学计算机科学系）
HyP-ASO: A Hybrid Policy-based Adaptive Search Optimization Framework for Large-Scale Integer Linear Programs
Ning Xu School of Mathematics and Statistics Nanjing University of Science and Technology（南京理工大学数学与统计学学院）；Junkai Zhang Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所）；Huigen Ye Department of Computer Science and Technology Tsinghua University（清华大学计算机科学与技术系）；Yang Wu Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所）；Hua Xu Department of Computer Science and Technology Tsinghua University（清华大学计算机科学与技术系）；Huiling Xu School of Mathematics and Statistics Nanjing University of Science and Technology（南京理工大学数学与统计学学院）；Yifan Zhang Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
US Air Force-MIT AI Accelerator（美国空军-麻省理工学院人工智能加速器）；MIT（麻省理工学院）
Integer programs provide a powerful abstraction for representing a wide range of real-world scheduling problems. Despite their ability to model general scheduling problems, solving large-scale integer programs (IP) remains a computational challenge in practice. The incorporation of more complex objectives such as robustness to disruptions further exacerbates the computational challenge. We present NICE (Neural network IP Coefficient Extraction), a novel technique that combines reinforcement learning and integer programming to tackle the problem of robust scheduling. More specifically, NICE use…
We present a data mining approach for reducing the search space of local search algorithms in a class of binary integer programs including the set covering and partitioning problems. The quality of locally optimal solutions typically improves if a larger neighborhood is used, while the computation time of searching the neighborhood increases exponentially. To overcome this, we extract variable associations from the instance to be solved in order to identify promising pairs of flipping variables in the neighborhood search. Based on this, we develop a 4-flip neighborhood local search algorithm t…
For an integer linear optimization (ILO) problem, persistency of its linear optimization (LO) relaxation is a property that for every optimal solution of the relaxation that assigns integer values to some variables, there exists an optimal solution of the ILO problem in which these variables retain the same values. Although persistency has been used to develop heuristic, approximation, and fixed-parameter algorithms for special cases of ILO, its applicability remains unknown in the literature. In this paper we reveal a maximal subclass of ILO such that its LO relaxation has persistency. Specif…
TU Darmstadt（达姆施塔特工业大学）；TU Berlin（柏林工业大学）；Zuse Institute Berlin（柏林祖斯研究所）
Recently non-convex optimization approaches for solving machine learning problems have gained significant attention. In this paper we explore non-convex boosting in classification by means of integer programming and demonstrate real-world practicability of the approach while circumventing shortcomings of convex boosting approaches. We report results that are comparable to or better than the current state-of-the-art.
We study the reformulation of integer linear programs by means of a mixed integer linear program with fewer integer variables. Such reformulations can be solved efficiently with mixed integer linear programming techniques. We exhibit examples that demonstrate how integer programs can be reformulated using far fewer integer variables. To this end, we introduce a generalization of total unimodularity called the \emph{affine TU-dimension} of a matrix and study related theory and algorithms for determining the affine TU-dimension of a matrix. We also present bounds on the number of integer variabl…
We consider discrete bilevel optimization problems where the follower solves an integer program with a fixed number of variables. Using recent results in parametric integer programming, we present polynomial time algorithms for pure and mixed integer bilevel problems. For the mixed integer case where the leader's variables are continuous, our algorithm also detects whether the infimum cost fails to be attained, a difficulty that has been identified but not directly addressed in the literature. In this case it yields a ``better than fully polynomial time'' approximation scheme with running time…
The notion of symmetry is defined in the context of Linear and Integer Programming. Symmetric integer programs are studied from a group theoretical viewpoint. We investigate the structure of integer solutions of integer programs and show that any integer program on n variables having an alternating group A_n as a group of symmetries can be solved in linear time in the number of variables.
We propose a new exact approach for solving integer linear programming (ILP) problems which we will call projective splitting algorithms (PSAs). Unlike classical methods for solving ILP problems, PSAs conduct the search for the optimal solution by generating candidate solutions tailored to specific values of the objective function. As a consequence of this strategy, the number of variables in the original ILP problem is systematically reduced without adding any additional constraint to the initial formulation. This is the first of a two-part series on PSAs. In this paper we focus on the resolu…
Binary Integer Programming (BIP) problems are of interest due in part to the difficulty they pose and because of their various applications, including those in graph theory, combinatorial optimization and network optimization. In this note, we explicitly state the Lovasz-Schrijver Semidefinite Programming (SDP) relaxation (in primal-standard form) for a BIP problem, a relaxation that yields a tighter upper-bound than the canonical Linear Programming relaxation.
In this paper, some deficiencies of a method proposed by Burdet and Johnson in 1977 for solving integer programming problems are discussed. Examples where the algorithm fails to solve the IP and ways to fix these errors are given.
In this paper, we propose a mixed-binary convex quadratic programming reformulation for the box-constrained nonconvex quadratic integer program and then implement IBM ILOG CPLEX 12.6 to solve the new model. Computational results demonstrate that our approach clearly outperform the very recent state-of-the-art solvers.
A standard quadratic program is an optimization problem that consists of minimizing a (nonconvex) quadratic form over the unit simplex. We focus on reformulating a standard quadratic program as a mixed integer linear programming problem. We propose two alternative mixed integer linear programming formulations. Our first formulation is based on casting a standard quadratic program as a linear program with complementarity constraints. We then employ binary variables to linearize the complementarity constraints. For the second formulation, we first derive an overestimating function of the objecti…
We introduce a new class of optimization problems called integer Minkowski programs. The formulation of such problems involves finitely many integer variables and nonlinear constraints involving functionals defined on families of discrete or polyhedral sets. We show that, under certain assumptions, it is possible to reformulate them as integer linear programs, by making use of integral generating sets. We then apply this technique to the network design problem for fractional and integral flows subject to survivability constraints.
The objective scaling ensemble approach is a novel two-phase heuristic for integer linear programming problems shown to be effective on a wide variety of integer linear programming problems. The technique identifies and aggregates multiple partial solutions to modify the problem formulation and significantly reduce the search space. An empirical analysis on publicly available benchmark problems demonstrate the efficacy of our approach by outperforming standard solution strategies implemented in modern optimization software.
The notion of symmetry is defined in the context of Linear and Integer Programming. Symmetric linear and integer programs are studied from a group theoretical viewpoint. We show that for any linear program there exists an optimal solution in the fixed point set of its symmetry group. Using this result, we develop an algorithm that allows for reducing the dimension of any linear program having a non-trivial group of symmetries.
We propose a trust-region method that solves a sequence of linear integer programs to tackle integer optimal control problems regularized with a total variation penalty. The total variation penalty allows us to prove the existence of minimizers of the integer optimal control problem. We introduce a local optimality concept for the problem, which arises from the infinite-dimensional perspective. In the case of a one-dimensional domain of the control function, we prove convergence of the iterates produced by our algorithm to points that satisfy first-order stationarity conditions for local optim…
Let $A$ be an integral nonnegative $m\times n$ matrix, $b$ be an integral nonnegative vector. It is suggested new method for reduction of integer program $\max \{cx| Ax=b, x\ge 0, x\in\mathbf{Z}^n\}$ to knapsack problem $\max \{c'x| fAx=fb, x\ge 0, x\in\mathbf{Z}^n\}$.
This paper deals with exploiting symmetry for solving linear and integer programming problems. Basic properties of linear representations of finite groups can be used to reduce symmetric linear programming to solving linear programs of lower dimension. Combining this approach with knowledge of the geometry of feasible integer solutions yields an algorithm for solving highly symmetric integer linear programs which only takes time which is linear in the number of constraints and quadratic in the dimension.
In this paper we give a new aggregation framework for linear Diophantine equations. In particular, we prove that an aggregated system of minimum size can be built in polynomial time. We also derive an analytic formula that gives the number of solutions of the system when it is possible to aggregate the system into one equation.
In this paper we generalize N-fold integer programs and two-stage integer programs with N scenarios to N-fold 4-block decomposable integer programs. We show that for fixed blocks but variable N, these integer programs are polynomial-time solvable for any linear objective. Moreover, we present a polynomial-time computable optimality certificate for the case of fixed blocks, variable N and any convex separable objective function. We conclude with two sample applications, stochastic integer programs with second-order dominance constraints and stochastic integer multi-commodity flows, which (for f…
This paper presents a mixed-integer quadratic programming formulation of an existing data-driven approach to computational elasticity. This formulation is suitable for application of a standard mixed-integer programming solver, which finds a global optimal solution. Therefore, the results obtained by the presented method can be used as benchmark instances for any other algorithm. Preliminary numerical experiments are performed to compare quality of solutions obtained by the proposed method and a heuristic used in the data-driven computational mechanics.
We consider the integer points in a unimodular cone K ordered by a lexicographic rule defined by a lattice basis. To each integer point x in K we associate a family of inequalities (lex-cuts) that defines the convex hull of the integer points in K that are not lexicographically smaller than x. The family of lex-cuts contains the Chvatal-Gomory cuts, but does not contain and is not contained in the family of split cuts. This provides a finite cutting plane method to solve the integer program min{cx : x \in S \cap Z^n }, where S \subset R^n is a compact set and c \in Z^n . We analyze the number …
We present simple compact proofs of the strong and weak duality theorems of tropical linear programming. It follows that there is no duality gap for a pair of tropical primal-dual problems. This result together with known properties of subeigenvectors enables us to directly solve a special tropical linear program with two-sided constraints. We also study the duality gap in tropical integer linear programming. A direct solution is available for the primal problem. An algorithm of quadratic complexity is presented for the dual problem. A direct solution is available provided that all coefficient…
The paper provides a connection between Commutative Algebra and Integer Programming and contains two parts. The first one is devoted to the asymptotic behavior of integer programs with a fixed cost linear functional and the constraint sets consisting of a finite system of linear equations or inequalities with integer coefficients depending linearly on $n$. An integer $N_*$ is determined such that the optima of these integer programs are a quasi-linear function of $n$ for all $n\ge N_*$. Using results in the first part, one can bound in the second part the indices of stability of the Castelnuov…
We present a new algorithm for computing a truncated Markov basis of a lattice. In general, this new algorithm is faster than existing methods. We then extend this new algorithm so that it solves the linear integer feasibility problem with promising results for equality knapsack problems. We also present a novel Groebner basis approach to solve a particular integer linear program as opposed to previous Groebner basis methods that effectively solved many different integer linear programs simultaneously. Initial results indicate that this optimisation algorithm performs better than previous Groe…
We overview our recently introduced theory of n-fold integer programming which enables the polynomial time solution of fundamental linear and nonlinear integer programming problems in variable dimension. We demonstrate its power by obtaining the first polynomial time algorithms in several application areas including multicommodity flows and privacy in statistical databases.
We investigate new methods for generating Lagrangian cuts to solve two-stage stochastic integer programs. Lagrangian cuts can be added to a Benders reformulation, and are derived from solving single scenario integer programming subproblems identical to those used in the nonanticipative Lagrangian dual of a stochastic integer program. While Lagrangian cuts have the potential to significantly strengthen the Benders relaxation, generating Lagrangian cuts can be computationally demanding. We investigate new techniques for generating Lagrangian cuts with the goal of obtaining methods that provide s…
This paper presents algorithms for solving multiobjective integer programming problems. The algorithm uses Barvinok's rational functions of the polytope that defines the feasible region and provides as output the entire set of nondominated solutions for the problem. Theoretical complexity results on the algorithm are provided in the paper. Specifically, we prove that encoding the entire set of nondominated solutions of the problem is polynomially doable, when the dimension of the decision space is fixed. In addition, we provide polynomial delay algorithms for enumerating this set. An implement…
We consider so-called squaring the square-puzzles where a given square (or rectangle) should be dissected into smaller squares. For a specific instance of such problems we demonstrate that a mathematically rigorous solution can be quite involved. As an alternative to exhaustive enumeration using tailored algorithms we describe the general approach of formulating the problem as an integer linear program.
A classic result of Cook et al. (1986) bounds the distances between optimal solutions of mixed-integer linear programs and optimal solutions of the corresponding linear relaxations. Their bound is given in terms of the number of variables and a parameter $ Δ$, which quantifies sub-determinants of the underlying linear inequalities. We show that this distance can be bounded in terms of $ Δ$ and the number of integer variables rather than the total number of variables. To this end, we make use of a result by Olson (1969) in additive combinatorics and demonstrate how it implies feasibility of cer…
We give an explicit geometric way to build mixed-integer programming (MIP) formulations for unions of polyhedra. The construction is simply described in terms of spanning hyperplanes in an r-dimensional linear space. The resulting MIP formulation is ideal, and uses exactly r integer variables and 2 x (# of spanning hyperplanes) general inequality constraints. We use this result to derive novel logarithmic-sized ideal MIP formulations for discontinuous piecewise linear functions and structures appearing in robotics and power systems problems.
The intention of this note is two-fold. First, we study integer optimization problems in standard form defined by $A \in\mathbb{Z}^{m\times{}n}$ and present an algorithm to solve such problems in polynomial-time provided that both the largest absolute value of an entry in $A$ and $m$ are constant. Then, this is applied to solve integer programs in inequality form in polynomial-time, where the absolute values of all maximal sub-determinants of $A$ lie between $1$ and a constant.
For over ten years, the constraint integer programming framework SCIP has been extended by capabilities for the solution of convex and nonconvex mixed-integer nonlinear programs (MINLPs). With the recently published version 8.0, these capabilities have been largely reworked and extended. This paper discusses the motivations for recent changes and provides an overview of features that are particular to MINLP solving in SCIP. Further, difficulties in benchmarking global MINLP solvers are discussed and a comparison with several state-of-the-art global MINLP solvers is provided.
We show that maximal $S$-free convex sets are polyhedra when $S$ is the set of integral points in some rational polyhedron of $\mathbb{R}^n$. This result extends a theorem of Lovász characterizing maximal lattice-free convex sets. Our theorem has implications in integer programming. In particular, we show that maximal $S$-free convex sets are in one-to-one correspondence with minimal inequalities.
It is well known that recurrent sandpile configurations can be characterized as the optimal solution of certain optimization problems. In this article, we present two new integer linear programming models, one that computes recurrent configurations and other that computes the order of the configuration. Finally, by using duality of linear programming, we are able to compute the identity configuration for the cone of a regular graph.
In this work, we present a modeling framework for minimax mixed 0-1 fuzzy linear problems. It is based on extending the usual rewriting of crisp minimax problems via auxiliary variables to model the maximum of a finite set of fuzzy linear functions. We establish that the considered problem can be equivalently formulated as a multiple objective mixed integer programming problem. The framework is applied to a fully fuzzy version of the capacitated center facility location problem.
Let X be the set of integer points in some polyhedron. We investigate the smallest number of facets of any polyhedron whose set of integer points is X. This quantity, which we call the relaxation complexity of X, corresponds to the smallest number of linear inequalities of any integer program having X as the set of feasible solutions that does not use auxiliary variables. We show that the use of auxiliary variables is essential for constructing polynomial size integer programming formulations in many relevant cases. In particular, we provide asymptotically tight exponential lower bounds on the…
The set of all group relaxations of an integer program contains certain special members called Gomory relaxations. A family of integer programs with a fixed coefficient matrix and cost vector but varying right hand sides is a Gomory family if every program in the family can be solved by one of its Gomory relaxations. In this paper, we characterize Gomory families. Every TDI system gives a Gomory family, and we construct Gomory families from matrices whose columns form a Hilbert basis for the cone they generate. The existence of Gomory families is related to the Hilbert covering problems that a…
Cutting planes are a key ingredient to successfully solve mixed-integer linear programs. For specific problems, their strength is often theoretically assessed by showing that they are facet-defining for the corresponding mixed-integer hull. In this paper we experimentally investigate the dimensions of faces induced by general-purpose cutting planes generated by a state-of-the-art solver. Therefore, we relate the dimension of each cutting plane to its impact in a branch-and-bound algorithm.
We consider the constrained assortment optimization problem under the mixed multinomial logit model. Even moderately sized instances of this problem are challenging to solve directly using standard mixed-integer linear optimization formulations. This has motivated recent research exploring customized optimization strategies and approximation techniques. In contrast, we develop a novel conic quadratic mixed-integer formulation. This new formulation, together with McCormick inequalities exploiting the capacity constraints, enables the solution of large instances using commercial optimization sof…
We address the Interval Data Min-Max Regret 0-1 Integer Linear Programming problem (MMR-ILP), a variant of the 0-1 Integer Linear Programming problem where the objective function coefficients are uncertain. We solve MMR-ILP using a Benders-like Decomposition Algorithm and two metaheuristics for min-max regret problems with interval data. Computational experiments developed on variations of MIPLIB instances show that the heuristics obtain good results in a reasonable computational time when compared to the Benders-like Decomposition algorithm.
This paper addresses the single-item single-stocking location stochastic lot sizing problem under the $(s, S) $ policy. We first present a mixed integer non-linear programming (MINLP) formulation for determining near-optimal $(s, S)$ policy parameters. To tackle larger instances, we then combine the previously introduced MINLP model and a binary search approach. These models can be reformulated as mixed integer linear programming (MILP) models which can be easily implemented and solved by using off-the-shelf optimisation software. Computational experiments demonstrate that optimality gaps of t…
Motivated by their success in the single-objective domain, we propose a very simple linear programming-based matheuristic for tri-objective binary integer programming. To tackle the problem, we obtain lower bound sets by means of the vector linear programming solver Bensolve. Then, simple heuristic approaches, such as rounding and path relinking, are applied to this lower bound set to obtain high-quality approximations of the optimal set of trade-off solutions. The proposed algorithm is compared to a recently suggested algorithm which is, to the best of our knowledge, the only existing matheur…
