In this paper, we study the convex quadratic optimization problem with indicator variables. For the bivariate case, we describe the convex hull of the epigraph in the original space of variables, and also give a conic quadratic extended formulation. Then, using the convex hull description for the bivariate case as a building block, we derive an extended SDP relaxation for the general case. This new formulation is stronger than other SDP relaxations proposed in the literature for the problem, including Shor's SDP relaxation, the optimal perspective relaxation as well as the optimal rank-one relaxation. Computational experiments indicate that the proposed formulations are quite effective in reducing the integrality gap of the optimization problems.
We consider in this paper a class of semi-continuous quadratic programming problems which arises in many real-world applications such as production planning, portfolio selection and subset selection in regression. We propose a lift-and-convexification approach to derive an equivalent reformulation of the original problem. This lift-and-convexification approach lifts the quadratic term involving $x$ only in the original objective function $f(x,y)$ to a quadratic function of both $x$ and $y$ and convexifies this equivalent objective function. While the continuous relaxation of our new reformulat…
Random projection techniques based on Johnson-Lindenstrauss lemma are used for randomly aggregating the constraints or variables of optimization problems while approximately preserving their optimal values, that leads to smaller-scale optimization problems. D'Ambrosio et al. have applied random projection to a quadratic optimization problem so as to decrease the number of decision variables. Although the problem size becomes smaller, the projected problem will also almost surely be non-convex if the original problem is non-convex, and hence will be hard to solve. In this paper, by focusing on …
Northwestern University（西北大学）；University of Southern California（南加州大学）
Motivated by modern regression applications, in this paper, we study the convexification of a class of convex optimization problems with indicator variables and combinatorial constraints on the indicators. Unlike most of the previous work on convexification of sparse regression problems, we simultaneously consider the nonlinear non-separable objective, indicator variables, and combinatorial constraints. Specifically, we give the convex hull description of the epigraph of the composition of a one-dimensional convex function and an affine function under arbitrary combinatorial constraints. As sp…
We study the generalization of split, k-branch split, and intersection cuts from Mixed Integer Linear Programming to the realm of Mixed Integer Nonlinear Programming. Constructing such cuts requires calculating the convex hull of the difference between a convex set and an open set with a simple geometric structure. We introduce two techniques to give precise characterizations of such convex hulls and use them to construct split, k-branch split, and intersection cuts for several classes of non-polyhedral sets. In particular, we give simple formulas for split cuts for essentially all convex sets…
We efficiently treat bilinear forms in the context of global optimization, by applying McCormick convexification and by extending an approach of Saxena, Bonami and Lee for symmetric quadratic forms to bilinear forms. A key application of our work is in treating "structural convexity" in a symmetric quadratic form.
This paper presents the SCvx algorithm, a successive convexification algorithm designed to solve non-convex constrained optimal control problems with global convergence and superlinear convergence-rate guarantees. The proposed algorithm can handle nonlinear dynamics and non-convex state and control constraints. It solves the original problem to optimality by successively linearizing non-convex dynamics and constraints about the solution of the previous iteration. The resulting convex subproblems are numerically tractable, and can be computed quickly and reliably using convex optimization solve…
The uncertainty of multiple power loads and renewable energy generations (PLREG) in power systems increases the complexity of power flow analysis for decision-makers. The chance-constrained method can be applied to model the optimization problems of power flow under uncertainty. This paper develops a novel solution approach for chance-constrained AC optimal power flow (CCACOPF) problem based on the data-driven convexification of power flow and a fast algorithm for scenario technique (FAST). This method is computationally effective for mainly two reasons. First, the original nonconvex AC power …
In this paper, we compare the strength of the optimal perspective reformulation and Shor's SDP relaxation. We prove these two formulations are equivalent for quadratic optimization problems with indicator variables.
We study quadratic optimization with indicator variables and an M-matrix, i.e., a PSD matrix with non-positive off-diagonal entries, which arises directly in image segmentation and portfolio optimization with transaction costs, as well as a substructure of general quadratic optimization problems. We prove, under mild assumptions, that the minimization problem is solvable in polynomial time by showing its equivalence to a submodular minimization problem. To strengthen the formulation, we decompose the quadratic function into a sum of simple quadratic functions with at most two indicator variabl…
In this paper, we consider convex quadratic optimization problems with indicator variables when the matrix $Q$ defining the quadratic term in the objective is sparse. We use a graphical representation of the support of $Q$, and show that if this graph is a path, then we can solve the associated problem in polynomial time. This enables us to construct a compact extended formulation for the closure of the convex hull of the epigraph of the mixed-integer convex problem. Furthermore, we propose a novel decomposition method for general (sparse) $Q$, which leverages the efficient algorithm for the p…
In this paper, we study the mixed-integer nonlinear set given by a separable quadratic constraint on continuous variables, where each continuous variable is controlled by an additional indicator. This set occurs pervasively in optimization problems with uncertainty and in machine learning. We show that optimization over this set is NP-hard. Despite this negative result, we characterize the structure of the convex hull, and show that it can be formally studied using polyhedral theory. Moreover, we show that although perspective relaxation in the literature for this set fails to match the struct…
University of California, Berkeley（加州大学伯克利分校）；University of Pittsburgh（匹兹堡大学）
Sparse regression models are increasingly prevalent due to their ease of interpretability and superior out-of-sample performance. However, the exact model of sparse regression with an $\ell_0$ constraint restricting the support of the estimators is a challenging (\NP-hard) non-convex optimization problem. In this paper, we derive new strong convex relaxations for sparse regression. These relaxations are based on the ideal (convex-hull) formulations for rank-one quadratic terms with indicator variables. The new relaxations can be formulated as semidefinite optimization problems in an extended s…
We consider the nonconvex set $\mathcal S_n = \{(x,X,z): X = x x^T, \; x (1-z) =0,\; x \geq 0,\; z \in \{0,1\}^n\}$, which is closely related to the feasible region of several difficult nonconvex optimization problems such as the best subset selection and constrained portfolio optimization. Utilizing ideas from convex analysis and disjunctive programming, we obtain an explicit description for the closure of the convex hull of $\mathcal S_2$ in the space of original variables. In order to generate valid inequalities corresponding to supporting hyperplanes of the convex hull of $\mathcal S_2$, w…
Northwestern University（西北大学）；University of California Berkeley（加州大学伯克利分校）；University of Southern California（南加州大学）
We consider the convex quadratic optimization problem with indicator variables and arbitrary constraints on the indicators. We show that a convex hull description of the associated mixed-integer set in an extended space with a quadratic number of additional variables consists of a single positive semidefinite constraint (explicitly stated) and linear constraints. In particular, convexification of this class of problems reduces to describing a polyhedral set in an extended formulation. While the vertex representation of this polyhedral set is exponential and an explicit linear inequality descri…
University of California Berkeley（加州大学伯克利分校）；University of Southern California（南加州大学）
We study the minimization of a rank-one quadratic with indicators and show that the underlying set function obtained by projecting out the continuous variables is supermodular. Although supermodular minimization is, in general, difficult, the specific set function for the rank-one quadratic can be minimized in linear time. We show that the convex hull of the epigraph of the quadratic can be obtaining from inequalities for the underlying supermodular set function by lifting them into nonlinear inequalities in the original space of variables. Explicit forms of the convex-hull description are giv…
We study the trust-region subproblem (TRS) of minimizing a nonconvex quadratic function over the unit ball with additional conic constraints. Despite having a nonconvex objective, it is known that the classical TRS and a number of its variants are polynomial-time solvable. In this paper, we follow a second-order cone (SOC) based approach to derive an exact convex reformulation of the TRS under a structural condition on the conic constraint. Our structural condition is immediately satisfied when there is no additional conic constraints, and it generalizes several such conditions studied in the …
We investigate a mixed 0-1 conic quadratic optimization problem with indicator variables arising in mean-risk optimization. The indicator variables are often used to model non-convexities such as fixed charges or cardinality constraints. Observing that the problem reduces to a submodular function minimization for its binary restriction, we derive three classes of strong convex valid inequalities by lifting the polymatroid inequalities on the binary variables. Computational experiments demonstrate the effectiveness of the inequalities in strengthening the convex relaxations and, thereby, improv…
We consider quadratic optimization in variables $(x,y)$ where $0\le x\le y$, and $y\in\{0,1\}^n$. Such binary $y$ are commonly refered to as "indicator" or "switching" variables and occur commonly in applications. One approach to such problems is based on representing or approximating the convex hull of the set $\{ (x,xx^T, yy^T) : 0\le x\le y\in\{0,1\}^n\}$. A representation for the case $n=1$ is known and has been widely used. We give an exact representation for the case $n=2$ by starting with a disjunctive representation for the convex hull and then eliminating auxilliary variables and cons…
This paper investigates the collaboration of multiple connected and automated vehicles (CAVs) in different scenarios. In general, the collaboration of CAVs can be formulated as a nonlinear and nonconvex model predictive control (MPC) problem. Most of the existing approaches available for utilization to solve such an optimization problem suffer from the drawback of considerable computational burden, which hinders the practical implementation in real time. This paper proposes the use of sequential convex programming (SCP), which is a powerful approach to solving the nonlinear and nonconvex MPC p…
We present a framework to build a multiobjective algorithm from single-objective ones. This framework addresses the $p \times n$-dimensional problem of finding p solutions in an n-dimensional search space, maximizing an indicator by dynamic subspace optimization. Each single-objective algorithm optimizes the indicator function given $p - 1$ fixed solutions. Crucially, dominated solutions minimize their distance to the empirical Pareto front defined by these $p - 1$ solutions. We instantiate the framework with CMA-ES as single-objective optimizer. The new algorithm, COMO-CMA-ES, is empirically …
This paper deals with the problem of finding the globally optimal subset of h elements from a larger set of n elements in d space dimensions so as to minimize a quadratic criterion, with an special emphasis on applications to computing the Least Trimmed Squares Estimator (LTSE) for robust regression. The computation of the LTSE is a challenging subset selection problem involving a nonlinear program with continuous and binary variables, linked in a highly nonlinear fashion. The selection of a globally optimal subset using the branch and bound (BB) algorithm is limited to problems in very low di…
We study a class of convex-concave saddle-point problems of the form $\min_x\max_y \langle Kx,y\rangle+f_{\cal{P}}(x)-h^\ast(y)$ where $K$ is a linear operator, $f_{\cal{P}}$ is the sum of a convex function $f$ with a Lipschitz-continuous gradient and the indicator function of a bounded convex polytope $\cal{P}$, and $h^\ast$ is a convex (possibly nonsmooth) function. Such problem arises, for example, as a Lagrangian relaxation of various discrete optimization problems. Our main assumptions are the existence of an efficient linear minimization oracle ($lmo$) for $f_{\cal{P}}$ and an efficient …
Sorbonne Université（索邦大学）；Inria（法国国家科学研究中心）；Centre Inria de Sorbonne Université（索邦大学Inria中心）；CNRS（法国国家科学研究中心）；ENPC（巴黎高等理工学院）；Institut Polytechnique de Paris（巴黎高等理工学院）；CERMICS（CERMICS研究中心）
We provide new insight into the convergence properties of the Douglas-Rachford algorithm for the problem $\min_x \{f(x)+g(x)\}$, where $f$ and $g$ are convex functions. Our approach relies on and highlights the natural primal-dual symmetry between the above problem and its Fenchel dual $\min_{u} \{ f^*(u) + g_*(u)\}$ where $g_*(u):=g^*(-u)$. Our main development is to show the linear convergence of the algorithm when a natural error bound condition on the Douglas-Rachford operator holds. We leverage our error bound condition approach to show and estimate the algorithm's linear rate of converge…
Northwestern University（西北大学）；University of Washington（华盛顿大学）；National Yang Ming Chiao Tung University（阳明交通大学）
Bayesian Networks (BNs) represent conditional probability relations among a set of random variables (nodes) in the form of a directed acyclic graph (DAG), and have found diverse applications in knowledge discovery. We study the problem of learning the sparse DAG structure of a BN from continuous observational data. The central problem can be modeled as a mixed-integer program with an objective function composed of a convex quadratic loss function and a regularization penalty subject to linear constraints. The optimal solution to this mathematical program is known to have desirable statistical …
