A polyhedral active set algorithm PASA is developed for solving a nonlinear optimization problem whose feasible set is a polyhedron. Phase one of the algorithm is the gradient projection method, while phase two is any algorithm for solving a linearly constrained optimization problem. Rules are provided for branching between the two phases. Global convergence to a stationary point is established, while asymptotically PASA performs only phase two when either a nondegeneracy assumption holds, or the active constraints are linearly independent and a strong second-order sufficient optimality condition holds.
The Polyhedral Active Set Algorithm (PASA) is designed to optimize a general nonlinear function over a polyhedron. Phase one of the algorithm is a nonmonotone gradient projection algorithm, while phase two is an active set algorithm that explores faces of the constraint polyhedron. A gradient-based implementation is presented, where a projected version of the conjugate gradient algorithm is employed in phase two. Asymptotically, only phase two is performed. Comparisons are given with IPOPT using polyhedral constrained problems from CUTEst and the Maros/Meszaros quadratic programming test set.
A Newton-type active set algorithm for large-scale minimization subject to polyhedral constraints is proposed. The algorithm consists of a gradient projection step, a second-order Newton-type step in the null space of the constraint matrix, and a set of rules for branching between the two steps. We show that the proposed method asymptotically takes the Newton step when the active constraints are linearly independent and a strong second-order sufficient optimality condition holds. We also show that the method has a quadratic rate of convergence under standard conditions. Numerical experiments a…
Model predictive control (MPC) is of increasing interest in applications for constrained control of multivariable systems. However, one of the major obstacles to its broader use is the computation time and effort required to solve a possibly non-convex optimal control problem (OCP) online. This paper introduces a sensitivity-based warmstarting strategy for systems with nonlinear dynamics and polyhedral constraints with the goal of reducing the computational footprint of MPC controllers. It predicts changes in the solution of the parameterized OCP as the parameter varies, by calculating the sem…
Given a nonlinear, univariate, bounded, and differentiable function $f(x)$, this article develops a sequence of Mixed Integer Linear Programming (MILP) and Linear Programming (LP) relaxations that converge to the graph of $f(x)$ and its convex hull, respectively. Theoretical convergence of the sequence of relaxations to the graph of the function and its convex hull is established. For nonlinear non-convex optimization problems, the relaxations presented in this article can be used to construct tight MILP and LP relaxations. These MILP and the LP relaxations can also be used with MILP-based and…
In this paper we study some relationships between polyhedral convex sets (PCS) and generalized polyhedral convex sets (GPCS). In particular, we clarify by a counterexample that the necessary and sufficient conditions for the separation of a convex set and a PCS obtained by Kung Fu Ng and Wen Song in [Fenchel duality in finite-dimensional setting and its applications, Nonlinear Anal. 55(2003), 845--858; Theorem~3.1] are no longer valid when considering GPCS instead of PCS. We also introduce and study the notions of generalized polyhedral set-valued mappings and optimal value functions generated…
Coordinate-wise Polyhedral Method for Eliciting Multivariate Linear Utility and Univariate Nonlinear Utility Functions
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong（香港中文大学系统科学与工程学系）
We provide novel dissipativity conditions for bounding the incremental L-1 gain of systems. Moreover, we adapt existing results on the L-infinity gain to the incremental setting and relate the incremental L-1 and L-infinity gain bounds through system adjoints. Building on work on optimization based approaches to constructing polyhedral Lyapunov functions, we make use of these conditions to obtain a Linear Programming based algorithm that can provide increasingly sharp bounds on the gains as a function of a given candidate polyhedral storage function or polyhedral set. The algorithm is also ext…
The classic pump scheduling or Optimal Water Flow (OWF) problem for water distribution networks (WDNs) minimizes the cost of power consumption for a given WDN over a fixed time horizon. In its exact form, the OWF is a computationally challenging mixed-integer nonlinear program (MINLP). It is complicated by nonlinear equality constraints that model network physics, discrete variables that model operational controls, and intertemporal constraints that model changes to storage devices. To address the computational challenges of the OWF, this paper develops tight polyhedral relaxations of the orig…
In this paper, we provide local convergence analysis for the two phase Nonlinear Polyhedral Active Set Algorithm (NPASA) designed to solve nonlinear programs. In particular, we establish local quadratic convergence of the primal iterates and global error estimator for NPASA under reasonable assumptions. Additionally, under the same set of assumptions we prove that only phase two of NPASA is executed after finitely many iterations. This paper is companion to a paper that provides motivation and global convergence analysis for NPASA.
Polyhedral estimate is a generic efficiently computable nonlinear in observations routine for recovering unknown signal belonging to a given convex compact set from noisy observation of signal's linear image. Risk analysis and optimal design of polyhedral estimates may be addressed through efficient bounding of optimal values of optimization problems. Such problems are typically hard; yet, it was shown in Juditsky, Nemirovski 2019 that nearly minimax optimal ("up to logarithmic factors") estimates can be efficiently constructed when the signal set is an ellitope - a member of a wide family of …
In this paper, we present a two phase method for solving nonlinear programming problems called Nonlinear Polyhedral Active Set Algorithm (NPASA) that has global and local convergence guarantees under reasonable assumptions. The first phase consists of an augmented Lagrangian method to ensure global convergence while the second phase is designed to promote fast local convergence by performing a balanced reduction of two error estimators for nonlinear programs. After presenting error estimators for nonlinear programs and our algorithm NPASA, we establish global convergence properties for NPASA. …
In this paper, we will demonstrate how to use a nonlinear polyhedral constrained optimization solver called the Polyhedral Active Set Algorithm (PASA) for solving a general singular control problem. We present methods of discretizing a general optimal control problem that involves the use of the gradient of the Lagrangian for computing the gradient of the cost functional so that PASA can be applied. When a numerical solution contains artifacts that resemble "chattering'', a phenomenon where the control oscillates wildly along the singular region, we recommend a method of regularizing the singu…
We investigate new convex relaxations for the pooling problem, a classic nonconvex production planning problem in which input materials are mixed in intermediate pools, with the outputs of these pools further mixed to make output products meeting given attribute percentage requirements. Our relaxations are derived by considering a set which arises from the formulation by considering a single product, a single attibute, and a single pool. The convex hull of the resulting nonconvex set is not polyhedral. We derive valid linear and convex nonlinear inequalities for the convex hull, and demonstrat…
Northwestern University（西北大学）；University of California Berkeley（加州大学伯克利分校）；University of Southern California（南加州大学）
We consider the convex quadratic optimization problem with indicator variables and arbitrary constraints on the indicators. We show that a convex hull description of the associated mixed-integer set in an extended space with a quadratic number of additional variables consists of a single positive semidefinite constraint (explicitly stated) and linear constraints. In particular, convexification of this class of problems reduces to describing a polyhedral set in an extended formulation. While the vertex representation of this polyhedral set is exponential and an explicit linear inequality descri…
This work proposes a framework for multistage adjustable robust optimization that unifies the treatment of three different types of endogenous uncertainty, where decisions, respectively, (i) alter the uncertainty set, (ii) affect the materialization of uncertain parameters, and (iii) determine the time when the true values of uncertain parameters are observed. We provide a systematic analysis of the different types of endogenous uncertainty and highlight the connection between optimization under endogenous uncertainty and active learning. We consider decision-dependent polyhedral uncertainty s…
This work is devoted to the development of a distributionally robust active fault diagnosis approach for a class of nonlinear systems, which takes into account any ambiguity in distribution information of the uncertain model parameters. More specifically, a new approach is presented using the total variation distance metric as an information constraint, and as a measure for the separation of multiple models based on the similarity of their output probability density functions. A practical aspect of the proposed approach is that different levels of ambiguity may be assigned to the models pertai…
In this paper, we study the mixed-integer nonlinear set given by a separable quadratic constraint on continuous variables, where each continuous variable is controlled by an additional indicator. This set occurs pervasively in optimization problems with uncertainty and in machine learning. We show that optimization over this set is NP-hard. Despite this negative result, we characterize the structure of the convex hull, and show that it can be formally studied using polyhedral theory. Moreover, we show that although perspective relaxation in the literature for this set fails to match the struct…
While the identification of nonlinear dynamical systems is a fundamental building block of model-based reinforcement learning and feedback control, its sample complexity is only understood for systems that either have discrete states and actions or for systems that can be identified from data generated by i.i.d. random inputs. Nonetheless, many interesting dynamical systems have continuous states and actions and can only be identified through a judicious choice of inputs. Motivated by practical settings, we study a class of nonlinear dynamical systems whose state transitions depend linearly on…
Argonne National Laboratory（阿贡国家实验室）；The University of Chicago（芝加哥大学）；University of California, Berkeley（加州大学伯克利分校）；International Computer Science Institute（国际计算机科学研究所）
We study nonlinear optimization problems with a stochastic objective and deterministic equality and inequality constraints, which emerge in numerous applications including finance, manufacturing, power systems and, recently, deep neural networks. We propose an active-set stochastic sequential quadratic programming (StoSQP) algorithm that utilizes a differentiable exact augmented Lagrangian as the merit function. The algorithm adaptively selects the penalty parameters of the augmented Lagrangian and performs a stochastic line search to decide the stepsize. The global convergence is established:…
Active Queue Management (AQM) for mitigating Internet congestion has been addressed via various feedback control syntheses, especially P, PI, and PID regulators, by using a linear approximation where the ``round trip time'', i.e., the delay, is assumed to be constant. This constraint is lifted here by using a nonlinear modeling with a variable delay, introduced more than 20 years ago. This delay, intimately linked to the congestion phenomenon, may be viewed as a ``flat output.'' All other system variables, especially the control variable, i.e., the packet loss ratio, are expressed as a functio…
