We develop an algorithmic theory of convex optimization over discrete sets. Using a combination of algebraic and geometric tools we are able to provide polynomial time algorithms for solving broad classes of convex combinatorial optimization problems and convex integer programming problems in variable dimension. We discuss some of the many applications of this theory including to quadratic programming, matroids, bin packing and cutting-stock problems, vector partitioning and clustering, multiway transportation problems, and privacy and confidential statistical data disclosure. Highlights of our work include a strongly polynomial time algorithm for convex and linear combinatorial optimization over any family presented by a membership oracle when the underlying polytope has few edge-directions; a new theory of so-termed n-fold integer programming, yielding polynomial time solution of important and natural classes of convex and linear integer programming problems in variable dimension; and a complete complexity classification of high dimensional transportation problems, with practical applications to fundamental problems in privacy and confidential statistical data disclosure.
Discrete convex functions are used in many areas, including operations research, discrete-event systems, game theory, and economics. The objective of this paper is to investigate basic operations such as direct sum, splitting, and aggregation that are related to network induction of discrete convex functions as well as discrete convex sets. Various kinds of discrete convex functions in discrete convex analysis are considered such as integrally convex functions, L-convex functions, M-convex functions, multimodular functions, and discrete midpoint convex functions.
Discrete convex functions are used in many areas, including operations research, discrete-event systems, game theory, and economics. The objective of this paper is to offer a survey on fundamental operations for various kinds of discrete convex functions in discrete convex analysis such as integrally convex functions, L-convex functions, M-convex functions, and multimodular functions.
For continuous functions, midpoint convexity characterizes convex functions. By considering discrete versions of midpoint convexity, several types of discrete convexities of functions, including integral convexity, L$^\natural$-convexity and global/local discrete midpoint convexity, have been studied. We propose a new type of discrete midpoint convexity that lies between L$^\natural$-convexity and integral convexity and is independent of global/local discrete midpoint convexity. The new convexity, named DDM-convexity, has nice properties satisfied by L$^\natural$-convexity and global/local dis…
We present a novel convex relaxation and a corresponding inference algorithm for the non-binary discrete tomography problem, that is, reconstructing discrete-valued images from few linear measurements. In contrast to state of the art approaches that split the problem into a continuous reconstruction problem for the linear measurement constraints and a discrete labeling problem to enforce discrete-valued reconstructions, we propose a joint formulation that addresses both problems simultaneously, resulting in a tighter convex relaxation. For this purpose a constrained graphical model is set up a…
We introduce the convex combinatorial optimization problem, a far reaching generalization of the standard linear combinatorial optimization problem. We show that it is strongly polynomial time solvable over any edge-guaranteed family, and discuss several applications.
We give a characterization of the controllability for discrete-time linear systems with convex output constraints. It extends all previously known characterizations in the literature, as well as our previous results on controllability of discrete-time linear systems with conic output constraints.
The present article is an exposition of a theory of discrete convex functions on certain graph structures, developed by the author in recent years. This theory is a spin-off of discrete convex analysis by Murota, and is motivated by combinatorial dualities in multiflow problems and the complexity classification of facility location problems on graphs. We outline the theory and algorithmic applications in combinatorial optimization problems.
This paper considers projection and convolution operations for integrally convex functions, which constitute a fundamental function class in discrete convex analysis. It is shown that the class of integrally convex functions is stable under projection, and this is also the case with the subclasses of integrally convex functions satisfying local or global discrete midpoint convexity. As is known in the literature, the convolution of two integrally convex functions may possibly fail to be integrally convex. We show that the convolution of an integrally convex function with a separable convex fun…
Integrally convex functions constitute a fundamental function class in discrete convex analysis. This paper shows that an integer-valued integrally convex function admits an integral subgradient and that the integral biconjugate of an integer-valued integrally convex function coincides with itself. The proof is based on the Fourier-Motzkin elimination. The latter result provides a unified proof of integral biconjugacy for various classes of integer-valued discrete convex functions, including L-convex, M-convex, L$_{2}$-convex, M$_{2}$-convex, BS-convex, and UJ-convex functions as well as multi…
We study a specific convex maximization problem in n-dimensional space. The conjectured solution is proved to be a vertex of the polyhedral feasible region, but only a partial proof of local maximality is known. Integer sequences with interesting patterns arise in the analysis, owing to the number theoretic origin of the problem.
Differential stability of convex discrete optimal control problems in Banach spaces is studied in this paper. By using some recent results of An and Yen [Appl. Anal. 94, 108--128 (2015)] on differential stability of parametric convex optimization problems under inclusion constraints, we obtain an upper estimate for the subdifferential of the optimal value function of a parametric convex discrete optimal control problem, where the objective function may be nondifferentiable. If the objective function is differentiable, the obtained upper estimate becomes an equality. It is shown that the singul…
In this paper we introduce the Ameso optimization problem, a special class of discrete optimization problems. We establish its basic properties and investigate the relation between Ameso optimization and the convex optimization. Further, we design an algorithm to solve a multi-dimensional Ameso problem by solving a sequence of one-dimensional Ameso problems. Finally, we demonstrate how the knapsack problem can be solved using the Ameso optimization framework.
We consider a problem of optimizing convex functionals over matroid bases. It is richly expressive and captures certain quadratic assignment and clustering problems. While generally NP-hard, we show it is polynomial time solvable when a suitable parameter is restricted.
In this paper we show that a convexifiability property of nonconvex quadratic programs with nonnegative variables and quadratic constraints guarantees zero duality gap between the quadratic programs and their semi-Lagrangian duals. More importantly, we establish that this convexifiability is hidden in classes of nonnegative homogeneous quadratic programs and discrete quadratic programs, such as mixed integer quadratic programs, revealing zero duality gaps. As an application, we prove that robust counterparts of uncertain mixed integer quadratic programs with objective data uncertainty enjoy ze…
In this paper, we study the problem of finding the Euclidean distance to a convex cone generated by a set of discrete points in $\mathbb{R}^n_+$. In particular, we are interested in problems where the discrete points are the set of feasible solutions of some binary linear programming constraints. This problem has applications in manufacturing, machine learning, clustering, pattern recognition, and statistics. Our problem is a high-dimensional constrained optimization problem. We propose a Frank-Wolfe based algorithm to solve this non-convex optimization problem with a convex-noncompact feasibl…
In this paper, we bring the celebrated max-weight features (myopic and discrete actions) to mainstream convex optimization. Myopic actions are important in control because decisions need to be made in an online manner and without knowledge of future events, and discrete actions because many systems have a finite (so non-convex) number of control decisions. For example, whether to transmit a packet or not in communication networks. Our results show that these two features can be encompassed in the subgradient method for the Lagrange dual problem by the use of stochastic and $ε$-subgradients. On…
We provide proof that the optimal value function of a convex parametrized optimization problem in Euclidean spaces is itself a convex function onto the extended real line.
We propose new sequential simulation-optimization algorithms for general convex optimization via simulation problems with high-dimensional discrete decision space. The performance of each choice of discrete decision variables is evaluated via stochastic simulation replications. If an upper bound on the overall level of uncertainties is known, our proposed simulation-optimization algorithms utilize the discrete convex structure and are guaranteed with high probability to find a solution that is close to the best within any given user-specified precision level. The proposed algorithms work for a…
Geometric Conditions for Lossless Convexification in Linear Optimal Control with Discrete-Valued Inputs: Real-Time Implementation for Spacecraft Rendezvous
We consider stochastic convex optimization with a strongly convex (but not necessarily smooth) objective. We give an algorithm which performs only gradient updates with optimal rate of convergence.
We establish that the min-sum message-passing algorithm and its asynchronous variants converge for a large class of unconstrained convex optimization problems.
In this paper we develop and analyse convex searches for Zames--Falb multipliers. We present two different approaches: Infinite Impulse Response (IIR) and Finite Impulse Response (FIR) multipliers. The set of FIR multipliers is complete in that any IIR multipliers can be phase-substituted by an arbitrarily large order FIR multiplier. We show that searches in discrete-time for FIR multipliers are effective even for large orders. As expected, the numerical results provide the best $\ell_{2}$-stability results in the literature for slope-restricted nonlinearities. Finally, we demonstrate that the…
We suggest a new greedy strategy for convex optimization in Banach spaces and prove its convergent rates under a suitable behavior of the modulus of uniform smoothness of the objective function.
This paper presents a convex sufficient condition for solving a system of nonlinear equations under parametric changes and proposes a sequential convex optimization method for solving robust optimization problems with nonlinear equality constraints. By bounding the nonlinearity with concave envelopes and using Brouwer's fixed point theorem, the sufficient condition is expressed in terms of closed-form convex inequality constraints. We extend the result to provide a convex sufficient condition for feasibility under bounded uncertainty. Using these conditions, a non-convex optimization problem c…
Motivated by the fact that not all nonconvex optimization problems are difficult to solve, we survey in this paper three widely-used ways to reveal the hidden convex structure for different classes of nonconvex optimization problems. Finally, ten open problems are raised.
This short communication develops a convex dual variational formulation for a large class of models in variational optimization. The results are established through basic tools of functional analysis, convex analysis and duality theory. The main duality principle is developed as an application to a Ginzburg-Landau type system in superconductivity in the absence of a magnetic field.
Random projection techniques based on Johnson-Lindenstrauss lemma are used for randomly aggregating the constraints or variables of optimization problems while approximately preserving their optimal values, that leads to smaller-scale optimization problems. D'Ambrosio et al. have applied random projection to a quadratic optimization problem so as to decrease the number of decision variables. Although the problem size becomes smaller, the projected problem will also almost surely be non-convex if the original problem is non-convex, and hence will be hard to solve. In this paper, by focusing on …
In Hilbert space, we propose a family of primal-dual dynamical system for affine constrained convex optimization problem. Several damping coefficients, time scaling coefficients, and perturbation terms are thus considered. By constructing the energy functions, we investigate the convergence rates with different choices of the damping coefficients and time scaling coefficients. Our results extend the inertial dynamical approaches for unconstrained convex optimization problems to affine constrained convex optimization problems.
A new and simple method for quasi-convex optimization is introduced from which its various applications can be derived. Especially, a global optimum under constrains can be approximated for all continuous functions.
This paper presents a piecewise convexification method for solving non-convex multi-objective optimization problems with box constraints. Based on the ideas of the $α$-based Branch and Bound (${\rm αBB}$) method of global optimization and the interval subdivision, a series of convex relaxation sub-multiobjective problems for this non-convex multi-objective optimization problem are firstly obtained, and these sub-problems constitute a piecewise convexification problem of the original problem on the whole box. We then construct the (approximate, weakly) efficient solution set of this piecewise c…
In this paper, we study a class of finite-time control problems for discrete-time positive linear systems with time-varying state parameters. Although several interesting control problems appearing in population biology, economics, and network epidemiology can be described as the class of finite-time control problems, an efficient solution to the control problem has not been yet found in the literature. In this paper, we propose an optimization framework for solving the class of finite-time control problems via convex optimization. We illustrate the effectiveness of the proposed method by nume…
CVXPY is a domain-specific language for convex optimization embedded in Python. It allows the user to express convex optimization problems in a natural syntax that follows the math, rather than in the restrictive standard form required by solvers. CVXPY makes it easy to combine convex optimization with high-level features of Python such as parallelism and object-oriented design. CVXPY is available at http://www.cvxpy.org/ under the GPL license, along with documentation and examples.
Our aim in this article is two-fold. We use the Charnes-Cooper scalarization technique to develop KKT type conditions to completely characterize Pareto minimizers of convex vector optimization problems and further, we use that scalarization technique to develop a simple and efficient algorithm for convex vector optimization problems. Numerical examples are presented to illustrate the use of our algorithm.
In this paper we consider the solvability of a non-convex regular polynomial vector optimization problem on a nonempty closed set. We introduce regularity conditions for the polynomial vector optimization problem and study properties and characterizations of the regularity conditions. Under the regularity conditions, we study nonemptiness and boundedness of the solution sets of the problem. As a consequence, we establish two Frank-Wolfe type theorems for the non-convex polynomial vector optimization problem. Finally, we investigate the solution stability of the non-convex regular polynomial ve…
Hidden convex optimization is such a class of nonconvex optimization problems that can be globally solved in polynomial time via equivalent convex programming reformulations. In this paper, we focus on checking local optimality in hidden convex optimization. We first introduce a class of hidden convex optimization problems by jointing the classical nonconvex trust-region subproblem (TRS) with convex optimization (CO), and then present a comprehensive study on local optimality conditions. In order to guarantee the existence of a necessary and sufficient condition for local optimality, we need m…
Geodesic convexity generalizes the notion of (vector space) convexity to nonlinear metric spaces. But unlike convex optimization, geodesically convex (g-convex) optimization is much less developed. In this paper we contribute to the understanding of g-convex optimization by developing iteration complexity analysis for several first-order algorithms on Hadamard manifolds. Specifically, we prove upper bounds for the global complexity of deterministic and stochastic (sub)gradient methods for optimizing smooth and nonsmooth g-convex functions, both with and without strong g-convexity. Our analysis…
We study the stable behaviour of discrete dynamical systems where the map is convex and monotone with respect to the standard positive cone. The notion of tangential stability for fixed points and periodic points is introduced, which is weaker than Lyapunov stability. Among others we show that the set of tangentially stable fixed points is isomorphic to a convex inf-semilattice, and a criterion is given for the existence of a unique tangentially stable fixed point. We also show that periods of tangentially stable periodic points are orders of permutations on $n$ letters, where $n$ is the dimen…
In this paper, we introduce a new class of nonsmooth convex functions called SOS-convex semialgebraic functions extending the recently proposed notion of SOS-convex polynomials. This class of nonsmooth convex functions covers many common nonsmooth functions arising in the applications such as the Euclidean norm, the maximum eigenvalue function and the least squares functions with $\ell_1$-regularization or elastic net regularization used in statistics and compressed sensing. We show that, under commonly used strict feasibility conditions, the optimal value and an optimal solution of SOS-convex…
This work is concerned with linear inverse problems where a distributed parameter is known a priori to only take on values from a given discrete set. This property can be promoted in Tikhonov regularization with the aid of a suitable convex but nondifferentiable regularization term. This allows applying standard approaches to show well-posedness and convergence rates in Bregman distance. Using the specific properties of the regularization term, it can be shown that convergence (albeit without rates) actually holds pointwise. Furthermore, the resulting Tikhonov functional can be minimized effic…
We study conjugate and Lagrange dualities for composite optimization problems within the framework of abstract convexity. We provide conditions for zero duality gap in conjugate duality. For Lagrange duality, intersection property is applied to obtain zero duality gap. Connection between Lagrange dual and conjugate dual is also established. Examples related to convex and weakly convex functions are given.
We investigate Lagrangian duality for nonconvex optimization problems. To this aim we use the $Φ$-convexity theory and minimax theorem for $Φ$-convex functions. We provide conditions for zero duality gap and strong duality. Among the classes of functions, to which our duality results can be applied, are prox-bounded functions, DC functions, weakly convex functions and paraconvex functions.
This paper introduces an add-on, supervisory scheme, referred to as Action Governor (AG), for discrete-time linear systems to enforce exclusion-zone avoidance requirements. It does so by monitoring, and minimally modifying when necessary, the nominal control signal to a constraint-admissible one. The AG operates based on set-theoretic techniques and online optimization. This paper establishes its theoretical foundation, discusses its computational realization, and uses two simulation examples to illustrate its effectiveness.
In this paper, we study the convex quadratic optimization problem with indicator variables. For the bivariate case, we describe the convex hull of the epigraph in the original space of variables, and also give a conic quadratic extended formulation. Then, using the convex hull description for the bivariate case as a building block, we derive an extended SDP relaxation for the general case. This new formulation is stronger than other SDP relaxations proposed in the literature for the problem, including Shor's SDP relaxation, the optimal perspective relaxation as well as the optimal rank-one rel…
State-of-the-art methods in convex and non-convex optimization employ higher-order derivative information, either implicitly or explicitly. We explore the limitations of higher-order optimization and prove that even for convex optimization, a polynomial dependence on the approximation guarantee and higher-order smoothness parameters is necessary. As a special case, we show Nesterov's accelerated cubic regularization method to be nearly tight.
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We present a distributed solution to optimizing a convex function composed of several non-convex functions. Each non-convex function is privately stored with an agent while the agents communicate with neighbors to form a network. We show that coupled consensus and projected gradient descent algorithm proposed in [1] can optimize convex sum of non-convex functions under an additional assumption on gradient Lipschitzness. We further discuss the applications of this analysis in improving privacy in distributed optimization.
This note establishes a limiting formula for the conic Lagrangian dual of a convex infinite optimization problem, correcting the classical version of Karney [Math. Programming 27 (1983) 75-82] for convex semi-infinite programs. A reformulation of the convex infinite optimization problem with a single constraint leads to a limiting formula for the corresponding Lagrangian dual, called sup-dual, and also for the primal problem in the case when strong Slater condition holds, which also entails strong sup-duality.
We derive new prox-functions on the simplex from additive random utility models of discrete choice. They are convex conjugates of the corresponding surplus functions. In particular, we explicitly derive the convexity parameter of discrete choice prox-functions associated with generalized extreme value models, and specifically with generalized nested logit models. Incorporated into subgradient schemes, discrete choice prox-functions lead to natural probabilistic interpretations of the iteration steps. As illustration we discuss an economic application of discrete choice prox-functions in consum…
In this paper we propose several adaptive gradient methods for stochastic optimization. Unlike AdaGrad-type of methods, our algorithms are based on Armijo-type line search and they simultaneously adapt to the unknown Lipschitz constant of the gradient and variance of the stochastic approximation for the gradient. We consider an accelerated and non-accelerated gradient descent for convex problems and gradient descent for non-convex problems. In the experiments we demonstrate superiority of our methods to existing adaptive methods, e.g. AdaGrad and Adam.
