We develop and analyze a set of new sequential simulation-optimization algorithms for large-scale multi-dimensional discrete optimization via simulation problems with a convexity structure. The "large-scale" notion refers to that the decision variable has a large number of values to choose from on each dimension. The proposed algorithms are targeted to identify a solution that is close to the optimal solution given any precision level with any given probability. To achieve this target, utilizing the convexity structure, our algorithm design does not need to scan all the choices of the decision variable, but instead sequentially draws a subset of choices of the decision variable and uses them to "localize" potentially near-optimal solutions to an adaptively shrinking region.
To show the power of the localization operation, we first consider one-dimensional large-scale problems. We propose the shrinking uniform sampling algorithm, which is proved to achieve the target with an optimal expected simulation cost under an asymptotic criterion. For multi-dimensional problems, we combine the idea of localization with subgradient information and propose a framework to design stochastic cutting-plane methods and the dimension reduction algorithm, whose expected simulation cost have a low dependence on the scale and the dimension of the problems. The proposed algorithms do not require prior information about the Lipschitz constant of the objective function and the simulation costs are upper bounded by a value that is independent of the Lipschitz constant. Finally, we propose an adaptive algorithm to deal with the unknown noise variance case under the assumption that the randomness of the system is Gaussian. We implement the proposed algorithms on both synthetic and queueing simulation optimization problems, and demonstrate better performances compared to benchmark methods.
This manuscript is a preliminary pre-print version of a journal submission by the authors, revisiting the problem of range measurement based localization of a signal source or a sensor. The major geometric difficulty of the problem comes from the non-convex structure of optimization tasks associated with range measurements, noting that the set of source locations corresponding to a certain distance measurement by a fixed point sensor is non-convex both in two and three dimensions. Differently from various recent approaches to this localization problem, all starting with a non-convex geometric …
We address the sensor network localization problem given noisy range measurements between pairs of nodes. We approach the non-convex maximum-likelihood formulation via a known simple convex relaxation. We exploit its favorable optimization properties to the full to obtain an approach that: is completely distributed, has a simple implementation at each node, and capitalizes on an optimal gradient method to attain fast convergence. We offer a parallel but also an asynchronous flavor, both with theoretical convergence guarantees and iteration complexity analysis. Experimental results establish le…
For continuous functions, midpoint convexity characterizes convex functions. By considering discrete versions of midpoint convexity, several types of discrete convexities of functions, including integral convexity, L$^\natural$-convexity and global/local discrete midpoint convexity, have been studied. We propose a new type of discrete midpoint convexity that lies between L$^\natural$-convexity and integral convexity and is independent of global/local discrete midpoint convexity. The new convexity, named DDM-convexity, has nice properties satisfied by L$^\natural$-convexity and global/local dis…
We propose an efficient solution to peer-to-peer localization in a wireless sensor network which works in two stages. At the first stage the optimization problem is relaxed into a convex problem, given in the form recently proposed by Soares, Xavier, and Gomes. The convex problem is efficiently solved in a distributed way by an ADMM approach, which provides a significant improvement in speed with respect to the original solution. In the second stage, a soft transition to the original, non-convex, non relaxed formulation is applied in such a way to force the solution towards a local minimum. Th…
We give a characterization of the controllability for discrete-time linear systems with convex output constraints. It extends all previously known characterizations in the literature, as well as our previous results on controllability of discrete-time linear systems with conic output constraints.
This work considers the problem of locating a single source from noisy range measurements to a set of nodes in a wireless sensor network. We propose two new techniques that we designate as Source Localization with Nuclear Norm (SLNN) and Source Localization with l1-norm (SL-l1), which extend to arbitrary real dimensions, including 3D, our prior work on 2D source localization formulated in the complex plane. Broadly, our approach is based on formulating a Maximum-Likelihood (ML) estimation problem for the source position, and then using convex relaxation techniques to obtain a semidefinite prog…
The present article is an exposition of a theory of discrete convex functions on certain graph structures, developed by the author in recent years. This theory is a spin-off of discrete convex analysis by Murota, and is motivated by combinatorial dualities in multiflow problems and the complexity classification of facility location problems on graphs. We outline the theory and algorithmic applications in combinatorial optimization problems.
The network localization problem with convex and non-convex distance constraints may be modeled as a nonlinear optimization problem. The existing localization techniques are mainly based on convex optimization. In those techniques, the non-convex distance constraints are either ignored or relaxed into convex constraints for using the convex optimization methods like SDP, least square approximation, etc.. We propose a method to solve the nonlinear non-convex network localization problem with noisy distance measurements without any modification of constraints in the general model. We use the non…
Distributed optimization has many applications, in communication networks, sensor networks, signal processing, machine learning, and artificial intelligence. Methods for distributed convex optimization are widely investigated, while those for non-convex objectives are not well understood. One of the first non-convex distributed optimization frameworks over an arbitrary interaction graph was proposed by Di Lorenzo and Scutari [IEEE Trans. on Signal and Information Processing over Network, 2 (2016), pp. 120-136], which iteratively applies a combination of local optimization with convex approxima…
We propose to solve large instances of the non-convex optimization problems reformulated with canonical duality theory. To this aim we propose an interior point potential reduction algorithm based on the solution of the primal-dual total complementarity (Lagrange) function. We establish the global convergence result for the algorithm under mild assumptions and demonstrate the method on instances of the Sensor Network Localization problem. Our numerical results are promising and show the possibility of devising efficient interior points methods for non-convex duality.
We study a specific convex maximization problem in n-dimensional space. The conjectured solution is proved to be a vertex of the polyhedral feasible region, but only a partial proof of local maximality is known. Integer sequences with interesting patterns arise in the analysis, owing to the number theoretic origin of the problem.
Differential stability of convex discrete optimal control problems in Banach spaces is studied in this paper. By using some recent results of An and Yen [Appl. Anal. 94, 108--128 (2015)] on differential stability of parametric convex optimization problems under inclusion constraints, we obtain an upper estimate for the subdifferential of the optimal value function of a parametric convex discrete optimal control problem, where the objective function may be nondifferentiable. If the objective function is differentiable, the obtained upper estimate becomes an equality. It is shown that the singul…
In this paper we introduce the Ameso optimization problem, a special class of discrete optimization problems. We establish its basic properties and investigate the relation between Ameso optimization and the convex optimization. Further, we design an algorithm to solve a multi-dimensional Ameso problem by solving a sequence of one-dimensional Ameso problems. Finally, we demonstrate how the knapsack problem can be solved using the Ameso optimization framework.
In this paper, we study the problem of finding the Euclidean distance to a convex cone generated by a set of discrete points in $\mathbb{R}^n_+$. In particular, we are interested in problems where the discrete points are the set of feasible solutions of some binary linear programming constraints. This problem has applications in manufacturing, machine learning, clustering, pattern recognition, and statistics. Our problem is a high-dimensional constrained optimization problem. We propose a Frank-Wolfe based algorithm to solve this non-convex optimization problem with a convex-noncompact feasibl…
``Localization'' has proven to be a valuable tool in the Statistical Learning literature as it allows sharp risk bounds in terms of the problem geometry. Localized bounds seem to be much less exploited in the Stochastic Optimization literature. In addition, there is an obvious interest in both communities in obtaining risk bounds that require weak moment assumptions or ``heavier-tails''. In this work we use a localization toolbox to derive risk bounds in two specific applications. The first is in portfolio risk minimization with conditional value-at-risk constraints. We consider a setting wher…
Discrete convex functions are used in many areas, including operations research, discrete-event systems, game theory, and economics. The objective of this paper is to investigate basic operations such as direct sum, splitting, and aggregation that are related to network induction of discrete convex functions as well as discrete convex sets. Various kinds of discrete convex functions in discrete convex analysis are considered such as integrally convex functions, L-convex functions, M-convex functions, multimodular functions, and discrete midpoint convex functions.
Discrete convex functions are used in many areas, including operations research, discrete-event systems, game theory, and economics. The objective of this paper is to offer a survey on fundamental operations for various kinds of discrete convex functions in discrete convex analysis such as integrally convex functions, L-convex functions, M-convex functions, and multimodular functions.
In this paper, we bring the celebrated max-weight features (myopic and discrete actions) to mainstream convex optimization. Myopic actions are important in control because decisions need to be made in an online manner and without knowledge of future events, and discrete actions because many systems have a finite (so non-convex) number of control decisions. For example, whether to transmit a packet or not in communication networks. Our results show that these two features can be encompassed in the subgradient method for the Lagrange dual problem by the use of stochastic and $ε$-subgradients. On…
We propose new sequential simulation-optimization algorithms for general convex optimization via simulation problems with high-dimensional discrete decision space. The performance of each choice of discrete decision variables is evaluated via stochastic simulation replications. If an upper bound on the overall level of uncertainties is known, our proposed simulation-optimization algorithms utilize the discrete convex structure and are guaranteed with high probability to find a solution that is close to the best within any given user-specified precision level. The proposed algorithms work for a…
Hybrid localization in GNSS-challenged environments using measured ranges and angles is becoming increasingly popular, in particular with the advent of multimodal communication systems. Here, we address the hybrid network localization problem using ranges and bearings to jointly determine the positions of a number of agents through a single maximum-likelihood (ML) optimization problem that seamlessly fuses all the available pairwise range and angle measurements. We propose a tight convex surrogate to the ML estimator, we examine practical measures for the accuracy of the relaxation, and we com…
In this paper, we consider a time-varying optimization approach to the problem of tracking a moving target using noisy time-of-arrival (TOA) measurements. Specifically, we formulate the problem as that of sequential TOA-based source localization and apply online gradient descent (OGD) to it to generate the position estimates of the target. To analyze the tracking performance of OGD, we first revisit the classic least-squares formulation of the (static) TOA-based source localization problem and elucidate its estimation and geometric properties. In particular, under standard assumptions on the T…
This work is concerned with linear inverse problems where a distributed parameter is known a priori to only take on values from a given discrete set. This property can be promoted in Tikhonov regularization with the aid of a suitable convex but nondifferentiable regularization term. This allows applying standard approaches to show well-posedness and convergence rates in Bregman distance. Using the specific properties of the regularization term, it can be shown that convergence (albeit without rates) actually holds pointwise. Furthermore, the resulting Tikhonov functional can be minimized effic…
This paper is devoted to distributed continuous-time and discrete-time optimization problems with nonuniform convex constraint sets and nonuniform stepsizes for general differentiable convex objective functions. The communication graphs are not required to be strongly connected at any time, the gradients of the local objective functions are not required to be bounded when their independent variables tend to infinity, and the constraint sets are not required to be bounded. For continuous-time multi-agent systems, a distributed continuous algorithm is first introduced where the stepsizes and the…
The paper deals with the optimal control problem described by second order evolution differential inclusions; to this end first we use an auxiliary problem with second order discrete and discrete-approximate inclusions. Then applying infimal convolution concept of convex functions, step by step we construct the dual problems for discrete, discrete-approximate and differential inclusions and prove duality results. It seems that the Euler-Lagrange type inclusions are "duality relations" for both primary and dual problems and that the dual problem for discrete-approximate problem make a bridge be…
In this paper, we consider a novel and robust maximum likelihood approach to localizing radiation sources with unknown statistics of the source signal strength. The result utilizes the smallest number of sensors required theoretically to localize the source. It is shown, that should the source lie in the open convex hull of the sensors, precisely $N+1$ are required in $\mathbb{R}^N, ~N \in \{1,\cdots,3\}$. It is further shown that the region of interest, the open convex hull of the sensors, is entirely devoid of false stationary points. An augmented gradient ascent algorithm with random projec…
We present a novel convex relaxation and a corresponding inference algorithm for the non-binary discrete tomography problem, that is, reconstructing discrete-valued images from few linear measurements. In contrast to state of the art approaches that split the problem into a continuous reconstruction problem for the linear measurement constraints and a discrete labeling problem to enforce discrete-valued reconstructions, we propose a joint formulation that addresses both problems simultaneously, resulting in a tighter convex relaxation. For this purpose a constrained graphical model is set up a…
This paper considers projection and convolution operations for integrally convex functions, which constitute a fundamental function class in discrete convex analysis. It is shown that the class of integrally convex functions is stable under projection, and this is also the case with the subclasses of integrally convex functions satisfying local or global discrete midpoint convexity. As is known in the literature, the convolution of two integrally convex functions may possibly fail to be integrally convex. We show that the convolution of an integrally convex function with a separable convex fun…
Integrally convex functions constitute a fundamental function class in discrete convex analysis. This paper shows that an integer-valued integrally convex function admits an integral subgradient and that the integral biconjugate of an integer-valued integrally convex function coincides with itself. The proof is based on the Fourier-Motzkin elimination. The latter result provides a unified proof of integral biconjugacy for various classes of integer-valued discrete convex functions, including L-convex, M-convex, L$_{2}$-convex, M$_{2}$-convex, BS-convex, and UJ-convex functions as well as multi…
We study the stable behaviour of discrete dynamical systems where the map is convex and monotone with respect to the standard positive cone. The notion of tangential stability for fixed points and periodic points is introduced, which is weaker than Lyapunov stability. Among others we show that the set of tangentially stable fixed points is isomorphic to a convex inf-semilattice, and a criterion is given for the existence of a unique tangentially stable fixed point. We also show that periods of tangentially stable periodic points are orders of permutations on $n$ letters, where $n$ is the dimen…
Geometric Conditions for Lossless Convexification in Linear Optimal Control with Discrete-Valued Inputs: Real-Time Implementation for Spacecraft Rendezvous
We derive new prox-functions on the simplex from additive random utility models of discrete choice. They are convex conjugates of the corresponding surplus functions. In particular, we explicitly derive the convexity parameter of discrete choice prox-functions associated with generalized extreme value models, and specifically with generalized nested logit models. Incorporated into subgradient schemes, discrete choice prox-functions lead to natural probabilistic interpretations of the iteration steps. As illustration we discuss an economic application of discrete choice prox-functions in consum…
In this paper we develop and analyse convex searches for Zames--Falb multipliers. We present two different approaches: Infinite Impulse Response (IIR) and Finite Impulse Response (FIR) multipliers. The set of FIR multipliers is complete in that any IIR multipliers can be phase-substituted by an arbitrarily large order FIR multiplier. We show that searches in discrete-time for FIR multipliers are effective even for large orders. As expected, the numerical results provide the best $\ell_{2}$-stability results in the literature for slope-restricted nonlinearities. Finally, we demonstrate that the…
In this paper, we study a class of finite-time control problems for discrete-time positive linear systems with time-varying state parameters. Although several interesting control problems appearing in population biology, economics, and network epidemiology can be described as the class of finite-time control problems, an efficient solution to the control problem has not been yet found in the literature. In this paper, we propose an optimization framework for solving the class of finite-time control problems via convex optimization. We illustrate the effectiveness of the proposed method by nume…
This paper introduces an add-on, supervisory scheme, referred to as Action Governor (AG), for discrete-time linear systems to enforce exclusion-zone avoidance requirements. It does so by monitoring, and minimally modifying when necessary, the nominal control signal to a constraint-admissible one. The AG operates based on set-theoretic techniques and online optimization. This paper establishes its theoretical foundation, discusses its computational realization, and uses two simulation examples to illustrate its effectiveness.
We develop an algorithmic theory of convex optimization over discrete sets. Using a combination of algebraic and geometric tools we are able to provide polynomial time algorithms for solving broad classes of convex combinatorial optimization problems and convex integer programming problems in variable dimension. We discuss some of the many applications of this theory including to quadratic programming, matroids, bin packing and cutting-stock problems, vector partitioning and clustering, multiway transportation problems, and privacy and confidential statistical data disclosure. Highlights of ou…
We investigate an enriched-categorical approach to a field of discrete mathematics. The main result is a duality theorem between a class of enriched categories (called $\overline{\mathbb{Z}}$- or $\overline{\mathbb{R}}$-categories) and that of what we call ($\overline{\mathbb{Z}}$- or $\overline{\mathbb{R}}$-) extended L-convex sets. We introduce extended L-convex sets as variants of certain discrete structures called L-convex sets and L-convex polyhedra, studied in the field of discrete convex analysis. We also introduce homomorphisms between extended L-convex sets. The theorem claims that th…
Compressive sensing (CS) exploits sparsity to recover sparse or compressible signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity is also used to enhance interpretability in machine learning and statistics applications: While the ambient dimension is vast in modern data analysis problems, the relevant information therein typically resides in a much lower dimensional space. However, many solutions proposed nowadays do not leverage the true underlying structure. Recent results in CS extend the simple sparsity idea to more sophisticated {\em structured} sparsity models,…
