This paper was presented at the Workshop on Algorithmic and Quantitative Aspects of Real Algebraic Geometry in Mathematics and Computer Science, held at DIMACS, Rutgers University, March 12-16, 2001
发表信息：
Algorithmic and quantitative real algebraic geometry, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 60, pp. 83--99, AMS, 2003. ISBN: 0-8218-2863-0.
We compare algorithms for global optimization of polynomial functions in many variables. It is demonstrated that existing algebraic methods (Gröbner bases, resultants, homotopy methods) are dramatically outperformed by a relaxation technique, due to N.Z. Shor and the first author, which involves sums of squares and semidefinite programming. This opens up the possibility of using semidefinite programming relaxations arising from the Positivstellensatz for a wide range of computational problems in real algebraic geometry.
This paper was presented at the Workshop on Algorithmic and Quantitative Aspects of Real Algebraic Geometry in Mathematics and Computer Science, held at DIMACS, Rutgers University, March 12-16, 2001.
This paper concerns a method for finding the minimum of a polynomial on a semialgebraic set, i.e., a set in $\re^m$ defined by finitely many polynomial equations and inequalities, using the Karush-Kuhn-Tucker (KKT) system and sum of squares (SOS) relaxations. This generalizes results in the recent paper \cite{njw_grad}, which considers minimizing polynomials on algebraic sets, i.e., sets in $\re^m$ defined by finitely many polynomial equations. Most of the theorems and conclusions in \cite{njw_grad} generalize to semialgebraic sets, even in the case where the semialgebraic set is not compact. …
We complete the complexity classification by degree of minimizing a polynomial over the integer points in a polyhedron in $\mathbb{R}^2$. Previous work shows that optimizing a quadratic polynomial over the integer points in a polyhedral region in $\mathbb{R}^2$ can be done in polynomial time, while optimizing a quartic polynomial in the same type of region is NP-hard. We close the gap by showing that this problem can be solved in polynomial time for cubic polynomials. Furthermore, we show that the problem of minimizing a homogeneous polynomial of any fixed degree over the integer points in a b…
This paper discusses how to find the global minimum of functions that are summations of small polynomials (``small'' means involving a small number of variables). Some sparse sum of squares (SOS) techniques are proposed. We compare their computational complexity and lower bounds with prior SOS relaxations. Under certain conditions, we also discuss how to extract the global minimizers from these sparse relaxations. The proposed methods are especially useful in solving sparse polynomial system and nonlinear least squares problems. Numerical experiments are presented, which show that the proposed…
We present a generic approach that allows us to develop a fully polynomial-time approximation scheme (FTPAS) for minimizing nonlinear functions over the integer points in a rational polyhedron in fixed dimension. The approach combines the subdivision strategy of Papadimitriou and Yannakakis (2000) with ideas similar to those commonly used to derive real algebraic certificates of positivity for polynomials. Our general approach is widely applicable. We apply it, for instance, to the Motzkin polynomial and to indefinite quadratic forms $x^T Q x$ in a fixed number of variables, where $Q$ has at m…
Recently classes of conic and discrete conic functions were introduced. In this paper we use the term convic instead conic. The class of convic functions properly includes the classes of convex functions, strictly quasiconvex functions and the class of quasiconvex polynomials. On the other hand, the class of convic functions is properly included in the class of quasiconvex functions. The discrete convic function is a discrete analogue of the convic function. Recently the lower bound $3^{n-1}\log (2 ρ-1)$ for the number of calls to the comparison oracle needed to find the minimum of the discret…
This paper considers the optimization problem of minimizing a rational function. We reformulate this problem as polynomial optimization by the technique of homogenization. These two problems are shown to be equivalent under some generic conditions. The exact Jacobian SDP relaxation method proposed by Nie is used to solve the resulting polynomial optimization. We also prove that the assumption of nonsingularity in Nie's method can be weakened as the finiteness of singularities. Some numerical examples are given to illustrate the efficiency of our method.
Two-view triangulation is a problem of minimizing a quadratic polynomial under an equality constraint. We derive a polynomial that encodes the local minimizers of this problem using the theory of Lagrange multipliers. This offers a simpler derivation of the critical points that are given in Hartley-Sturm [6].
This paper discusses the frequency function of multiple-valued Dirichlet minimizing functions in the special case when the domain and range are both two dimensional. It shows that the frequency function must be of value k/2 for some nonnegative integer k. Futhermore, by looking at the blowing-up functions, we characterize the local behavior of the original Dirichlet minimizing function.
The space $J^k$ of $k$-jets of a real function of one real variable $x$ admits the structure of Carnot group type. As such, $J^k$ admits a submetry (\sR submersion) onto the Euclidean plane. Horizontal lifts of Euclidean lines (which are the left-translates of horizontal one-parameter subgroups) are thus globally minimizing geodesics on $J^k$. All $J^k$-geodesics, minimizing or not, are constructed from degree $k$ polynomials in $x$ according to Anzaldo-Meneses and Monroy-Peréz, reviewed here. The constant polynomials correspond to the horizontal lifts of lines. Which other polynomials yield g…
This paper is concerned with minimizing a sum of rational functions over a compact set of high-dimension. Our approach relies on the second Lasserre's hierarchy (also known as the upper bounds hierarchy) formulated on the pushforward measure in order to work in a space of smaller dimension. We show that in the general case the minimum can be approximated as closely as desired from above with a hierarchy of semidefinite programs problems or, in the particular case of a single fraction, with a hierarchy of generalized eigenvalue problems. We numerically illustrate the potential of using the push…
In this paper we develop algorithms to solve generalized weighted Fermat-Torricelli problems with positive and negative weights and multifacility location problems involving distances generated by Minkowski gauges. We also introduce a new model of clustering based on squared distances to convex sets. Using the Nesterov smoothing technique and an algorithm for minimizing differences of convex functions called the DCA introduced by Tao and An, we develop effective algorithms for solving these problems.
We show that absolutely minimizing functions relative to a convex Hamiltonian $H:\mathbb{R}^n \to \mathbb{R}$ are uniquely determined by their boundary values under minimal assumptions on $H.$ Along the way, we extend the known equivalences between comparison with cones, convexity criteria, and absolutely minimizing properties, to this generality. These results perfect a long development in the uniqueness/existence theory of the archetypal problem of the calculus of variations in $L^\infty.$
We consider the optimization problem of minimizing a polynomial f(x) subject to polynomial constraints h(x)=0, g(x)>=0. Lasserre's hierarchy is a sequence of sum of squares relaxations for finding the global minimum. Let K be the feasible set. We prove the following results: i) If the real variety V_R(h) is finite, then Lasserre's hierarchy has finite convergence, no matter the complex variety V_C(h) is finite or not. This solves an open question in Laurent's survey. ii) If K and V_R(h) have the same vanishing ideal, then the finite convergence of Lasserre's hierarchy is independent of the cho…
Interpolation-based trust-region methods are an important class of algorithms for Derivative-Free Optimization which rely on locally approximating an objective function by quadratic polynomial interpolation models, frequently built from less points than there are basis components. Often, in practical applications, the contribution of the problem variables to the objective function is such that many pairwise correlations between variables are negligible, implying, in the smooth case, a sparse structure in the Hessian matrix. To be able to exploit Hessian sparsity, existing optimization approach…
This paper extends the algorithm schemes proposed in \cite{Nesterov2007a} and \cite{Nesterov2007b} to the minimization of the sum of a composite objective function and a convex function. Two proximal point-type schemes are provided and their global convergence is investigated. The worst case complexity bound is also estimated under certain Lipschitz conditions and nondegeneratedness. The algorithm is then accelerated to get a faster convergence rate for the strongly convex case.
The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In this paper, we present a new convergence result on the the DRA applied to the problem of minimizing a convex function subject to a linear constraint. Indeed, a normal solution may be found even when the …
In view of the minimization of a function which is the sum of a differentiable function $f$ and a convex function $g$ we introduce descent methods which can be viewed as produced by inexact auxiliary problem principleor inexact variable metric forward-backward algorithm. Assuming that the global objective function satisfies the Kurdyka-Lojasiewicz inequalitywe prove the convergence of the proposed algorithm weakening assumptions found in previous works.
Several optimization schemes have been known for convex optimization problems. However, numerical algorithms for solving nonconvex optimization problems are still underdeveloped. A progress to go beyond convexity was made by considering the class of functions representable as differences of convex functions. In this paper, we introduce a generalized proximal point algorithm to minimize the difference of a nonconvex function and a convex function. We also study convergence results of this algorithm under the main assumption that the objective function satisfies the Kurdyka - Łojasiewicz propert…
A bilevel hierarchical clustering model is commonly used in designing optimal multicast networks. In this paper, we consider two different formulations of the bilevel hierarchical clustering problem, a discrete optimization problem which can be shown to be NP-hard. Our approach is to reformulate the problem as a continuous optimization problem by making some relaxations on the discreteness conditions. Then Nesterov's smoothing technique and a numerical algorithm for minimizing differences of convex functions called the DCA are applied to cope with the nonsmoothness and nonconvexity of the prob…
We consider the problem of minimizing a polynomial function over the integer lattice. Though impossible in general, we use a known sufficient condition for the existence of continuous minimizers to guarantee the existence of integer minimizers as well. In case this condition holds, we use sos programming to compute the radius of a p-norm ball which contains all integer minimizers. We prove that this radius is smaller than the radius known from the literature. Furthermore, we derive a new class of underestimators of the polynomial function. Using a Stellensatz from real algebraic geometry and a…
We consider global efficiency of algorithms for minimizing a sum of a convex function and a composition of a Lipschitz convex function with a smooth map. The basic algorithm we rely on is the prox-linear method, which in each iteration solves a regularized subproblem formed by linearizing the smooth map. When the subproblems are solved exactly, the method has efficiency $\mathcal{O}(\varepsilon^{-2})$, akin to gradient descent for smooth minimization. We show that when the subproblems can only be solved by first-order methods, a simple combination of smoothing, the prox-linear method, and a fa…
This letter studies the problem of minimizing increasing set functions, or equivalently, maximizing decreasing set functions, over the base of a matroid. This setting has received great interest, since it generalizes several applied problems including actuator and sensor placement problems in control theory, multi-robot task allocation problems, video summarization, and many others. We study two greedy heuristics, namely, the forward and the reverse greedy algorithms. We provide two novel performance guarantees for the approximate solutions obtained by these heuristics depending on both the su…
We discuss non-Euclidean deterministic and stochastic algorithms for optimization problems with strongly and uniformly convex objectives. We provide accuracy bounds for the performance of these algorithms and design methods which are adaptive with respect to the parameters of strong or uniform convexity of the objective: in the case when the total number of iterations $N$ is fixed, their accuracy coincides, up to a logarithmic in $N$ factor with the accuracy of optimal algorithms.
In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The proposed iteration, which will be called Proximal Subgradient Splitting Method, extends the classical subgradient iteration for important classes of problems, exploiting the additive structure of the objective function. The weak convergence of the generated sequence was established using different stepsizes and under suitable assumptions. Moreover, we analyze t…
We study a bi-objective optimization problem, which for a given positive real number $n$ aims to find a vector $X = \{x_0,\cdots,x_{k-1}\} \in \mathbb{R}^{k}_{\ge 0}$ such that $\sum_{i=0}^{k-1} x_i = n$, minimizing the maximum of $k$ functions of objective type one, $\max_{i=0}^{k-1} f_i(x_i)$, and the sum of $k$ functions of objective type two, $\sum_{i=0}^{k-1} g_i(x_i)$. This problem arises in the optimization of applications for performance and energy on high performance computing platforms. We first propose an algorithm solving the problem for the case where all the functions of objectiv…
We consider the problem of minimizing a sum of clipped convex functions; applications include clipped empirical risk minimization and clipped control. While the problem of minimizing the sum of clipped convex functions is NP-hard, we present some heuristics for approximately solving instances of these problems. These heuristics can be used to find good, if not global, solutions and appear to work well in practice. We also describe an alternative formulation, based on the perspective transformation, which makes the problem amenable to mixed-integer convex programming and yields computationally …
In this article we provide an experimental algorithm that in many cases gives us an upper bound of the global infimum of a real polynomial on $\R^{n}$. It is very well known that to find the global infimum of a real polynomial on $\R^{n}$, often reduces to solve a hierarchy of positive semidefinite programs, called moment relaxations. The algorithm that we present involves to solve a series of positive semidefinite programs whose feasible set is included in the feasible set of a moment relaxation. Our additional constraint try to provoke a flatness condition, like used by Curto and Fialkow, fo…
In this work we propose a distributed randomized block coordinate descent method for minimizing a convex function with a huge number of variables/coordinates. We analyze its complexity under the assumption that the smooth part of the objective function is partially block separable, and show that the degree of separability directly influences the complexity. This extends the results in [Richtarik, Takac: Parallel coordinate descent methods for big data optimization] to a distributed environment. We first show that partially block separable functions admit an expected separable overapproximation…
Proximal Newton-type methods for minimizing composite functions
Institute for Computational and Mathematical Engineering, Stanford University（斯坦福大学计算与数学工程研究所）；Department of Management Science and Engineering, Stanford University（斯坦福大学管理科学与工程系）
We provide investment advice for an individual who wishes to minimize her lifetime poverty, with a penalty for bankruptcy or ruin. We measure poverty via a non-negative, non-increasing function of (running) wealth. Thus, the lower wealth falls and the longer wealth stays low, the greater the penalty. This paper generalizes the problems of minimizing the probability of lifetime ruin and minimizing expected lifetime occupation, with the poverty function serving as a bridge between the two. To illustrate our model, we compute the optimal investment strategies for a specific poverty function and t…
We show that Dykstra's splitting for projecting onto the intersection of convex sets can be extended to minimize the sum of convex functions and a regularizing quadratic. We give conditions for which convergence to the primal minimizer holds so that more than one convex function can be minimized at a time, the convex functions are not necessarily sampled in a cyclic manner, and the SHQP strategy for problems involving the intersection of more than one convex set can be applied. When the sum does not involve the regularizing quadratic, we discuss an approximate proximal point method combined wi…
In the simplest case, we obtain a general solution to a problem of minimizing an integral of a nondecreasing right continuous stochastic process from zero to some nonnegative random variable tau, under the constraints that for some nonnegative random variable T, tau is between zero and T a.s. and the expected value of tau is some alpha. The nondecreasing process and T are allowed to be dependent. In fact a more general setup involving sigma-finite measures, rather than just probability measures is considered and some consequences for families of stochastic processes are given as special cases.…
We settle the computational complexity of fundamental questions related to multicriteria integer linear programs, when the dimensions of the strategy space and of the outcome space are considered fixed constants. In particular we construct: 1. polynomial-time algorithms to exactly determine the number of Pareto optima and Pareto strategies; 2. a polynomial-space polynomial-delay prescribed-order enumeration algorithm for arbitrary projections of the Pareto set; 3. an algorithm to minimize the distance of a Pareto optimum from a prescribed comparison point with respect to arbitrary polyhedral n…
The Frank Wolfe algorithm (FW) is a popular projection-free alternative for solving large-scale constrained optimization problems. However, the FW algorithm suffers from a sublinear convergence rate when minimizing a smooth convex function over a compact convex set. Thus, exploring techniques that yield a faster convergence rate becomes crucial. A classic approach to obtain faster rates is to combine previous iterates to obtain the next iterate. In this work, we extend this approach to the FW setting and show that the optimal way to combine the past iterates is using a set of orthogonal Jacobi…
We present a new method for minimizing the sum of a differentiable convex function and an $\ell_1$-norm regularizer. The main features of the new method include: $(i)$ an evolving set of indices corresponding to variables that are predicted to be nonzero at a solution (i.e., the support); $(ii)$ a reduced-space subproblem defined in terms of the predicted support; $(iii)$ conditions that determine how accurately each subproblem must be solved, which allow for Newton, Newton-CG, and coordinate-descent techniques to be employed; $(iv)$ a computationally practical condition that determines when t…
Given a separation oracle $\mathsf{SO}$ for a convex function $f$ defined on $\mathbb{R}^n$ that has an integral minimizer inside a box with radius $R$, we show how to find an exact minimizer of $f$ using at most (a) $O(n (n \log \log (n)/\log (n) + \log(R)))$ calls to $\mathsf{SO}$ and $\mathsf{poly}(n, \log(R))$ arithmetic operations, or (b) $O(n \log(nR))$ calls to $\mathsf{SO}$ and $\exp(O(n)) \cdot \mathsf{poly}(\log(R))$ arithmetic operations. When the set of minimizers of $f$ has integral extreme points, our algorithm outputs an integral minimizer of $f$. This improves upon the previous…
We show that for the problem of minimizing (or maximizing) the ratio of two supermodular functions, no bounded approximation ratio can be achieved via polynomial number of queries, if the two supermodular functions are both monotone non-decreasing or non-increasing.
In this paper, we provide near-optimal accelerated first-order methods for minimizing a broad class of smooth nonconvex functions that are strictly unimodal on all lines through a minimizer. This function class, which we call the class of smooth quasar-convex functions, is parameterized by a constant $γ\in (0,1]$, where $γ= 1$ encompasses the classes of smooth convex and star-convex functions, and smaller values of $γ$ indicate that the function can be "more nonconvex." We develop a variant of accelerated gradient descent that computes an $ε$-approximate minimizer of a smooth $γ$-quasar-convex…
