In this paper we consider a problem, called convex projection, of projecting a convex set onto a subspace. We will show that to a convex projection one can assign a particular multi-objective convex optimization problem, such that the solution to that problem also solves the convex projection (and vice versa), which is analogous to the result in the polyhedral convex case considered in arXiv:1507.00228. In practice, however, one can only compute approximate solutions in the (bounded or self-bounded) convex case, which solve the problem up to a given error tolerance. We will show that for approximate solutions a similar connection can be proven, but the tolerance level needs to be adjusted. That is, an approximate solution of the convex projection solves the multi-objective problem only with an increased error. Similarly, an approximate solution of the multi-objective problem solves the convex projection with an increased error. In both cases the tolerance is increased proportionally to a multiplier. These multipliers are deduced and shown to be sharp. These results allow to compute approximate solutions to a convex projection problem by computing approximate solutions to the corresponding multi-objective convex optimization problem, for which algorithms exist in the bounded case.
For completeness, we will also investigate the potential generalization of the following result to the convex case. In arXiv:1507.00228, it has been shown for the polyhedral case, how to construct a polyhedral projection associated to any given vector linear program and how to relate their solutions. This in turn yields an equivalence between polyhedral projection, multi-objective linear programming and vector linear programming. We will show that only some parts of this result can be generalized to the convex case, and discuss the limitations.
This note conducts a comparative study of some approximating properties of the metric projection, generalized projection, and generalized metric projection in uniformly convex and uniformly smooth Banach spaces. We prove that the inverse images of the metric projections are closed and convex cones, but they are not necessarily convex. In contrast, inverse images of the generalized projection are closed and convex cones. Furthermore, the inverse images of the generalized metric projection are neither a convex set nor a cone. We also prove that the distance from a point to its projection at a co…
A remarkable example of a nonempty closed convex set in the Euclidean plane for which the directional derivative of the metric projection mapping fails to exist was constructed by A. Shapiro. In this paper, we revisit and modify that construction to obtain a convex set with smooth boundary which possesses the same property.
In this work we consider an iterative method for solving the quasi-convex feasibility problem. We firstly introduce the so-called star subgradient projection operator and present some useful properties. We subsequently obtain a convergence result of the considered method by using properties of the introduced nonlinear operator.
We propose a generalization of the method of cyclic projections, which uses the lengths of projection steps carried out in the past to learn about the geometry of the problem and decides on this basis which projections to carry out in the future. We prove the convergence of this algorithm and illustrate its behavior in a first numerical study.
Finding a point in the intersection of a collection of closed convex sets, that is the convex feasibility problem, represents the main modeling strategy for many computational problems. In this paper we analyze new stochastic reformulations of the convex feasibility problem in order to facilitate the development of new algorithmic schemes. We also analyze the conditioning problem parameters using certain (linear) regularity assumptions on the individual convex sets. Then, we introduce a general random projection algorithmic framework, which extends to the random settings many existing projecti…
In this paper, we studied the equilibrium problem where the bi-function may be quasiconvex with respect to the second variable and the feasible set is the intersection of a finite number of convex sets. We propose a projection-algorithm, where the projection can be computed independently onto each component set. The convergence of the algorithm is investigated and numerical examples for a variational inequality problem involving affine fractional operator are provided to demonstrate the behavior of the algorithm.
This paper deals with the convex feasibility problem, where the feasible set is given as the intersection of a (possibly infinite) number of closed convex sets. We assume that each set is specified algebraically as a convex inequality, where the associated convex function is general (possibly non-differentiable). For finding a point satisfying all the convex inequalities we design and analyze random projection algorithms using special subgradient iterations and extrapolated stepsizes. Moreover, the iterate updates are performed based on parallel random observations of several constraint compon…
This paper is devoted to the general problem of projection onto a polyhedral convex cone generated by a finite set of generators.This problem is reformulated into projection onto the polytope obtained by simple truncation of the original cone. Then it can be solved with just two closely related projections onto the same bounded polytope. This approach's computational performance is conditioned by the crucial tool's efficiency for solving the fundamental problem of finding the least norm element in a convex hull of a given finite set of points. In our numerical experiments, we used for this pur…
We show that the minimum distance projection in the L1-norm from an interior point onto the boundary of a convex set is achieved by a single, unidimensional projection. Application of this characterization when the convex set is a polyhedron leads to either an elementary minmax problem or a set of easily solved linear programs, depending upon whether the polyhedron is given as the intersection of a set of half spaces or as the convex hull of a set of extreme points. The outcome is an easier and more straightforward derivation of the special case results given in a recent paper by Briec.
In this paper, we propose a new inexact version of the projected subgradient method to solve nondifferentiable constrained convex optimization problems. The method combine $ε$-subgradient method with a procedure to obtain a feasible inexact projection onto the constraint set. Asymptotic convergence results and iteration-complexity bounds for the sequence generated by the method employing the well known exogenous stepsizes, Polyak's stepsizes, and dynamic stepsizes are established.
Two new optimization techniques based on projections onto convex space (POCS) framework for solving convex and some non-convex optimization problems are presented. The dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. If the cost function is a convex function in R^N the corresponding set is a convex set in R^(N+1). The iterative optimization approach starts with an arbitrary initial estimate in R^(N+1) and an orthogonal projection is performed onto one of the sets in a sequential manner at each step of the optimization problem. The …
This paper considers a conceptual version of a convex optimization algorithm whic is based on replacing a convex optimization problem with the root-finding problem for the approximate sub-differential mapping which is solved by repeated projection onto the epigraph of conjugate function. Whilst the projection problem is not exactly solvable in finite space-time it can be approximately solved up to arbitrary precision by simple iterative methods, which use linear support functions of the epigraph. It seems therefore useful to study computational characteristics of the idealized version of this …
The basic tool for solving problems in metric geometry and isotonic regression is the metric projection onto closed convex cones. Isotonicity of these projections with respect to a given order relation can facilitate finding the solutions of the above problems. In the recent note "A. B. Németh and S.Z. Németh: Isotonic regression and isotonic projection. Linear Algebra and its Applications, 494: 80-89 (2016)" this problem was studied for the coordinate-wise ordering. This study was the starting point for further investigations, such as the ones presented here. The order relation in the Euclide…
The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where the constraints are smooth and the number of constraints is significantly smaller than the dimension. The runtime of existing approaches to solving such problems is either cubic in the dimension or polynomial in the inverse of the target accuracy. Conversely, we propose a simple and efficient primal-dual approach, with a runtime that scales …
In this paper we present a new algorithmic realization of a projection-based scheme for general convex constrained optimization problem. The general idea is to transform the original optimization problem to a sequence of feasibility problems by iteratively constraining the objective function from above until the feasibility problem is inconsistent. For each of the feasibility problems one may apply any of the existing projection methods for solving it. In particular, the scheme allows the use of subgradient projections and does not require exact projections onto the constraints sets as in exis…
The task of projecting onto $\ell_p$ norm balls is ubiquitous in statistics and machine learning, yet the availability of actionable algorithms for doing so is largely limited to the special cases of $p = \left\{ 0, 1,2, \infty \right\}$. In this paper, we introduce novel, scalable methods for projecting onto the $\ell_p$ ball for general $p>0$. For $p \geq1 $, we solve the univariate Lagrangian dual via a dual Newton method. We then carefully design a bisection approach for $p<1$, presenting theoretical and empirical evidence of zero or a small duality gap in the non-convex case. The success …
The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection o…
In this work we study the method of Bregman projections for deterministic and stochastic convex feasibility problems with three types of control sequences for the selection of sets during the algorithmic procedure: greedy, random, and adaptive random. We analyze in depth the case of affine feasibility problems showing that the iterates generated by the proposed methods converge Q-linearly and providing also explicit global and local rates of convergence. This work generalizes from one hand recent developments in randomized methods for the solution of linear systems based on orthogonal projecti…
In this paper we present a new iterative projection method for finding the closest point in the intersection of convex sets to any arbitrary point in a Hilbert space. This method, termed AAMR for averaged alternating modified reflections, can be viewed as an adequate modification of the Douglas--Rachford method that yields a solution to the best approximation problem. Under a constraint qualification at the point of interest, we show strong convergence of the method. In fact, the so-called strong CHIP fully characterizes the convergence of the AAMR method for every point in the space. We repor…
Random projection techniques based on Johnson-Lindenstrauss lemma are used for randomly aggregating the constraints or variables of optimization problems while approximately preserving their optimal values, that leads to smaller-scale optimization problems. D'Ambrosio et al. have applied random projection to a quadratic optimization problem so as to decrease the number of decision variables. Although the problem size becomes smaller, the projected problem will also almost surely be non-convex if the original problem is non-convex, and hence will be hard to solve. In this paper, by focusing on …
In this paper, the problem of computing the projection, and therefore the minimum distance, from a point onto a Minkowski sum of general convex sets is studied. Our approach is based on the minimum norm duality theorem originally stated by Nirenberg and the Nesterov smoothing techniques. It is shown that projection points onto a Minkowski sum of sets can be represented as the sum of points on constituent sets so that, at these points, all of the sets share the same normal vector which is the negative of the dual solution. The proposed NESMINO algorithm improves the theoretical bound on number …
The Frank-Wolfe method solves smooth constrained convex optimization problems at a generic sublinear rate of $\mathcal{O}(1/T)$, and it (or its variants) enjoys accelerated convergence rates for two fundamental classes of constraints: polytopes and strongly-convex sets. Uniformly convex sets non-trivially subsume strongly convex sets and form a large variety of \textit{curved} convex sets commonly encountered in machine learning and signal processing. For instance, the $\ell_p$-balls are uniformly convex for all $p > 1$, but strongly convex for $p\in]1,2]$ only. We show that these sets systema…
This paper considers projection and convolution operations for integrally convex functions, which constitute a fundamental function class in discrete convex analysis. It is shown that the class of integrally convex functions is stable under projection, and this is also the case with the subclasses of integrally convex functions satisfying local or global discrete midpoint convexity. As is known in the literature, the convolution of two integrally convex functions may possibly fail to be integrally convex. We show that the convolution of an integrally convex function with a separable convex fun…
This article studies optional and predictable projections of integrands and convex-valued stochastic processes. The existence and uniqueness are shown under general conditions that are analogous to those for conditional expectations of integrands and random sets. In the convex case, duality correspondences between the projections and projections of epigraphs are given. These results are used to study projections of set-valued integrands. Consistently with the general theory of stochastic processes, projections are not constructed using reference measures on the optional and predictable sigma-a…
Fourier-Motzkin elimination is a projection algorithm for solving finite linear programs. We extend Fourier-Motzkin elimination to semi-infinite linear programs which are linear programs with finitely many variables and infinitely many constraints. Applying projection leads to new characterizations of important properties for primal-dual pairs of semi-infinite programs such as zero duality gap, feasibility, boundedness, and solvability. Extending the Fourier-Motzkin elimination procedure to semi-infinite linear programs yields a new classification of variables that is used to determine the exi…
We consider convex constrained optimization problems that also include a cardinality constraint. In general, optimization problems with cardinality constraints are difficult mathematical programs which are usually solved by global techniques from discrete optimization. We assume that the region defined by the convex constraints can be written as the intersection of a finite collection of convex sets, such that it is easy and inexpensive to project onto each one of them (e.g., boxes, hyper-planes, or half-spaces). Taking advantage of a recently developed continuous reformulation that relaxes th…
A spectrahedron is a set defined by a linear matrix inequality. A projection of a spectrahedron is often called a semidefinitely representable set. We show that the convex hull of a finite union of such projections is again a projection of a spectrahedron. This improves upon the result of Helton and Nie, who prove the same result in the case of bounded sets.
The article deals with operations defined on convex polyhedra or polyhedral convex functions. Given two convex polyhedra, operations like Minkowski sum, intersection and closed convex hull of the union are considered. Basic operations for one convex polyhedron are, for example, the polar, the conical hull and the image under affine transformation. The concept of a P-representation of a convex polyhedron is introduced. It is shown that many polyhedral calculus operations can be expressed explicitly in terms of P-representations. We point out that all the relevant computational effort for polyhe…
Convex feasibility problems require to find a point in the intersection of a finite family of convex sets. We propose to solve such problems by performing set-enlargements and applying a new kind of projection operators called valiant projectors. A valiant projector onto a convex set implements a special relaxation strategy, proposed by Goffin in 1971, that dictates the move toward the projection according to the distance from the set. Contrary to past realizations of this strategy, our valiant projection operator implements the strategy in a continuous fashion. We study properties of valiant …
This paper presents a piecewise convexification method for solving non-convex multi-objective optimization problems with box constraints. Based on the ideas of the $α$-based Branch and Bound (${\rm αBB}$) method of global optimization and the interval subdivision, a series of convex relaxation sub-multiobjective problems for this non-convex multi-objective optimization problem are firstly obtained, and these sub-problems constitute a piecewise convexification problem of the original problem on the whole box. We then construct the (approximate, weakly) efficient solution set of this piecewise c…
The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: i) Many well-known operator splitting methods, such as forward-backward splitting (FBS) and Douglas-Rachford splitting (DRS), actually define a fixed-point mapping; ii) The optimal solutions of the composite convex program and the solutions of a system of nonlinear equations derived from the fixed-point mapping are equivalent. Solving this kind of system of nonlinear equations enables us to develop second-order type methods. …
The circumcentered-reflection method (CRM) has been applied for solving convex feasibility problems. CRM iterates by computing a circumcenter upon a composition of reflections with respect to convex sets. Since reflections are based on exact projections, their computation might be costly. In this regard, we introduce the circumcentered approximate-reflection method (CARM), whose reflections rely on outer-approximate projections. The appeal of CARM is that, in rather general situations, the approximate projections we employ are available under low computational cost. We derive convergence of CA…
In this paper, we study the rate of convergence of the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets. We establish an explicit convergence rate estimate which relies on the maximum degree of the polynomials that generate the basic semi-algebraic convex sets and the dimension of the underlying space. We achieve our results by exploiting the algebraic structure of the basic semi-algebraic convex sets.
This paper introduces a new method of partitioning the solution space of a multi-objective optimisation problem for parallel processing, called Efficient Projection Partitioning. This method projects solutions down into a single dimension, greatly reducing the cost of partitioning the search space. We test EPP on a variety of randomly generated multi-objective combinatorial optimisation problems. The results are compared with the state of the art in parallel partitioning, and we show that in all scenarios tested, our new algorithm performs significantly better. Our proposed method allows the g…
Although Nesterov's accelerated gradient (NAG) methods have been studied from various perspectives, it remains unclear why the most popular forms of NAG must handle convex and strongly convex objective functions separately. Motivated by this inconsistency, we propose an NAG method that unifies the existing ones for the convex and strongly convex cases. We first design a Lagrangian function that continuously extends the first Bregman Lagrangian to the strongly convex setting. As a specific case of the Euler--Lagrange equation for this Lagrangian, we derive an ordinary differential equation (ODE…
In a general Hilbert framework, we consider continuous gradient-like dynamical systems for constrained multiobjective optimization involving non-smooth convex objective functions. Our approach is in the line of a previous work where was considered the case of convex di erentiable objective functions. Based on the Yosida regularization of the subdi erential operators involved in the system, we obtain the existence of strong global trajectories. We prove a descent property for each objective function, and the convergence of trajectories to weak Pareto minima. This approach provides a dynamical e…
The convex feasibility problem consists in finding a point in the intersection of a finite family of closed convex sets. When the intersection is empty, a best compromise is to search for a point that minimizes the sum of the squared distances to the sets. In 2001, de Pierro conjectured that the limit cycles generated by the $\varepsilon$-under-relaxed cyclic projection method converge when $\varepsilon\downarrow 0$ towards a least squares solution. While the conjecture has been confirmed under fairly general conditions, we show that it is false in general by constructing a system of three com…
Technion - Israel Institute of Technology（以色列理工学院）
Projection-free optimization algorithms, which are mostly based on the classical Frank-Wolfe method, have gained significant interest in the machine learning community in recent years due to their ability to handle convex constraints that are popular in many applications, but for which computing projections is often computationally impractical in high-dimensional settings, and hence prohibit the use of most standard projection-based methods. In particular, a significant research effort was put on projection-free methods for online learning. In this paper we revisit the Online Frank-Wolfe (OFW)…
The purpose of this paper is to study a non-convex fuzzy multi-objective quadratic programming problem, in which both the technological coefficients and resources are fuzzy with nonlinear membership function. A computational procedure to find a fuzzy efficient solution of this problem is developed. A numerical example is given to illustrate the procedure.
