We show that the minimum distance projection in the L1-norm from an interior point onto the boundary of a convex set is achieved by a single, unidimensional projection. Application of this characterization when the convex set is a polyhedron leads to either an elementary minmax problem or a set of easily solved linear programs, depending upon whether the polyhedron is given as the intersection of a set of half spaces or as the convex hull of a set of extreme points. The outcome is an easier and more straightforward derivation of the special case results given in a recent paper by Briec.
We study the projection onto the set of feasible inputs and the set of feasible solutions of a polynomial optimisation problem (POP). Our motivation is increasing the robustness of solvers for POP: Without a priori guarantees of feasibility of a particular instance, one should like to perform the projection onto the set of feasible inputs prior to running a solver. Without a certificate of optimality, one should like to project the output of the solver onto the set of feasible solutions subsequently. We study the computational complexity, formulations, and convexifications of the projections. …
It is well-known that the sequence of iterations of the composition of projections onto closed affine subspaces converges linearly to the projection onto the intersection of the affine subspaces when the sum of the corresponding linear subspaces is closed. Inspired by this, in this work, we systematically study the relation between the projection onto intersection of halfspaces and hyperplanes, and the composition of projections onto halfspaces and hyperplanes. In addition, as by-products, we provide the Karush-Kuhn-Tucker conditions for characterizing the optimal solution of convex optimizati…
In Hilbert space setting we prove local lipchitzness of projections onto parametric polyhedral sets represented as solutions to systems of inequalities and equations with parameters appearing both in left-hand-sides and right-hand-sides of the constraints. In deriving main results we assume that data are locally Lipschitz functions of parameter and the relaxed constant rank constraint qualification condition is satisfied.
This mini-paper presents a fast and simple algorithm to compute the projection onto the canonical simplex $\triangle^n$. Utilizing the Moreau's identity, we show that the problem is essentially a univariate minimization and the objective function is strictly convex and continuously differentiable. Moreover, it is shown that there are at most n candidates which can be computed explicitly, and the minimizer is the only one that falls into the correct interval.
This note conducts a comparative study of some approximating properties of the metric projection, generalized projection, and generalized metric projection in uniformly convex and uniformly smooth Banach spaces. We prove that the inverse images of the metric projections are closed and convex cones, but they are not necessarily convex. In contrast, inverse images of the generalized projection are closed and convex cones. Furthermore, the inverse images of the generalized metric projection are neither a convex set nor a cone. We also prove that the distance from a point to its projection at a co…
We address the problem of projecting a point onto a quadratic hypersurface, more specifically a central quadric. We show how this problem reduces to finding a given root of a scalar-valued nonlinear function. We completely characterize one of the optimal solutions of the projection as either the unique root of this nonlinear function on a given interval, or as a point that belongs to a finite set of computable solutions. We then leverage this projection and the recent advancements in splitting methods to compute the projection onto the intersection of a box and a quadratic hypersurface with al…
A wedge (i.e., a closed nonempty set in the Euclidean space stable under addition and multiplication with non-negative scalars) induces by a standard way a semi-order (a reflexive and transitive binary relation) in the space. The wedges admitting isotone metric projection with respect to the semi-order induced by them are characterized. The obtained result is used to show that the monotone wedge (called monotone cone in regression theory) admits isotone projection.
The basic tool for solving problems in metric geometry and isotonic regression is the metric projection onto closed convex cones. Isotonicity of these projections with respect to a given order relation can facilitate finding the solutions of the above problems. In the recent note "A. B. Németh and S.Z. Németh: Isotonic regression and isotonic projection. Linear Algebra and its Applications, 494: 80-89 (2016)" this problem was studied for the coordinate-wise ordering. This study was the starting point for further investigations, such as the ones presented here. The order relation in the Euclide…
The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where the constraints are smooth and the number of constraints is significantly smaller than the dimension. The runtime of existing approaches to solving such problems is either cubic in the dimension or polynomial in the inverse of the target accuracy. Conversely, we propose a simple and efficient primal-dual approach, with a runtime that scales …
In this paper, we provide a finitely terminated yet efficient approach to compute the Euclidean projection onto the ordered weighted $\ell_1$ (OWL1) norm ball. In particular, an efficient semismooth Newton method is proposed for solving the dual of a reformulation of the original projection problem. Global and local quadratic convergence results, as well as the finite termination property, of the algorithm are proved. Numerical comparisons with the two best-known methods demonstrate the efficiency of our method. In addition, we derive the generalized Jacobian of the studied projector which, we…
By using Moreau's decomposition theorem for projecting onto cones, the problem of projecting onto a simplicial cone is reduced to finding the unique solution of a nonsmooth system of equations. It is shown that Picard's method applied to the system of equations associated to the problem of projecting onto a simplicial cone generates a sequence that converges linearly to the solution of the system. Numerical experiments are presented making the comparison between Picard's and semi-smooth Newton's methods to solve the nonsmooth system associated with the problem of projecting a point onto a simp…
In this paper we consider a problem, called convex projection, of projecting a convex set onto a subspace. We will show that to a convex projection one can assign a particular multi-objective convex optimization problem, such that the solution to that problem also solves the convex projection (and vice versa), which is analogous to the result in the polyhedral convex case considered in arXiv:1507.00228. In practice, however, one can only compute approximate solutions in the (bounded or self-bounded) convex case, which solve the problem up to a given error tolerance. We will show that for appro…
This paper introduces the monotone extended second-order cone (MESOC), which is related to the monotone cone and the second-order cone. Some properties of the MESOC are presented and its dual cone is computed. Projecting onto the MESOC is reduced to the pool-adjacent-violators algorithm (PAVA) of isotonic regression. An application of MESOC to portfolio optimisation is provided. Some broad descriptions of possible MESOC-regression models are also outlined.
A remarkable example of a nonempty closed convex set in the Euclidean plane for which the directional derivative of the metric projection mapping fails to exist was constructed by A. Shapiro. In this paper, we revisit and modify that construction to obtain a convex set with smooth boundary which possesses the same property.
Two new optimization techniques based on projections onto convex space (POCS) framework for solving convex and some non-convex optimization problems are presented. The dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. If the cost function is a convex function in R^N the corresponding set is a convex set in R^(N+1). The iterative optimization approach starts with an arbitrary initial estimate in R^(N+1) and an orthogonal projection is performed onto one of the sets in a sequential manner at each step of the optimization problem. The …
In this paper, the problem of computing the projection, and therefore the minimum distance, from a point onto a Minkowski sum of general convex sets is studied. Our approach is based on the minimum norm duality theorem originally stated by Nirenberg and the Nesterov smoothing techniques. It is shown that projection points onto a Minkowski sum of sets can be represented as the sum of points on constituent sets so that, at these points, all of the sets share the same normal vector which is the negative of the dual solution. The proposed NESMINO algorithm improves the theoretical bound on number …
In this work we consider an iterative method for solving the quasi-convex feasibility problem. We firstly introduce the so-called star subgradient projection operator and present some useful properties. We subsequently obtain a convergence result of the considered method by using properties of the introduced nonlinear operator.
We propose a generalization of the method of cyclic projections, which uses the lengths of projection steps carried out in the past to learn about the geometry of the problem and decides on this basis which projections to carry out in the future. We prove the convergence of this algorithm and illustrate its behavior in a first numerical study.
Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the g…
Mixed norms that promote structured sparsity have numerous applications in signal processing and machine learning problems. In this work, we present a new algorithm, based on a Newton root search technique, for computing the projection onto the $\ell_{\infty,1}$ ball, which has found application in cognitive neuroscience and classification tasks. Numerical simulations show that our proposed method is between 8 and 10 times faster on average, and up to 20 times faster for very sparse solutions, than the previous state of the art. Tests on real functional magnetic resonance image data show that,…
Finding a point in the intersection of a collection of closed convex sets, that is the convex feasibility problem, represents the main modeling strategy for many computational problems. In this paper we analyze new stochastic reformulations of the convex feasibility problem in order to facilitate the development of new algorithmic schemes. We also analyze the conditioning problem parameters using certain (linear) regularity assumptions on the individual convex sets. Then, we introduce a general random projection algorithmic framework, which extends to the random settings many existing projecti…
In this paper, we studied the equilibrium problem where the bi-function may be quasiconvex with respect to the second variable and the feasible set is the intersection of a finite number of convex sets. We propose a projection-algorithm, where the projection can be computed independently onto each component set. The convergence of the algorithm is investigated and numerical examples for a variational inequality problem involving affine fractional operator are provided to demonstrate the behavior of the algorithm.
This paper deals with the convex feasibility problem, where the feasible set is given as the intersection of a (possibly infinite) number of closed convex sets. We assume that each set is specified algebraically as a convex inequality, where the associated convex function is general (possibly non-differentiable). For finding a point satisfying all the convex inequalities we design and analyze random projection algorithms using special subgradient iterations and extrapolated stepsizes. Moreover, the iterate updates are performed based on parallel random observations of several constraint compon…
This paper is devoted to the general problem of projection onto a polyhedral convex cone generated by a finite set of generators.This problem is reformulated into projection onto the polytope obtained by simple truncation of the original cone. Then it can be solved with just two closely related projections onto the same bounded polytope. This approach's computational performance is conditioned by the crucial tool's efficiency for solving the fundamental problem of finding the least norm element in a convex hull of a given finite set of points. In our numerical experiments, we used for this pur…
In this paper, we propose a new inexact version of the projected subgradient method to solve nondifferentiable constrained convex optimization problems. The method combine $ε$-subgradient method with a procedure to obtain a feasible inexact projection onto the constraint set. Asymptotic convergence results and iteration-complexity bounds for the sequence generated by the method employing the well known exogenous stepsizes, Polyak's stepsizes, and dynamic stepsizes are established.
This paper considers a conceptual version of a convex optimization algorithm whic is based on replacing a convex optimization problem with the root-finding problem for the approximate sub-differential mapping which is solved by repeated projection onto the epigraph of conjugate function. Whilst the projection problem is not exactly solvable in finite space-time it can be approximately solved up to arbitrary precision by simple iterative methods, which use linear support functions of the epigraph. It seems therefore useful to study computational characteristics of the idealized version of this …
The von Neumann-Halperin method of alternating projections converges strongly to the projection of a given point onto the intersection of finitely many closed affine subspaces. We propose acceleration schemes making use of two ideas: Firstly, each projection onto an affine subspace identifies a hyperplane of codimension 1 containing the intersection, and secondly, it is easy to project onto a finite intersection of such hyperplanes. We give conditions for which our accelerations converge strongly. Finally, we perform numerical experiments to show that these accelerations perform well for a mat…
In this paper we present a new algorithmic realization of a projection-based scheme for general convex constrained optimization problem. The general idea is to transform the original optimization problem to a sequence of feasibility problems by iteratively constraining the objective function from above until the feasibility problem is inconsistent. For each of the feasibility problems one may apply any of the existing projection methods for solving it. In particular, the scheme allows the use of subgradient projections and does not require exact projections onto the constraints sets as in exis…
The space of finite games can be decomposed into three orthogonal subspaces [5], which are the subspaces of pure potential games, nonstrategic games and pure harmonic games. The orthogonal projections onto these subspaces are represented as the Moore-Penrose inverses of the corresponding linear operators (i.e., matrices) [5]. Although the representation is compact and nice, no analytic method is given to calculate Moore- Penrose inverses of these linear operators. Hence using their results, one cannot verify whether a finite game belongs to one of these subspaces. In this paper, jumping over c…
We provide formulas for projectors onto a polyhedral set, i.e. the intersection of a finite number of halfspaces. To this aim we formulate the problem of finding the projection as a convex optimization problem and we solve explicitly sufficient and necessary optimality conditions. This approach has already been successfully applied in deriving formulas for projection onto the intersection of two halfspaces. We also discuss possible generalizations to Banach spaces.
The task of projecting onto $\ell_p$ norm balls is ubiquitous in statistics and machine learning, yet the availability of actionable algorithms for doing so is largely limited to the special cases of $p = \left\{ 0, 1,2, \infty \right\}$. In this paper, we introduce novel, scalable methods for projecting onto the $\ell_p$ ball for general $p>0$. For $p \geq1 $, we solve the univariate Lagrangian dual via a dual Newton method. We then carefully design a bisection approach for $p<1$, presenting theoretical and empirical evidence of zero or a small duality gap in the non-convex case. The success …
The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection o…
We provide an elementary proof of a simple, efficient algorithm for computing the Euclidean projection of a point onto the probability simplex. We also show an application in Laplacian K-modes clustering.
A new deconvolution algorithm based on orthogonal projections onto the epigraph set of a convex cost function is presented. In this algorithm, the dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. As the utilized cost function is a convex function in $R^N$, the corresponding epigraph set is also a convex set in $R^{N+1}$. The deconvolution algorithm starts with an arbitrary initial estimate in $R^{N+1}$. At each step of the iterative algorithm, first deconvolution projections are performed onto the epigraphs, later an orthogonal pro…
The projector onto the Minkowski sum of closed convex sets is generally not equal to the sum of individual projectors. In this work, we provide a complete answer to the question of characterizing the instances where such an equality holds. Our results unify and extend the case of linear subspaces and Zarantonello's results for projectors onto cones. A detailed analysis in the case of convex combinations is also carried out. We establish the partial sum property for projectors onto convex cones, and we also present various examples as well as a detailed analysis in the univariate case.
Both wavelet denoising and denosing methods using the concept of sparsity are based on soft-thresholding. In sparsity based denoising methods, it is assumed that the original signal is sparse in some transform domains such as the wavelet domain and the wavelet subsignals of the noisy signal are projected onto L1-balls to reduce noise. In this lecture note, it is shown that the size of the L1-ball or equivalently the soft threshold value can be determined using linear algebra. The key step is an orthogonal projection onto the epigraph set of the L1-norm cost function.
Sets of bilinear constraints are important in various machine learning models. Mathematically, they are hyperbolas in a product space. In this paper, we give a complete formula for projections onto sets of bilinear constraints or hyperbolas in a general Hilbert space.
This paper considers projection and convolution operations for integrally convex functions, which constitute a fundamental function class in discrete convex analysis. It is shown that the class of integrally convex functions is stable under projection, and this is also the case with the subclasses of integrally convex functions satisfying local or global discrete midpoint convexity. As is known in the literature, the convolution of two integrally convex functions may possibly fail to be integrally convex. We show that the convolution of an integrally convex function with a separable convex fun…
This article studies optional and predictable projections of integrands and convex-valued stochastic processes. The existence and uniqueness are shown under general conditions that are analogous to those for conditional expectations of integrands and random sets. In the convex case, duality correspondences between the projections and projections of epigraphs are given. These results are used to study projections of set-valued integrands. Consistently with the general theory of stochastic processes, projections are not constructed using reference measures on the optional and predictable sigma-a…
A spectrahedron is a set defined by a linear matrix inequality. A projection of a spectrahedron is often called a semidefinitely representable set. We show that the convex hull of a finite union of such projections is again a projection of a spectrahedron. This improves upon the result of Helton and Nie, who prove the same result in the case of bounded sets.
Two new optimization techniques based on projections onto convex space (POCS) framework for solving convex optimization problems are presented. The dimension of the minimization problem is lifted by one and sets corresponding to the cost function are defined. If the cost function is a convex function in R^N the corresponding set is also a convex set in R^{N+1}. The iterative optimization approach starts with an arbitrary initial estimate in R^{N+1} and an orthogonal projection is performed onto one of the sets in a sequential manner at each step of the optimization problem. The method provides…
In this paper, we study the rate of convergence of the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets. We establish an explicit convergence rate estimate which relies on the maximum degree of the polynomials that generate the basic semi-algebraic convex sets and the dimension of the underlying space. We achieve our results by exploiting the algebraic structure of the basic semi-algebraic convex sets.
Bilkent University（比尔肯特大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）
A new signal processing framework based on making orthogonal Projections onto the Epigraph Set of a Convex cost function (PESC) is developed. In this way it is possible to solve convex optimization problems using the well-known Projections onto Convex Set (POCS) approach. In this algorithm, the dimension of the minimization problem is lifted by one and a convex set corresponding to the epigraph of the cost function is defined. If the cost function is a convex function in $R^N$, the corresponding epigraph set is also a convex set in R^{N+1}. The PESC method provides globally optimal solutions f…
Technion - Israel Institute of Technology（以色列理工学院）
Projection-free optimization algorithms, which are mostly based on the classical Frank-Wolfe method, have gained significant interest in the machine learning community in recent years due to their ability to handle convex constraints that are popular in many applications, but for which computing projections is often computationally impractical in high-dimensional settings, and hence prohibit the use of most standard projection-based methods. In particular, a significant research effort was put on projection-free methods for online learning. In this paper we revisit the Online Frank-Wolfe (OFW)…
We consider the set of pairs of orthogonal vectors in Hilbert space, which is also called the cross because it is the union of the horizontal and vertical axes in the Euclidean plane when the underlying space is the real line. Crosses, which are nonconvex sets, play a significant role in various branches of nonsmooth analysis such as feasibility problems and optimization problems. In this work, we study crosses and show that in infinite-dimensional settings, they are never weakly (sequentially) closed. Nonetheless, crosses do turn out to be proximinal (i.e., they always admit projections) and …
