Projection algorithms are well known for their simplicity and flexibility in solving feasibility problems. They are particularly important in practice due to minimal requirements for software implementation and maintenance. In this work, we study linear convergence of several projection algorithms for systems of finitely many closed sets. The results complement contemporary research on the same topic.
We establish sufficient conditions for finite convergence of the alternating projections method for two non-intersecting and potentially nonconvex sets. Our results are based on a generalization of the concept of intrinsic transversality, which until now has been restricted to sets with nonempty intersection. In the special case of a polyhedron and closed half space, our sufficient conditions define the minimum distance between the two sets that is required for alternating projections to converge in a single iteration.
Convergence Rates for Stochastic Proximal and Projection Estimators
Universidad de Chile（智利大学）；Center for Mathematical Modeling (CMM), Universidad de Chile（智利大学数学建模中心）；Instituto de Ciencias de la Ingeniería, Universidad de O’Higgins（奥希金斯大学工程科学研究所）
Convergence Rates for Variational Inequality Projection Neural Networks with a State-Dependent Metric
King Fahd University of Petroleum & Minerals（阿卜杜勒阿齐兹国王石油矿物大学）；Interdisciplinary Research Center for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals（阿卜杜勒阿齐兹国王石油矿物大学智能交通与物流交叉研究中心）
We investigate the properties of the simultaneous projection method as applied to countably infinitely many closed and linear subspaces of a real Hilbert space. We establish the optimal error bound for linear convergence of this method, which we express in terms of the cosine of the Friedrichs angle computed in an infinite product space. In addition, we provide estimates and alternative expressions for the above-mentioned number. Furthermore, we relate this number to the dichotomy theorem and to super-polynomially fast convergence. We also discuss polynomial convergence of the simultaneous pro…
Convergence Rate Analysis of SOAP with Arbitrary Orthogonal Projection Matrices
Institute of Robotics and Automatic Information Systems, College of Artificial Intelligence, Nankai University（南开大学人工智能学院机器人与自动信息研究所）；National Key Lab of General AI, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能全国重点实验室）
This paper combines two ingredients in order to get a rather surprising result on one of the most studied, elegant and powerful tools for solving convex feasibility problems, the method of alternating projections (MAP). Going back to names such as Kaczmarz and von Neumann, MAP has the ability to track a pair of points realizing minimum distance between two given closed convex sets. Unfortunately, MAP may suffer from arbitrarily slow convergence, and sublinear rates are essentially only surpassed in the presence of some Lipschitzian error bound, which is our first ingredient. The second one is …
The Frank-Wolfe method solves smooth constrained convex optimization problems at a generic sublinear rate of $\mathcal{O}(1/T)$, and it (or its variants) enjoys accelerated convergence rates for two fundamental classes of constraints: polytopes and strongly-convex sets. Uniformly convex sets non-trivially subsume strongly convex sets and form a large variety of \textit{curved} convex sets commonly encountered in machine learning and signal processing. For instance, the $\ell_p$-balls are uniformly convex for all $p > 1$, but strongly convex for $p\in]1,2]$ only. We show that these sets systema…
We consider the popular and classical method of alternating projections for finding a point in the intersection of two closed sets. By situating the algorithm in a metric space, equipped only with well-behaved geodesics and angles (in the sense of Alexandrov), we are able to highlight the two key geometric ingredients in a standard intuitive analysis of local linear convergence. The first is a transversality-like condition on the intersection; the second is a convexity-like condition on one set: "uniform approximation by geodesics."
The main contributions of this paper are the proposition and the convergence analysis of a class of inertial projection-type algorithm for solving variational inequality problems in real Hilbert spaces where the underline operator is monotone and uniformly continuous. We carry out a unified analysis of the proposed method under very mild assumptions. In particular, weak convergence of the generated sequence is established and nonasymptotic $O(1/n)$ rate of convergence is established, where $n$ denotes the iteration counter. We also present some experimental results to illustrate the profits ga…
We study the convergence rate of the Circumcentered-Reflection Method (CRM) for solving the convex feasibility problem and compare it with the Method of Alternating Projections (MAP). Under an error bound assumption, we prove that both methods converge linearly, with asymptotic constants depending on a parameter of the error bound, and that the one derived for CRM is strictly better than the one for MAP. Next, we analyze two classes of fairly generic examples. In the first one, the angle between the convex sets approaches zero near the intersection, so that the MAP sequence converges sublinear…
We consider continuous-time dynamics for distributed optimization with set constraints in the paper. To handle the computational complexity of projection-based dynamics due to solving a general quadratic optimization subproblem with projection, we propose a distributed projection-free dynamics by employing the Frank-Wolfe method, also known as the conditional gradient algorithm. The process searches a feasible descent direction by solving an alternative linear optimization instead of a quadratic one. To make the approach applicable over weight-balanced digraphs, we design a dynamics for the co…
We propose a generalization of the method of cyclic projections, which uses the lengths of projection steps carried out in the past to learn about the geometry of the problem and decides on this basis which projections to carry out in the future. We prove the convergence of this algorithm and illustrate its behavior in a first numerical study.
In this work we study the method of Bregman projections for deterministic and stochastic convex feasibility problems with three types of control sequences for the selection of sets during the algorithmic procedure: greedy, random, and adaptive random. We analyze in depth the case of affine feasibility problems showing that the iterates generated by the proposed methods converge Q-linearly and providing also explicit global and local rates of convergence. This work generalizes from one hand recent developments in randomized methods for the solution of linear systems based on orthogonal projecti…
In this work, we study a novel class of projection-based algorithms for linearly constrained problems (LCPs) which have a lot of applications in statistics, optimization, and machine learning. Conventional primal gradient-based methods for LCPs call a projection after each (stochastic) gradient descent, resulting in that the required number of projections equals that of gradient descents (or total iterations). Motivated by the recent progress in distributed optimization, we propose the delayed projection technique that calls a projection once for a while, lowering the projection frequency and …
