Let us consider two sequences of closed convex sets $\{A_n\}$ and $\{B_n\}$ converging with respect to the Attouch-Wets convergence to $A$ and $B$, respectively. Given a starting point $a_0$, we consider the sequences of points obtained by projecting on the "perturbed" sets, i.e., the sequences $\{a_n\}$ and $\{b_n\}$ defined inductively by $b_n=P_{B_n}(a_{n-1})$ and $a_n=P_{A_n}(b_n)$.
Suppose that $A\cap B$ (or a suitable substitute if $A \cap B=\emptyset$) is bounded, we prove that if the couple $(A,B)$ is (boundedly) regular then the couple $(A,B)$ is $d$-stable, i.e., for each $\{a_n\}$ and $\{b_n\}$ as above we have $\mathrm{dist}(a_n,A\cap B)\to 0$ and $\mathrm{dist}(b_n,A\cap B)\to 0$.
In this work we consider an iterative method for solving the quasi-convex feasibility problem. We firstly introduce the so-called star subgradient projection operator and present some useful properties. We subsequently obtain a convergence result of the considered method by using properties of the introduced nonlinear operator.
The convex restriction of the power flow feasible sets identifies the convex subset of power injections where the solution for power flow is guaranteed to exist and satisfy the operational constraints. In contrast to convex relaxations, the convex restriction provides a sufficient condition for power flow feasibility and is particularly useful for problems involving uncertainty in the power generation and demand. In this paper, we present a general framework of constructing convex restriction of an algebraic set defined by equality and inequality constraints and apply the framework to power fl…
The implicit convex feasibility problem attempts to find a point in the intersection of a finite family of convex sets, some of which are not explicitly determined but may vary. We develop simultaneous and sequential projection methods capable of handling such problems and demonstrate their applicability to image denoising in a specific medical imaging situation. By allowing the variable sets to undergo scaling, shifting and rotation, this work generalizes previous results wherein the implicit convex feasibility problem was used for cooperative wireless sensor network positioning where sets ar…
Finding a point in the intersection of a collection of closed convex sets, that is the convex feasibility problem, represents the main modeling strategy for many computational problems. In this paper we analyze new stochastic reformulations of the convex feasibility problem in order to facilitate the development of new algorithmic schemes. We also analyze the conditioning problem parameters using certain (linear) regularity assumptions on the individual convex sets. Then, we introduce a general random projection algorithmic framework, which extends to the random settings many existing projecti…
In this paper, we propose a catalog of iterative methods for solving the Split Feasibility Problem in the non-convex setting. We study four different optimization formulations of the problem, where each model has advantageous in different settings of the problem. For each model, we study relevant iterative algorithms, some of which are well-known in this area and some are new. All the studied methods, including the well-known CQ Algorithm, are proven to have global convergence guarantees in the non-convex setting under mild conditions on the problem's data.
We propose a generalization of the method of cyclic projections, which uses the lengths of projection steps carried out in the past to learn about the geometry of the problem and decides on this basis which projections to carry out in the future. We prove the convergence of this algorithm and illustrate its behavior in a first numerical study.
The 2-sets convex feasibility problem aims at finding a point in the intersection of two closed convex sets $A$ and $B$ in a normed space $X$. More generally, we can consider the problem of finding (if possible) two points in $A$ and $B$, respectively, which minimize the distance between the sets. In the present paper, we study some stability properties for the convex feasibility problem: we consider two sequences of sets, each of them converging, with respect to a suitable notion of set convergence, respectively, to $A$ and $B$. Under appropriate assumptions on the original problem, we ensure…
The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection o…
We propose finitely convergent methods for solving convex feasibility problems defined over a possibly infinite pool of constraints. Following other works in this area, we assume that the interior of the solution set is nonempty and that certain overrelaxation parameters form a divergent series. We combine our methods with a very general class of deterministic control sequences where, roughly speaking, we require that sooner or later we encounter a violated constraint if one exists. This requirement is satisfied, in particular, by the cyclic, repetitive and remotest set controls. Moreover, it …
In this work we study the method of Bregman projections for deterministic and stochastic convex feasibility problems with three types of control sequences for the selection of sets during the algorithmic procedure: greedy, random, and adaptive random. We analyze in depth the case of affine feasibility problems showing that the iterates generated by the proposed methods converge Q-linearly and providing also explicit global and local rates of convergence. This work generalizes from one hand recent developments in randomized methods for the solution of linear systems based on orthogonal projecti…
In this paper, we consider the well-known following shape optimization problem: $$λ_2(Ω^*)=\min_{\stackrel{|Ω|=V_0} {Ω\textrm{ convex}}} λ_2(Ω),$$ where $λ_2(\Om)$ denotes the second eigenvalue of the Laplace operator with homogeneous Dirichlet boundary conditions in $\Om\subset\R^2$, and $|\Om|$ is the area of $\Om$. We prove, under some technical assumptions, that any optimal shape $Ω^*$ is $\mathcal{C}^{1,\frac{1}{2}}$ and is not $\C^{1,α}$ for any $α>\frac{1}{2}$. We also derive from our strategy some more general regularity results, in the framework of partially overdetermined boundary va…
This work concerns the local convergence theory of Newton and quasi-Newton methods for convex-composite optimization: minimize f(x):=h(c(x)), where h is an infinite-valued proper convex function and c is C^2-smooth. We focus on the case where h is infinite-valued piecewise linear-quadratic and convex. Such problems include nonlinear programming, mini-max optimization, estimation of nonlinear dynamics with non-Gaussian noise as well as many modern approaches to large-scale data analysis and machine learning. Our approach embeds the optimality conditions for convex-composite optimization problem…
For the existence of strong duality in convex optimization regularity conditions play an indisputable role. We mainly deal in this paper with regularity conditions formulated by means of different generalizations of the notion of interior of a set. The primal-dual pair we investigate is a general one expressed in the language of a perturbation function and by employing its Fenchel-Moreau conjugate. After providing an overview on the generalized interior-point conditions that exist in the literature we introduce several new ones formulated by means of the quasi interior and quasi-relative inter…
We further investigate the uniform regularity property of collections of sets via primal and dual characterizing constants. These constants play an important role in determining convergence rates of projection algorithms for solving feasibility problems.
In this paper we consider the solvability of a non-convex regular polynomial vector optimization problem on a nonempty closed set. We introduce regularity conditions for the polynomial vector optimization problem and study properties and characterizations of the regularity conditions. Under the regularity conditions, we study nonemptiness and boundedness of the solution sets of the problem. As a consequence, we establish two Frank-Wolfe type theorems for the non-convex polynomial vector optimization problem. Finally, we investigate the solution stability of the non-convex regular polynomial ve…
We analyze sequences generated by interior point methods (IPMs) in convex and nonconvex settings. We prove that moving the primal feasibility at the same rate as the barrier parameter $μ$ ensures the Lagrange multiplier sequence remains bounded, provided the limit point of the primal sequence has a Lagrange multiplier. This result does not require constraint qualifications. We also guarantee the IPM finds a solution satisfying strict complementarity if one exists. On the other hand, if the primal feasibility is reduced too slowly, then the algorithm converges to a point of minimal complementar…
We study variational inequalities which are governed by a strongly monotone and Lipschitz continuous operator $F$ over a closed and convex set $S$. We assume that $S=C\cap A^{-1}(Q)$ is the nonempty solution set of a (multiple-set) split convex feasibility problem, where $C$ and $Q$ are both closed and convex subsets of two real Hilbert spaces $\mathcal H_1$ and $\mathcal H_2$, respectively, and the operator $A$ acting between them is linear. We consider a modification of the gradient projection method the main idea of which is to replace at each step the metric projection onto $S$ by another …
The circumcentered-reflection method (CRM) has been applied for solving convex feasibility problems. CRM iterates by computing a circumcenter upon a composition of reflections with respect to convex sets. Since reflections are based on exact projections, their computation might be costly. In this regard, we introduce the circumcentered approximate-reflection method (CARM), whose reflections rely on outer-approximate projections. The appeal of CARM is that, in rather general situations, the approximate projections we employ are available under low computational cost. We derive convergence of CA…
The ancient concept of circumcenter has recently given birth to the Circumcentered-Reflection method (CRM). CRM was first employed to solve best approximation problems involving affine subspaces. In this setting, it was shown to outperform the most prestigious projection based schemes, namely, the Douglas-Rachford method (DRM) and the method of alternating projections (MAP). We now prove convergence of CRM for finding a point in the intersection of a finite number of closed convex sets. This striking result is derived under a suitable product space reformulation in which a point in the interse…
In this paper we give general recommendations for successful application of the Douglas-Rachford reflection method to convex and non-convex real matrix-completion problems. These guidelines are demonstrated by various illustrative examples.
In this paper, we propose a new algorithm combining the Douglas-Rachford (DR) algorithm and the Frank-Wolfe algorithm, also known as the conditional gradient (CondG) method, for solving the classic convex feasibility problem. Within the algorithm, which will be named {\it Approximate Douglas-Rachford (ApDR) algorithm}, the CondG method is used as a subroutine to compute feasible inexact projections on the sets under consideration, and the ApDR iteration is defined based on the DR iteration. The ApDR algorithm generates two sequences, the main sequence, based on the DR iteration, and its corres…
Convex relaxations of AC optimal power flow (AC-OPF) problems have attracted significant interest as in several instances they provably yield the global optimum to the original non-convex problem. If, however, the relaxation is inexact, the obtained solution is not AC-feasible. The quality of the obtained solution is essential for several practical applications of AC-OPF, but detailed analyses are lacking in existing literature. This paper aims to cover this gap. We provide an in-depth investigation of the solution characteristics when convex relaxations are inexact, we assess the most promisi…
In this paper, we consider convex feasibility problems where the underlying sets are loosely coupled, and we propose several algorithms to solve such problems in a distributed manner. These algorithms are obtained by applying proximal splitting methods to convex minimization reformulations of convex feasibility problems. We also put forth distributed convergence tests which enable us to establish feasibility or infeasibility of the problem distributedly, and we provide convergence rate results. Under the assumption that the problem is feasible and boundedly linearly regular, these convergence …
In this work we focus on the convex feasibility problem (CFP) in Hilbert space. A specific method in this area that has gained a lot of interest in recent years is the Douglas-Rachford (DR) algorithm. This algorithm was originally introduced in 1956 for solving stationary and non-stationary heat equations. Then in 1979, Lions and Mercier adjusted and extended the algorithm with the aim of solving CFPs and even more general problems, such as finding zeros of the sum of two maximally monotone operators. Many developments which implement various concepts concerning this algorithm have occurred du…
This paper improves the algorithms based on supporting halfspaces and quadratic programming for convex set intersection problems in our earlier paper in several directions. First, we give conditions so that much smaller quadratic programs (QPs) and approximate projections arising from partially solving the QPs are sufficient for multiple-term superlinear convergence for nonsmooth problems. Second, we identify additional regularity, which we call the second order supporting hyperplane property (SOSH), that gives multiple-term quadratic convergence. Third, we show that these fast convergence res…
The split feasibility problem is to find an element in the intersection of a closed set $C$ and the linear preimage of another closed set $D$, assuming the projections onto $C$ and $D$ are easy to compute. This class of problems arises naturally in many contemporary applications such as compressed sensing. While the sets $C$ and $D$ are typically assumed to be convex in the literature, in this paper, we allow both sets to be possibly nonconvex. We observe that, in this setting, the split feasibility problem can be formulated as an optimization problem with a difference-of-convex objective so t…
We study the usage of regularity properties of collections of sets in convergence analysis of alternating projection methods for solving feasibility problems. Several equivalent characterizations of these properties are provided. Two settings of inexact alternating projections are considered and the corresponding convergence estimates are established and discussed.
This paper investigates regularity properties of two non-negative sparsity sets: non-negative sparse vectors, and low-rank positive semi-definite matrices. Novel formulae for their Mordukhovich normal cones are given and used to formulate sufficient conditions for non-convex notions of regularity to hold. Our results provide a useful tool for justifying the application of projection methods to certain rank constrained feasibility problems.
Algorithms for convex feasibility find or approximate a point in the intersection of given closed convex sets. Typically there are only finitely many convex sets, but the case of infinitely many convex sets also has some applications. In this context, a \emph{string} is a finite sequence of points each of which is obtained from the previous point by considering one of the convex sets. In a \emph{string-averaging} algorithm, an iterative step from a first point to a second point computes a number of strings, all starting with the first point, and then calculates a (weighted) average of those st…
We focus here on the analysis of the regularity or singularity of solutions $\Om_{0}$ to shape optimization problems among convex planar sets, namely: $$ J(\Om_{0})=\min\{J(\Om),\ \Om\ \textrm{convex},\ Ω\in\mathcal S_{ad}\}, $$ where $\mathcal S_{ad}$ is a set of 2-dimensional admissible shapes and $J:\mathcal{S}_{ad}\rightarrow\R$ is a shape functional. Our main goal is to obtain qualitative properties of these optimal shapes by using first and second order optimality conditions, including the infinite dimensional Lagrange multiplier due to the convexity constraint. We prove two types of res…
In this paper, we study different kinds of normal properties for infinite system of arbitrarily many convex sets in a Banach space and provide the dual characterization for the normal property in terms of the extended Jamenson property for arbitrarily many weak*-closed convex cones in the dual space. Then, we use the normal property and the extended Jamenson property to study CHIP, strong CHIP and linear regularity for the infinite case of arbitrarily many convex sets and establish equivalent relationship among these properties. In particular, we extend main results in [3] on normal property, …
The classical convex feasibility problem in a finite dimensional Euclidean space is studied in the present paper. We are interested in two cases. First, we assume to know how to compute an exact project onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method can be used for computing efficiently an inexact projection on it. Second, we assume that both sets involved are compact such that the CondG method can be used for computing efficiently inexact projections on them. We combine alternating projection method with CondG method to design a ne…
We analyze a simple randomized subgradient method for approximating solutions to stochastic systems of convex functional constraints, the only input to the algorithm being the size of minibatches. By introducing a new notion of what is meant for a point to approximately solve the constraints, determining bounds on the expected number of iterations reduces to determining a hitting time for a compound Bernoulli process, elementary probability. Besides bounding the expected number of iterations quite generally, we easily establish concentration inequalities on the number of iterations, and more i…
We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence of fundamental algorithms.
Motivated by nonconvex, inconsistent feasibility problems in imaging, the relaxed alternating averaged reflections algorithm, or relaxed Douglas-Rachford algorithm (DR$λ$), was first proposed over a decade ago. Convergence results for this algorithm are limited either to convex feasibility or consistent nonconvex feasibility with strong assumptions on the regularity of the underlying sets. Using an analytical framework depending only on metric subregularity and pointwise almost averagedness, we analyze the convergence behavior of DR$λ$ for feasibility problems that are both nonconvex and incon…
We revisit the so-called sampling and discarding approach used to quantify the probability of constraint violation of a solution to convex scenario programs when some of the original samples are allowed to be discarded. Motivated by two scenario programs that possess analytic solutions and the fact that the existing bound for scenario programs with discarded constraints is not tight, we analyze a removal scheme that consists of a cascade of optimization problems, where at each step we remove a superset of the active constraints. By relying on results from compression learning theory, we show t…
In this paper, we show that the subadditive dual of a feasible conic mixed-integer program (MIP) is a strong dual whenever it is feasible. Moreover, we show that this dual feasibility condition is equivalent to feasibility of the conic dual of the continuous relaxation of the conic MIP. In addition, we prove that all known conditions and other 'natural' conditions for strong duality, such as strict mixed-integer feasibility, boundedness of the feasible set or essentially strict feasibility imply that the subadditive dual is feasible. As an intermediate result, we extend the so-called 'finitene…
We propose first order algorithms for convex optimization problems where the feasible set is described by a large number of convex inequalities that is to be explored by subgradient projections. The first algorithm is an adaptation of a subgradient algorithm, and has convergence rate $1/\sqrt{k}$. The second algorithm has convergence rate 1/k when (1) one has linear metric inequality in the feasible set, (2) the objective function is strongly convex, differentiable and has Lipschitz gradient, and (3) it is easy to optimize the objective function on the intersection of two halfspaces. This seco…
The distributed non-smooth resource allocation problem over multi-agent networks is studied in this paper, where each agent is subject to globally coupled network resource constraints and local feasibility constraints described in terms of general convex sets. To solve such a problem, two classes of novel distributed continuous-time algorithms via differential inclusions and projection operators are proposed. Moreover, the convergence of the algorithms is analyzed by the Lyapunov functional theory and nonsmooth analysis. We illustrate that the first algorithm can globally converge to the exact…
In this paper, we establish sublinear and linear convergence of fixed point iterations generated by averaged operators in a Hilbert space. Our results are achieved under a bounded Hölder regularity assumption which generalizes the well-known notion of bounded linear regularity. As an application of our results, we provide a convergence rate analysis for Krasnoselskii-Mann iterations, the cyclic projection algorithm, and the Douglas-Rachford feasibility algorithm along with some variants. In the important case in which the underlying sets are convex sets described by convex polynomials in a fin…
In this paper we present a new algorithmic realization of a projection-based scheme for general convex constrained optimization problem. The general idea is to transform the original optimization problem to a sequence of feasibility problems by iteratively constraining the objective function from above until the feasibility problem is inconsistent. For each of the feasibility problems one may apply any of the existing projection methods for solving it. In particular, the scheme allows the use of subgradient projections and does not require exact projections onto the constraints sets as in exis…
This paper presents a convex sufficient condition for solving a system of nonlinear equations under parametric changes and proposes a sequential convex optimization method for solving robust optimization problems with nonlinear equality constraints. By bounding the nonlinearity with concave envelopes and using Brouwer's fixed point theorem, the sufficient condition is expressed in terms of closed-form convex inequality constraints. We extend the result to provide a convex sufficient condition for feasibility under bounded uncertainty. Using these conditions, a non-convex optimization problem c…
Dalian University of Technology（大连理工大学）；Peng Cheng Laboratory（鹏城实验室）；The University of Hong Kong（香港大学）；University of Victoria（维多利亚大学）；Southern University of Science and Technology（南方科技大学）
This paper firstly proposes a convex bilevel optimization paradigm to formulate and optimize popular learning and vision problems in real-world scenarios. Different from conventional approaches, which directly design their iteration schemes based on given problem formulation, we introduce a task-oriented energy as our latent constraint which integrates richer task information. By explicitly re-characterizing the feasibility, we establish an efficient and flexible algorithmic framework to tackle convex models with both shrunken solution space and powerful auxiliary (based on domain knowledge an…
The Douglas-Rachford method, a projection algorithm designed to solve continuous optimization problems, forms the basis of a useful heuristic for solving combinatorial optimization problems. In order to successfully use the method, it is necessary to formulate the problem at hand as a feasibility problem with constraint sets having efficiently computable nearest points. In this self-contained tutorial, we develop the convergence theory of projection algorithms within the framework of fixed point iterations, explain how to devise useful feasibility problem formulations, and demonstrate the appl…
