In this work we study a constrained monotone inclusion involving the normal cone to a closed vector subspace and a priori information on primal solutions. We model this information by imposing that solutions belongs to the fixed point set of an averaged nonexpansive mapping. We characterize the solutions using an auxiliary inclusion that involves the partial inverse operator. Then, we propose the primal-dual partial inverse splitting and we prove its weak convergence to a solution of the inclusion, generalizing several methods in the literature. The efficiency of the proposed method is illustrated in two non-smooth convex optimization problems whose constraints have vector subspace structure. Finally, the proposed algorithm is applied to find a solution to a stochastic arc capacity expansion problem in transport networks.
In this paper we provide an algorithm for solving constrained composite primal-dual monotone inclusions, i.e., monotone inclusions in which a priori information on primal-dual solutions is represented via closed convex sets. The proposed algorithm incorporates a projection step onto the a priori information sets and generalizes the method proposed in [Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38, 667-681 (2013)]. Moreover, under the presence of strong monotonicity, we derive an accelerated scheme inspired on [Chambolle, A.; …
We propose a primal-dual splitting algorithm for solving monotone inclusions involving a mixture of sums, linear compositions, and parallel sums of set-valued and Lipschitzian operators. An important feature of the algorithm is that the Lipschitzian operators present in the formulation can be processed individually via explicit steps, while the set-valued operators are processed individually via their resolvents. In addition, the algorithm is highly parallel in that most of its steps can be executed simultaneously. This work brings together and notably extends various types of structured monot…
In this paper, we propose a randomized intertial block-coordinate primaldual fixed point algorithm to solve a wide array of monotone inclusion problems base on the modification of the heavy ball method of Nesterov. These methods rely on a sweep of blocks of variables which are activated at each iteration according to a random rule. To this end we formulate the inertial version of the Krasnosel'skii-Mann algorithm for approximating the set of fixed points of a quasinonexpansive operator, for which we also provide an exhaustive convergence analysis. As a by-product, we can obtain some intertial …
In this work, we study resolvent splitting algorithms for solving composite monotone inclusion problems. The objective of these general problems is finding a zero in the sum of maximally monotone operators composed with linear operators. Our main contribution is establishing the first primal-dual splitting algorithm for composite monotone inclusions with minimal lifting. Specifically, the proposed scheme reduces the dimension of the product space where the underlying fixed point operator is defined, in comparison to other algorithms, without requiring additional evaluations of the resolvent op…
We propose new primal-dual decomposition algorithms for solving systems of inclusions involving sums of linearly composed maximally monotone operators. The principal innovation in these algorithms is that they are block-iterative in the sense that, at each iteration, only a subset of the monotone operators needs to be processed, as opposed to all operators as in established methods. Deterministic strategies are used to select the blocks of operators activated at each iteration. In addition, we allow for operator processing "lags", permitting asynchronous implementation. The decomposition phase…
In this paper we present a novel derivation for an existing node-based algorithm for distributed optimisation termed the primal-dual method of multipliers (PDMM). In contrast to its initial derivation, in this work monotone operator theory is used to connect PDMM with other first-order methods such as Douglas-Rachford splitting and the alternating direction method of multipliers thus providing insight to the operation of the scheme. In particular, we show how PDMM combines a lifted dual form in conjunction with Peaceman-Rachford splitting to remove the need for collaboration between nodes per …
Spingarn's method of partial inverses has found many applications in nonlinear analysis and in optimization. We show that it can be employed to solve composite monotone inclusions in duality, thus opening a new range of applications for the partial inverse formalism. The versatility of the resulting primal-dual splitting algorithm is illustrated through applications to structured monotone inclusions and optimization.
We investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in {\em Y. Drori, S. Sabach, and M. Teboulle, A simple algorithm for a class of nonsmooth convex-concave saddle-point problems, 2015} and {\em I. Loris and C. Verhoeven, On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty, 2011} for saddl…
In this paper we study the relaxed primal-dual algorithm for solving composite monotone inclusions in real Hilbert spaces with critical preconditioners. Our approach is based in new results on the asymptotic behaviour of Krasnosel'ski\uı-Mann (KM) iterations defined in the range of monotone self-adjoint linear operators. These results generalize the convergence of classical KM iterations aiming at approximating fixed points. We prove that the relaxed primal-dual algorithm with critical preconditioners define KM iterations in the range of a particular monotone self-adjoint linear operator with …
Primal Methods for Constrained Variational Inequalities: Optimal Rates, Feasibility Trade-offs, and Lower Bounds
Tiangong University（天津工业大学）；University of Hertfordshire（赫特福德大学）；Sefako Makgatho Health Sciences University（塞法科·马卡托健康科学大学）；Zhejiang Normal University（浙江师范大学）；China Medical University（中国医科大学）；Academy of Romanian Scientists（罗马尼亚科学院）
We introduce and investigate the convergence properties of an inertial forward-backward-forward splitting algorithm for approaching the set of zeros of the sum of a maximally monotone operator and a single-valued monotone and Lipschitzian operator. By making use of the product space approach, we expand it to the solving of inclusion problems involving mixtures of linearly composed and parallel-sum type monotone operators. We obtain in this way an inertial forward-backward-forward primal-dual splitting algorithm having as main characteristic the fact that in the iterative scheme all operators a…
We propose a new primal-dual splitting method for solving composite inclusions involving Lipschitzian, and parallel-sum-type monotone operators. Our approach extends the framework in \cite{Siopt4} to a more general class of monotone inclusions in a nontrivial fashion. The main idea is to represent the solution set of both the primal and dual problems using their associated Kuhn-Tucker set, and then develop a projected method to successively approximate a feasible point of the Kuhn-Tucker set. We propose a splitting algorithm based on the resolvent of each maximally monotone operator to constru…
In this paper we provide a splitting method for finding a zero of the sum of a maximally monotone operator, a lipschitzian monotone operator, and a normal cone to a closed vectorial subspace of a real Hilbert space. The problem is characterized by a simpler monotone inclusion involving only two operators: the partial inverse of the maximally monotone operator with respect to the vectorial subspace and a suitable lipschitzian monotone operator. By applying the Tseng's method in this context we obtain a splitting algorithm that exploits the whole structure of the original problem and generalizes…
A general primal-dual splitting algorithm for solving systems of structured coupled monotone inclusions in Hilbert spaces is introduced and its asymptotic behavior is analyzed. Each inclusion in the primal system features compositions with linear operators, parallel sums, and Lipschitzian operators. All the operators involved in this structured model are used separately in the proposed algorithm, most steps of which can be executed in parallel. This provides a flexible solution method applicable to a variety of problems beyond the reach of the state-of-the-art. Several applications are discuss…
Monotone inclusions involving the sum of three maximally monotone operators or more have received much attention in recent years. In this paper, we propose three splitting algorithms for finding a zero of the sum of four monotone operators, which are two maximally monotone operators, one monotone Lipschitz operator, and one cocoercive operator. These three splitting algorithms are based on the forward-reflected-Douglas-Rachford splitting algorithm, backward-forward-reflected-backward splitting algorithm, and backward-reflected-forward-backward splitting algorithm, respectively. As applications…
We consider the problem of solving dual monotone inclusions involving sums of composite parallel-sum type operators. A feature of this work is to exploit explicitly the cocoercivity of some of the operators appearing in the model. Several splitting algorithms recently proposed in the literature are recovered as special cases.
We propose a novel dynamically weighted inertial forward-backward algorithm (DWIFOB) for solving structured monotone inclusion problems. The scheme exploits the globally convergent forward-backward algorithm with deviations in [26] as the basis and combines it with the extrapolation technique used in Anderson acceleration to improve local convergence. We also present a globally convergent primal-dual variant of DWIFOB and numerically compare its performance to the primal-dual method of Chambolle-Pock and a Tikhonov regularized version of Anderson acceleration applied to the same problem. In al…
The principle underlying this paper is the basic observation that the problem of simultaneously solving a large class of composite monotone inclusions and their duals can be reduced to that of finding a zero of the sum of a maximally monotone operator and a linear skew-adjoint operator. An algorithmic framework is developed for solving this generic problem in a Hilbert space setting. New primal-dual splitting algorithms are derived from this framework for inclusions involving composite monotone operators, and convergence results are established. These algorithms draw their simplicity and effic…
Kuhn-Tucker points play a fundamental role in the analysis and the numerical solution of monotone inclusion problems, providing in particular both primal and dual solutions. We propose a class of strongly convergent algorithms for constructing the best approximation to a reference point from the set of Kuhn-Tucker points of a general Hilbertian composite monotone inclusion problem. Applications to systems of coupled monotone inclusions are presented. Our framework does not impose additional assumptions on the operators present in the formulation, and it does not require knowledge of the norm o…
We focus on the linear convergence of generalized proximal point algorithms for solving monotone inclusion problems. Under the assumption that the associated monotone operator is metrically subregular or that the inverse of the monotone operator is Lipschitz continuous, we provide Q-linear and R-linear convergence results on generalized proximal point algorithms. Comparisons between our results and related ones in the literature are presented in remarks of this work.
We propose a splitting algorithm for solving a system of composite monotone inclusions formulated in the form of the extended set of solutions in real Hilbert spaces. The resluting algorithm is a an extension of the algorithm in [4]. The weak convergence of the algorithm proposed is proved. Applications to minimization problems is demonstrated.
We give criteria for weak and strong invariant closed sets for differential inclusions given in $\mathbb{R}^{n}$ and governed by Lipschitz Cusco perturbations of maximal monotone operators. Correspondingly, we provide different characterizations for the\ associated strong Lyapunov functions. The resulting conditions only depend on the data of the system.
We introduce the first operator splitting method for composite monotone inclusions outside of Hilbert spaces. The proposed primal-dual method constructs iteratively the best Bregman approximation to an arbitrary point from the Kuhn-Tucker set of a composite monotone inclusion. Strong convergence is established in reflexive Banach spaces without requiring additional restrictions on the monotone operators or knowledge of the norms of the linear operators involved in the model. The monotone operators are activated via Bregman distance-based resolvent operators. The method is novel even in Euclide…
We propose a variable metric forward-backward splitting algorithm and prove its convergence in real Hilbert spaces. We then use this framework to derive primal-dual splitting algorithms for solving various classes of monotone inclusions in duality. Some of these algorithms are new even when specialized to the fixed metric case. Various applications are discussed.
We propose an inertial forward-backward splitting algorithm to compute the zero of a sum of two monotone operators allowing for stochastic errors in the computation of the operators. More precisely, we establish almost sure convergence in real Hilbert spaces of the sequence of iterates to an optimal solution. Then, based on this analysis, we introduce two new classes of stochastic inertial primal-dual splitting methods for solving structured systems of composite monotone inclusions and prove their convergence. Our results extend to the stochastic and inertial setting various types of structure…
Monotone inclusions have wide applications in solving various convex optimization problems arising in signal and image processing, machine learning, and medical image reconstruction. In this paper, we propose a new splitting algorithm for finding a zero of the sum of a maximally monotone operator, a monotone Lipschitzian operator, and a cocoercive operator, which is called outer reflected forward-backward splitting algorithm. Under mild conditions on the iterative parameters, we prove the convergence of the proposed algorithm. As applications, we employ the proposed algorithm to solve composit…
We propose a new class of primal-dual Fejer monotone algorithms for solving systems of com- posite monotone inclusions. Our construction is inspired by a framework used by Eckstein and Svaiter for the basic problem of finding a zero of the sum of two monotone operators. At each iteration, points in the graph of the monotone operators present in the model are used to con- struct a half-space containing the Kuhn-Tucker set associated with the system. The primal-dual update is then obtained via a relaxed projection of the current iterate onto this half-space. An important feature that distinguish…
In this paper, based on a double inertial extrapolation steps strategy and relaxation techniques, we introduce a new Tseng splitting method with double inertial extrapolation steps and self-adaptive step sizes for solving monotone inclusion problems in real Hilbert spaces. Finally, several numerical experiments are provided to illustrate the performance and theoretical outcomes of our algorithm.
In this paper, we introduce for the first time a class of state-dependent maximal monotone differential inclusions. Then the existence and uniqueness of solutions are obtained by using an implicit discretization scheme and a kind of hypo-monotonicity assumption respectively. In addition, a characterization for nonsmooth Lyapunov pairs associated with such systems is provided. Our result can be applied to study state-dependent sweeping processes and Lur'e dynamical systems. It is new even the involved maximal monotone operators depend only on the time.
In this paper, we propose a numerical approach for solving composite primal-dual monotone inclusions with a priori information. The underlying a priori information set is represented by the intersection of fixed point sets of a finite number of operators, and we propose and algorithm that activates the corresponding set by following a finite-valued random variable at each iteration. Our formulation is flexible and includes, for instance, deterministic and Bernoulli activations over cyclic schemes, and Kaczmarz-type random activations. The almost sure convergence of the algorithm is obtained by…
The proximal extrapolated gradient method \cite{Malitsky18a} is an extension of the projected reflected gradient method \cite{Malitsky15}. Both methods were proposed for solving the classic variational inequalities. In this paper, we investigate the projected reflected gradient method, in the general setting, for solving monotone inclusions involving Lipschitzian operators. As a result, we obtain a simple method for finding a zero point of the sum of two monotone operators where one of them is Lipschizian. We also show that one can improve the range of the stepsize of this method for the case …
A parallel splitting method is proposed for solving systems of coupled monotone inclusions in Hilbert spaces. Convergence is established for a wide class of coupling schemes. Unlike classical alternating algorithms, which are limited to two variables and linear coupling, our parallel method can handle an arbitrary number of variables as well as nonlinear coupling schemes. The breadth and flexibility of the proposed framework is illustrated through applications in the areas of evolution inclusions, dynamical games, signal recovery, image decomposition, best approximation, network flows, and var…
We introduce a generalized forward-backward splitting method with penalty term for solving monotone inclusion problems involving the sum of a finite number of maximally monotone operators and the normal cone to the nonempty set of zeros of another maximal monotone operator. We show weak ergodic convergence of the generated sequence of iterates to a solution of the considered monotone inclusion problem, provided the condition corresponded to the Fitzpatrick function of the operator describing the set of the normal cone is fulfilled. Under strong monotonicity of an operator, we show strong conve…
This paper studies the iteration-complexity of new regularized hybrid proximal extragradient (HPE)-type methods for solving monotone inclusion problems (MIPs). The new (regularized HPE-type) methods essentially consist of instances of the standard HPE method applied to regularizations of the original MIP. It is shown that its pointwise iteration-complexity considerably improves the one of the HPE method while approaches (up to a logarithmic factor) the ergodic iteration-complexity of the latter method.
We provide two weakly convergent algorithms for finding a zero of the sum of a maximally monotone operator, a cocoercive operator, and the normal cone to a closed vector subspace of a real Hilbert space. The methods exploit the intrinsic structure of the problem by activating explicitly the cocoercive operator in the first step, and taking advantage of a vector space decomposition in the second step. The second step of the first method is a Douglas-Rachford iteration involving the maximally monotone operator and the normal cone. In the second method it is a proximal step involving the partial …
Resolvents of set-valued operators play a central role in various branches of mathematics and in particular in the design and the analysis of splitting algorithms for solving monotone inclusions. We propose a generalization of this notion, called warped resolvent, which is constructed with the help of an auxiliary operator. The properties of warped resolvents are investigated and connections are made with existing notions. Abstract weak and strong convergence principles based on warped resolvents are proposed and shown to not only provide a synthetic view of splitting algorithms but to also co…
In this paper we provide a splitting algorithm for solving coupled monotone inclusions in a real Hilbert space involving the sum of a normal cone to a vector subspace, a maximally monotone, a monotone-Lipschitzian, and a cocoercive operator. The proposed method takes advantage of the intrinsic properties of each operator and generalizes the method of partial inverses and the forward-backward-half forward splitting, among other methods. At each iteration, our algorithm needs two computations of the Lipschitzian operator while the cocoercive operator is activated only once. By using product spac…
We propose and analyze the convergence of a novel stochastic algorithm for solving monotone inclusions that are the sum of a maximal monotone operator and a monotone, Lipschitzian operator. The propose algorithm requires only unbiased estimations of the Lipschitzian operator. We obtain the rate $\mathcal{O}(log(n)/n)$ in expectation for the strongly monotone case, as well as almost sure convergence for the general case. Furthermore, in the context of application to convex-concave saddle point problems, we derive the rate of the primal-dual gap. In particular, we also obtain $\mathcal{O}(1/n)$ …
In this work, we propose a simple modification of the forward-backward splitting method for finding a zero in the sum of two monotone operators. Our method converges under the same assumptions as Tseng's forward-backward-forward method, namely, it does not require cocoercivity of the single-valued operator. Moreover, each iteration only requires one forward evaluation rather than two as is the case for Tseng's method. Variants of the method incorporating a linesearch, relaxation and inertia, or a structured three operator inclusion are also discussed.
The aim of this survey is to present the main important techniques and tools from variational analysis used for first and second order dynamical systems of implicit type for solving monotone inclusions and non-smooth optimization problems. The differential equations are expressed by means of the resolvent (in case of a maximally monotone set valued operator) or the proximal operator for non-smooth functions. The asymptotic analysis of the trajectories generated relies on Lyapunov theory, where the appropriate energy functional plays a decisive role. While the most part of the paper is related …
In this paper we provide a generalization of the Douglas-Rachford splitting (DRS) and the primal-dual algorithm (Vu 2013, Condat 2013) for solving monotone inclusions in a real Hilbert space involving a general linear operator. The proposed method allows for primal and dual non-standard metrics and activates the linear operator separately from the monotone operators appearing in the inclusion. In the simplest case when the linear operator has full range, it reduces to classical DRS. Moreover, the weak convergence of primal-dual sequences to a Kuhn-Tucker point is guaranteed, generalizing the m…
In this work we propose a new splitting technique, namely Asymmetric Forward-Backward-Adjoint splitting, for solving monotone inclusions involving three terms, a maximally monotone, a cocoercive and a bounded linear operator. Classical operator splitting methods, like Douglas-Rachford and Forward-Backward splitting are special cases of our new algorithm. Asymmetric Forward-Backward-Adjoint splitting unifies, extends and sheds light on the connections between many seemingly unrelated primal-dual algorithms for solving structured convex optimization problems proposed in recent years. More import…
Tseng's algorithm finds a zero of the sum of a maximally monotone operator and a monotone continuous operator by evaluating the latter twice per iteration. In this paper, we modify Tseng's algorithm for finding a zero of the sum of three operators, where we add a cocoercive operator to the inclusion. Since the sum of a cocoercive and a monotone-Lipschitz operator is monotone and Lipschitz, we could use Tseng's method for solving this problem, but implementing both operators twice per iteration and without taking into advantage the cocoercivity property of one operator. Instead, in our approach…
In this paper, we study the existence and the stability in the sense of Lyapunov of solutions for\ differential inclusions governed by the normal cone to a prox-regular set and subject to a Lipschitzian perturbation. We prove that such, apparently, more general nonsmooth dynamics can be indeed remodelled into the classical theory of differential inclusions involving maximal monotone operators. This result is new in the literature and permits us to make use of the rich and abundant achievements in this class of monotone operators to derive the desired existence result and stability analysis, as…
