Using the theory of Hilbert direct integrals, we introduce and study a monotonicity-preserving operation, termed the integral resolvent mixture. It combines arbitrary families of monotone operators acting on different spaces and linear operators. As a special case, we investigate the resolvent expectation, an operation which combines monotone operators in such a way that the resulting resolvent is the Lebesgue expectation of the individual resolvents. Along the same lines, we introduce an operation that mixes arbitrary families of convex functions defined on different spaces and linear operators to create a composite convex function. Such constructs have so far been limited to finite families of operators and functions. The subdifferential of the integral proximal mixture is shown to be the integral resolvent mixture of the individual subdifferentials. Applications to the relaxation of systems of composite monotone inclusions are presented.
We propose a level proximal subdifferential for a proper lower semicontinuous function. Level proximal subdifferential is a uniform refinement of the well-known proximal subdifferential, and has the pleasant feature that its resolvent always coincides with the proximal mapping of a function. It turns out that the resolvent representation of proximal mapping in terms of Mordukhovich limiting subdifferential is only valid for hypoconvex functions. We also provide properties of level proximal subdifferential and numerous examples to illustrate our results.
We show that the set of fixed points of the average of two resolvents can be found from the set of fixed points for compositions of two resolvents associated with scaled monotone operators. Recently, the proximal average has attracted considerable attention in convex analysis. Our results imply that the minimizers of proximal-average functions can be found from the set of fixed points for compositions of two proximal mappings associated with scaled convex functions. When both convex functions in the proximal average are indicator functions of convex sets, least squares solutions can be complet…
Within convex analysis, a rich theory with various applications has been evolving since the proximal average of convex functions was first introduced over a decade ago. When one considers the subdifferential of the proximal average, a natural averaging operation of the subdifferentials of the averaged functions emerges. In the present paper we extend the reach of this averaging operation to the framework of monotone operator theory in Hilbert spaces, transforming it into the resolvent average. The theory of resolvent averages contains many desirable properties. In particular, we study a detail…
A new framework for analyzing Fejer convergent algorithms is presented. Using this framework we define a very general class of Fejer convergent algorithms and establish its convergence properties. We also introduce a new definition of approximations of resolvents which preserve some useful features of the exact resolvent, and use this concept to present an unifying view of the Forward-Backward splitting method, Tseng's Modified Forward-Backward splitting method and Korpelevich's method. We show that methods based on families of approximate resolvents fall within the aforementioned class of Fej…
Resolvents of set-valued operators play a central role in various branches of mathematics and in particular in the design and the analysis of splitting algorithms for solving monotone inclusions. We propose a generalization of this notion, called warped resolvent, which is constructed with the help of an auxiliary operator. The properties of warped resolvents are investigated and connections are made with existing notions. Abstract weak and strong convergence principles based on warped resolvents are proposed and shown to not only provide a synthetic view of splitting algorithms but to also co…
We introduce and investigate the resolvent order, which is a binary relation on the set of firmly nonexpansive mappings. It unifies well-known orders introduced by Loewner (for positive semidefinite matrices) and by Zarantonello (for projectors onto convex cones). A connection with Moreau's order of convex functions is also presented. We also construct partial orders on (quotient sets of) proximal mappings and convex functions. Various examples illustrate our results.
The proximal point algorithm is a widely used tool for solving a variety of convex optimization problems such as finding zeros of maximally monotone operators, fixed points of nonexpansive mappings, as well as minimizing convex functions. The algorithm works by applying successively so-called "resolvent" mappings associated to the original object that one aims to optimize. In this paper we abstract from the corresponding resolvents employed in these problems the natural notion of jointly firmly nonexpansive families of mappings. This leads to a streamlined method of proving weak convergence of…
We investigate the asymptotic behavior of sequences generated by the proximal point algorithm for convex functions in complete geodesic spaces with curvature bounded above. Using the notion of resolvents of such functions, which was recently introduced by the authors, we show the existence of minimizers of convex functions under the boundedness assumptions on such sequences as well as the convergence of such sequences to minimizers of given functions.
The correspondence between the monotonicity of a (possibly) set-valued operator and the firm nonexpansiveness of its resolvent is a key ingredient in the convergence analysis of many optimization algorithms. Firmly nonexpansive operators form a proper subclass of the more general - but still pleasant from an algorithmic perspective - class of averaged operators. In this paper, we introduce the new notion of conically nonexpansive operators which generalize nonexpansive mappings. We characterize averaged operators as being resolvents of comonotone operators under appropriate scaling. As a conse…
We revisit the operator splitting schemes proposed in a recent work of [Some extensions of the operator splitting schemes based on Lagrangian and primal-dual: A unified proximal point analysis, Feng Xue, Optimization, 2022, doi: 10.1080/02331934.2022.2057309], and further analyze the convergence of the generalized Bregman distance and the primal-dual gap of these algorithms within a unified proximal point framework. The possibility of reduction to a simple resolvent is also discussed by exploiting the structure and possible degeneracy of the underlying metric.
We investigate the asymptotic behavior of a stochastic version of the forward-backward splitting algorithm for finding a zero of the sum of a maximally monotone set-valued operator and a cocoercive operator in Hilbert spaces. Our general setting features stochastic approximations of the cocoercive operator and stochastic perturbations in the evaluation of the resolvents of the set-valued operator. In addition, relaxations and not necessarily vanishing proximal parameters are allowed. Weak and strong almost sure convergence properties of the iterates is established under mild conditions on the …
We propose a primal-dual splitting algorithm for solving monotone inclusions involving a mixture of sums, linear compositions, and parallel sums of set-valued and Lipschitzian operators. An important feature of the algorithm is that the Lipschitzian operators present in the formulation can be processed individually via explicit steps, while the set-valued operators are processed individually via their resolvents. In addition, the algorithm is highly parallel in that most of its steps can be executed simultaneously. This work brings together and notably extends various types of structured monot…
We propose a new primal-dual splitting method for solving composite inclusions involving Lipschitzian, and parallel-sum-type monotone operators. Our approach extends the framework in \cite{Siopt4} to a more general class of monotone inclusions in a nontrivial fashion. The main idea is to represent the solution set of both the primal and dual problems using their associated Kuhn-Tucker set, and then develop a projected method to successively approximate a feasible point of the Kuhn-Tucker set. We propose a splitting algorithm based on the resolvent of each maximally monotone operator to constru…
In this work, we propose and analyse two splitting algorithms for finding a zero of the sum of three monotone operators, one of which is assumed to be Lipschitz continuous. Each iteration of these algorithms require one forward evaluation of the Lipschitz continuous operator and one resolvent evaluation of each of the other two operators. By specialising to two operator inclusions, we recover the forward-reflected-backward and the reflected-forward-backward splitting methods as particular cases. The inspiration for the proposed algorithms arises from interpretations of the aforementioned refle…
We introduce the first operator splitting method for composite monotone inclusions outside of Hilbert spaces. The proposed primal-dual method constructs iteratively the best Bregman approximation to an arbitrary point from the Kuhn-Tucker set of a composite monotone inclusion. Strong convergence is established in reflexive Banach spaces without requiring additional restrictions on the monotone operators or knowledge of the norms of the linear operators involved in the model. The monotone operators are activated via Bregman distance-based resolvent operators. The method is novel even in Euclide…
We investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in {\em Y. Drori, S. Sabach, and M. Teboulle, A simple algorithm for a class of nonsmooth convex-concave saddle-point problems, 2015} and {\em I. Loris and C. Verhoeven, On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty, 2011} for saddl…
Inertial algorithms for minimizing nonsmooth and nonconvex functions as the inertial proximal alternating linearized minimization algorithm (iPALM) have demonstrated their superiority with respect to computation time over their non inertial variants. In many problems in imaging and machine learning, the objective functions have a special form involving huge data which encourage the application of stochastic algorithms. While algorithms based on stochastic gradient descent are still used in the majority of applications, recently also stochastic algorithms for minimizing nonsmooth and nonconvex …
