Weak convergence on Douglas-Rachford method

The Cyclic Douglas-Rachford Algorithm with r-sets-Douglas-Rachford Operators

The Douglas-Rachford (DR) algorithm is an iterative procedure that uses sequential reflections onto convex sets and which has become popular for convex feasibility problems. In this paper we propose a structural generalization that allows to use $r$-sets-DR operators in a cyclic fashion. We prove convergence and present numerical illustrations of the potential advantage of such operators with $r>2$ over the classical $2$-sets-DR operators in a cyclic algorithm.

A parameterized Douglas-Rachford Splitting algorithm for nonconvex optimization

In this paper, we study a parameterized Douglas-Rachford splitting method for a class of nonconvex optimization problem. A new merit function is constructed to establish the convergence of the whole sequence generated by the parameterized Douglas-Rachford splitting method. We then apply the parameterized Douglas-Rachford splitting method to three important classes of nonconvex optimization problems arising in data science: sparsity constrained least squares problem, feasibility problem and low rank matrix completion. Numerical results validate the effectiveness of the parameterized Douglas-Rac…

Circumcentering the Douglas--Rachford method

We introduce and study a geometric modification of the Douglas-Rach\-ford method called the Circumcentered-Douglas-Rachford method. This method iterates by taking the intersection of bisectors of reflection steps for solving certain classes of feasibility problems. The convergence analysis is established for best approximation problems involving two (affine) subspaces and both our theoretical and numerical results compare favorably to the original Douglas-Rachford method. Under suitable conditions, it is shown that the linear rate of convergence of the Circumcentered-Douglas-Rachford method is…

Douglas-Rachford Splitting: Complexity Estimates and Accelerated Variants

We propose a new approach for analyzing convergence of the Douglas-Rachford splitting method for solving convex composite optimization problems. The approach is based on a continuously differentiable function, the Douglas-Rachford Envelope (DRE), whose stationary points correspond to the solutions of the original (possibly nonsmooth) problem. By proving the equivalence between the Douglas-Rachford splitting method and a scaled gradient method applied to the DRE, results from smooth unconstrained optimization are employed to analyze convergence properties of DRS, to tune the method and to deriv…

The Douglas--Rachford Algorithm Converges Only Weakly

We show that the weak convergence of the Douglas--Rachford algorithm for finding a zero of the sum of two maximally monotone operators cannot be improved to strong convergence. Likewise, we show that strong convergence can fail for the method of partial inverses.

Douglas-Rachford Feasibility Methods for Matrix Completion Problems

In this paper we give general recommendations for successful application of the Douglas-Rachford reflection method to convex and non-convex real matrix-completion problems. These guidelines are demonstrated by various illustrative examples.

Finding magic squares with the Douglas-Rachford algorithm

In this expository paper, we show how to use the Douglas-Rachford algorithm as a successful heuristic for finding magic squares. The Douglas-Rachford algorithm is an iterative projection method for solving feasibility problems. Although its convergence is only guaranteed in the convex setting, the algorithm has been successfully applied to a number of similar nonconvex problems, such as solving Sudoku puzzles. We present two formulations of the nonconvex feasibility problem of finding magic squares, which are inspired by those of Sudoku, and test the Douglas-Rachford algorithm on them.

A remark on the convergence of the Douglas-Rachford iteration in a non-convex setting

Using the construction of a Lyapunov function, it is shown that the Douglas-Rachford iteration with respect to a sphere and a line in $\mathbb R^d$ is robustly $\mathcal{KL}$-stable. This implies a convergence which is stronger than uniform convergence on compact sets.

The Douglas-Rachford Algorithm for Weakly Convex Penalties

The Douglas-Rachford algorithm is widely used in sparse signal processing for minimizing a sum of two convex functions. In this paper, we consider the case where one of the functions is weakly convex but the other is strongly convex so that the sum is convex. We provide a condition that ensures the convergence of the same Douglas-Rachford iterations, provided that the strongly convex function is smooth. We also present a modified Douglas-Rachford algorithm that does not impose a smoothness condition for the convex function. We then provide a discussion on the convergence speed of the two types…

Ergodic behaviour of a Douglas-Rachford operator away from the origin

It is shown that away from the origin, the Douglas-Rachford operator with respect to a sphere and a convex set in a Hilbert space can be approximated by a another operator which satisfies a weak ergodic theorem. Similar results for other projection and reflection operators are also discussed.

The Cyclic Douglas-Rachford Method for Inconsistent Feasibility Problems

We analyse the behaviour of the newly introduced cyclic Douglas-Rachford algorithm for finding a point in the intersection of a finite number of closed convex sets. This work considers the case in which the target intersection set is possibly empty.

A Douglas-Rachford Splitting Method for Solving Equilibrium Problems

We propose a splitting method for solving an equilibrium problem involving the sum of two bifunctions satisfying standard conditions. We prove that this problem is equivalent to find a zero of two appropriate maximally monotone operators. Our algorithm is a consequence of the Douglas--Rachford splitting applied to this auxiliary monotone inclusion. Connections between monotone inclusions and equilibrium problems are studied.

The Fun is Finite: Douglas-Rachford and Sudoku Puzzle -- Finite Termination and Local Linear Convergence

In recent years, the Douglas-Rachford splitting method has been shown to be effective at solving many non-convex optimization problems. In this paper we present a local convergence analysis for non-convex feasibility problems and show that both finite termination and local linear convergence are obtained. For a generalization of the Sudoku puzzle, we prove that the local linear rate of convergence of Douglas-Rachford is exactly $\frac{\sqrt{5}}{5}$ and independent of puzzle size. For the $s$-queens problem we prove that Douglas-Rachford converges after a finite number of iterations. Numerical …

Linear convergence of the generalized Douglas-Rachford algorithm for feasibility problems

In this paper, we study the generalized Douglas-Rachford algorithm and its cyclic variants which include many projection-type methods such as the classical Douglas-Rachford algorithm and the alternating projection algorithm. Specifically, we establish several local linear convergence results for the algorithm in solving feasibility problems with finitely many closed possibly nonconvex sets under different assumptions. Our findings not only relax some regularity conditions but also improve linear convergence rates in the literature. In the presence of convexity, the linear convergence is global…

关于一致单调性存在且无最小化子情况下的Douglas-Rachford与Peaceman-Rachford算法

On the Douglas-Rachford and Peaceman-Rachford algorithms in the presence of uniform monotonicity and the absence of minimizers

本文证明了在一致单调且$3^*$单调（非次微分）条件下，Douglas-Rachford与Peaceman-Rachford算法的影子序列收敛性，并用数值实验验证。

Global convergence of a non-convex Douglas-Rachford iteration

We establish a region of convergence for the proto-typical non-convex Douglas-Rachford iteration which finds a point on the intersection of a line and a circle. Previous work on the non-convex iteration [2] was only able to establish local convergence, and was ineffective in that no explicit region of convergence could be given.

Recent Results on Douglas-Rachford Methods for Combinatorial Optimization Problems

We discuss recent positive experiences applying convex feasibility algorithms of Douglas--Rachford type to highly combinatorial and far from convex problems.

通过Moreau型包络的视角理解Douglas-Rachford分裂方法

Understanding the Douglas-Rachford splitting method through the lenses of Moreau-type envelopes

本文通过Douglas-Rachford包络分析弱凸优化问题的Douglas-Rachford分裂方法，利用上图收敛技术证明其逼近目标函数，并在温和假设下建立全局收敛与局部线性收敛率，数值实验验证了其与凸情形相当的性能。

The Douglas-Rachford algorithm for two (not necessarily intersecting) affine subspaces

The Douglas--Rachford algorithm is a classical and very successful splitting method for finding the zeros of the sums of monotone operators. When the underlying operators are normal cone operators, the algorithm solves a convex feasibility problem. In this paper, we provide a detailed study of the Douglas--Rachford iterates and the corresponding {shadow sequence} when the sets are affine subspaces that do not necessarily intersect. We prove strong convergence of the shadows to the nearest generalized solution. Our results extend recent work from the consistent to the inconsistent case. Various…

Iteration complexity of an inexact Douglas-Rachford method and of a Douglas-Rachford-Tseng's F-B four-operator splitting method for solving monotone inclusions

In this paper, we propose and study the iteration complexity of an inexact Douglas-Rachford splitting (DRS) method and a Douglas-Rachford-Tseng's forward-backward (F-B) splitting method for solving two-operator and four-operator monotone inclusions, respectively. The former method (although based on a slightly different mechanism of iteration) is motivated by the recent work of J. Eckstein and W. Yao, in which an inexact DRS method is derived from a special instance of the hybrid proximal extragradient (HPE) method of Solodov and Svaiter, while the latter one combines the proposed inexact DRS …

On the Minimal Displacement Vector of the Douglas-Rachford Operator

The Douglas-Rachford algorithm can be represented as the fixed point iteration of a firmly nonexpansive operator. When the operator has no fixed points, the algorithm's iterates diverge, but the difference between consecutive iterates converges to the so-called minimal displacement vector, which can be used to certify infeasibility of an optimization problem. In this paper, we establish new properties of the minimal displacement vector, which allow us to generalize some existing results.

用于不相容非凸可行性问题的循环松弛 Douglas–Rachford 分裂法

Cyclic Relaxed Douglas-Rachford Splitting for Inconsistent Nonconvex Feasibility

本文提出循环松弛 Douglas–Rachford 算法以处理非凸不相容可行性问题，刻画其不动点及影子与循环投影不动点的关系，并给出局部定量收敛条件。

Finding the forward-Douglas-Rachford-forward method

We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas--Rachford and Forward-backward-forward methods respectively solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas--Rachford and Forward-backward-forward and show that it solves the 3 operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas--Rachford and…

New Douglas-Rachford algorithmic structures and their convergence analyses

In this paper we study new algorithmic structures with Douglas- Rachford (DR) operators to solve convex feasibility problems. We propose to embed the basic two-set-DR algorithmic operator into the String-Averaging Projections (SAP) and into the Block-Iterative Pro- jection (BIP) algorithmic structures, thereby creating new DR algo- rithmic schemes that include the recently proposed cyclic Douglas- Rachford algorithm and the averaged DR algorithm as special cases. We further propose and investigate a new multiple-set-DR algorithmic operator. Convergence of all these algorithmic schemes is studi…

A Lyapunov-type approach to convergence of the Douglas-Rachford algorithm

The Douglas-Rachford projection algorithm is an iterative method used to find a point in the intersection of closed constraint sets. The algorithm has been experimentally observed to solve various nonconvex feasibility problems which current theory cannot sufficiently explain. In this paper, we prove convergence of the Douglas-Rachford algorithm in a potentially nonconvex setting. Our analysis relies on the existence of a Lyapunov-type functional whose convexity properties are not tantamount to convexity of the original constraint sets. Moreover, we provide various nonconvex examples in which …

关于失效为近映射的Douglas-Rachford算子

On Douglas-Rachford operators that fail to be proximal mappings

研究Douglas-Rachford算子在某些情况下并非近映射的性质，通过分析对称线性关系的极大单调性，证明该算子通常不构成近映射。

Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMM

Recently, several convergence rate results for Douglas-Rachford splitting and the alternating direction method of multipliers (ADMM) have been presented in the literature. In this paper, we show global linear convergence rate bounds for Douglas-Rachford splitting and ADMM under strong convexity and smoothness assumptions. We further show that the rate bounds are tight for the class of problems under consideration for all feasible algorithm parameters. For problems that satisfy the assumptions, we show how to select step-size and metric for the algorithm that optimize the derived convergence ra…

用于广义DC规划问题的惯性参数化Douglas-Rachford算法

Inertial parameterized Douglas-Rachford for generalized DC programming problems

该研究针对广义DC规划问题，提出两种惯性参数化Douglas-Rachford型算法，经实验验证其在稀疏恢复、鲁棒分类任务中性能优于现有同类算法。

Relative-error inertial-relaxed inexact versions of Douglas-Rachford and ADMM splitting algorithms

This paper derives new inexact variants of the Douglas-Rachford splitting method for maximal monotone operators and the alternating direction method of multipliers (ADMM) for convex optimization. The analysis is based on a new inexact version of the proximal point algorithm that includes both an inertial step and overrelaxation. We apply our new inexact ADMM method to LASSO and logistic regression problems and obtain somewhat better computational performance than earlier inexact ADMM methods.

Douglas–Rachford 是最佳投影方法

Douglas--Rachford is the best projection method

本文利用圆动力学证明，平面上两闭凸锥的Douglas–Rachford方法有限收敛当且仅当其算子不动点集非平凡，并指出其优于其他投影方法，统一并扩展了局部多面体情形下的有限收敛充分条件。

关于求解欧几里得空间中（不一定是凸的）可行性问题的Douglas-Rachford算法有限收敛性的研究

On the finite convergence of the Douglas-Rachford algorithm for solving (not necessarily convex) feasibility problems in Euclidean spaces

本文研究了求解欧几里得空间中非凸可行性问题的Douglas-Rachford算法的有限收敛性，提出了多种新的充分收敛条件，并通过多个例子验证了结果。

On Douglas-Rachford splitting that generally fails to be a proximal mapping: A degenerate proximal point analysis

Based on a degenerate proximal point analysis, we show that the Douglas-Rachford splitting can be reduced to a well-defined resolvent, but generally fails to be a proximal mapping. This extends the recent result of [Bauschke, Schaad and Wang. Math. Program., 168 (2018), pp. 55-61] to more general setting. The related concepts and consequences are also discussed. In particular, the results regarding the maximal and cyclic monotonicity are instrumental for analyzing many operator splitting algorithms.

关于Douglas-Rachford算法中算子顺序的顺序

On the order of the operators in the Douglas-Rachford algorithm

本文研究了Douglas-Rachford算法中两个算子顺序的影响，证明了反射算子在两个Douglas-Rachford算子的不动点集之间建立双射关系，并通过例子展示了结果。

On Convergence of Heuristics Based on Douglas-Rachford Splitting and ADMM to Minimize Convex Functions over Nonconvex Sets

Recently, heuristics based on the Douglas-Rachford splitting algorithm and the alternating direction method of multipliers (ADMM) have found empirical success in minimizing convex functions over nonconvex sets, but not much has been done to improve the theoretical understanding of them. In this paper, we investigate convergence of these heuristics. First, we characterize optimal solutions of minimization problems involving convex cost functions over nonconvex constraint sets. We show that these optimal solutions are related to the fixed point set of the underlying nonconvex Douglas-Rachford op…

On Slater's condition and finite convergence of the Douglas-Rachford algorithm

The Douglas-Rachford algorithm is a classical and very successful method for solving optimization and feasibility problems. In this paper, we provide novel conditions sufficient for finite convergence in the context of convex feasibility problems. Our analysis builds upon, and considerably extends, pioneering work by Spingarn. Specifically, we obtain finite convergence in the presence of Slater's condition in the affine-polyhedral and in a hyperplanar-epigraphical case. Various examples illustrate our results. Numerical experiments demonstrate the competitiveness of the Douglas-Rachford algori…

基于 Douglas–Rachford 包络的通信高效拟牛顿分布式优化

Communication efficient quasi-Newton distributed optimization based on the Douglas-Rachford envelope

针对客户端–服务器分布式优化，本文利用 Douglas–Rachford 分裂构造免线搜索的 BFGS 方法，以每轮每客户端仅收发一个向量实现最少通信和超线性收敛，并通过实验验证其降本优势。

Shadow Douglas--Rachford Splitting for Monotone Inclusions

In this work, we propose a new algorithm for finding a zero in the sum of two monotone operators where one is assumed to be single-valued and Lipschitz continuous. This algorithm naturally arises from a non-standard discretization of a continuous dynamical system associated with the Douglas--Rachford splitting algorithm. More precisely, it is obtained by performing an explicit, rather than implicit, discretization with respect to one of the operators involved. Each iteration of the proposed algorithm requires the evaluation of one forward and one backward operator.

An enhanced formulation for solving graph coloring problems with the Douglas-Rachford algorithm

We study the behavior of the Douglas-Rachford algorithm on the graph vertex-coloring problem. Given a graph and a number of colors, the goal is to find a coloring of the vertices so that all adjacent vertex pairs have different colors. In spite of the combinatorial nature of this problem, the Douglas-Rachford algorithm was recently shown to be a successful heuristic for solving a wide variety of graph coloring instances, when the problem was cast as a feasibility problem on binary indicator variables. In this work we consider a different formulation, based on semidefinite programming. The much…

Tight Linear Convergence Rate Bounds for Douglas-Rachford Splitting and ADMM

Douglas-Rachford splitting and the alternating direction method of multipliers (ADMM) can be used to solve convex optimization problems that consist of a sum of two functions. Convergence rate estimates for these algorithms have received much attention lately. In particular, linear convergence rates have been shown by several authors under various assumptions. One such set of assumptions is strong convexity and smoothness of one of the functions in the minimization problem. The authors recently provided a linear convergence rate bound for such problems. In this paper, we show that this rate bo…

Stadium norm and Douglas-Rachford splitting: a new approach to road design optimization

The basic optimization problem of road design is quite challenging due to a objective function that is the sum of nonsmooth functions and the presence of set constraints. In this paper, we model and solve this problem by employing the Douglas-Rachford splitting algorithm. This requires a careful study of new proximity operators related to minimizing area and to the stadium norm. We compare our algorithm to a state-of-the-art projection algorithm. Our numerical results illustrate the potential of this algorithm to significantly reduce cost in road design.

Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence results

Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising globa…

On the Douglas-Rachford algorithm for solving possibly inconsistent optimization problems

More than 40 years ago, Lions and Mercier introduced in a seminal paper the Douglas-Rachford algorithm. Today, this method is well recognized as a classical and highly successful splitting method to find minimizers of the sum of two (not necessarily smooth) convex functions. While the underlying theory has matured, one case remains a mystery: the behaviour of the shadow sequence when the given functions have disjoint domains. Building on previous work, we establish for the first time weak and value convergence of the shadow sequence generated by the Douglas-Rachford algorithm in a setting of u…

最优控制对偶性与Douglas-Rachford算法

Optimal Control Duality and the Douglas-Rachford Algorithm

本文揭示了加权最小能量控制问题对偶与Douglas-Rachford算法的关系，导出DR算子不动点表达式以构建最优性证书，并通过示例验证。

双重闭集的Douglas-Rachford方法的线性收敛性

Linear Convergence of the Douglas-Rachford Method for Two Closed Sets

本文研究了欧几里得空间中两个闭集（可能非凸）的Douglas-Rachford方法，证明在一定正则条件下方法具有局部R线性收敛性，凸情况下收敛性为全局。

Affine nonexpansive operators, Attouch-Théra duality and the Douglas-Rachford algorithm

The Douglas-Rachford splitting algorithm was originally proposed in 1956 to solve a system of linear equations arising from the discretization of a partial differential equation. In 1979, Lions and Mercier brought forward a very powerful extension of this method suitable to solve optimization problems. In this paper, we revisit the original affine setting. We provide a powerful convergence result for finding a zero of the sum of two maximally monotone affine relations. As a by product of our analysis, we obtain results concerning the convergence of iterates of affine nonexpansive mappings as w…

Adaptive Douglas-Rachford splitting algorithm for the sum of two operators

The Douglas-Rachford algorithm is a classical and powerful splitting method for minimizing the sum of two convex functions and, more generally, finding a zero of the sum of two maximally monotone operators. Although this algorithm is well understood when the involved operators are monotone or strongly monotone, the convergence theory for weakly monotone settings is far from being complete. In this paper, we propose an adaptive Douglas-Rachford splitting algorithm for the sum of two operators, one of which is strongly monotone while the other one is weakly monotone. With appropriately chosen pa…

Solving graph coloring problems with the Douglas-Rachford algorithm

We present the Douglas-Rachford algorithm as a successful heuristic for solving graph coloring problems. Given a set of colors, these type of problems consist in assigning a color to each node of a graph, in such a way that every pair of adjacent nodes are assigned with different colors. We formulate the graph coloring problem as an appropriate feasibility problem that can be effectively solved by the Douglas-Rachford algorithm, despite the nonconvexity arising from the combinatorial nature of the problem. Different modifications of the graph coloring problem and applications are also presente…

Approximate Douglas-Rachford algorithm for two-sets convex feasibility problems

In this paper, we propose a new algorithm combining the Douglas-Rachford (DR) algorithm and the Frank-Wolfe algorithm, also known as the conditional gradient (CondG) method, for solving the classic convex feasibility problem. Within the algorithm, which will be named {\it Approximate Douglas-Rachford (ApDR) algorithm}, the CondG method is used as a subroutine to compute feasible inexact projections on the sets under consideration, and the ApDR iteration is defined based on the DR iteration. The ApDR algorithm generates two sequences, the main sequence, based on the DR iteration, and its corres…

On the Asymptotic Behavior of the Douglas-Rachford and Proximal-Point Algorithms for Convex Optimization

The authors in (Banjac et al., 2019) recently showed that the Douglas-Rachford algorithm provides certificates of infeasibility for a class of convex optimization problems. In particular, they showed that the difference between consecutive iterates generated by the algorithm converges to certificates of primal and dual strong infeasibility. Their result was shown in a finite dimensional Euclidean setting and for a particular structure of the constraint set. In this paper, we extend the result to real Hilbert spaces and a general nonempty closed convex set. Moreover, we show that the proximal-p…

Tight Global Linear Convergence Rate Bounds for Douglas-Rachford Splitting

Recently, several authors have shown local and global convergence rate results for Douglas-Rachford splitting under strong monotonicity, Lipschitz continuity, and cocoercivity assumptions. Most of these focus on the convex optimization setting. In the more general monotone inclusion setting, Lions and Mercier showed a linear convergence rate bound under the assumption that one of the two operators is strongly monotone and Lipschitz continuous. We show that this bound is not tight, meaning that no problem from the considered class converges exactly with that rate. In this paper, we present tigh…

The Degenerate Variable Metric Proximal Point Algorithm and Adaptive Stepsizes for Primal-Dual Douglas-Rachford

In this paper the degenerate preconditioned proximal point algorithm will be combined with the idea of varying preconditioners leading to the degenerate variable metric proximal point algorithm. The weak convergence of the resulting iteration will be proven. From the perspective of the degenerate variable metric proximal point algorithm, a version of the primal-dual Douglas-Rachford method with varying preconditioners will be derived and a proof of its weak convergence which is based on the previous results for the proximal point algorithm, is provided, too. After that, we derive a heuristic o…

在Hadamard流形上高效求解Douglas-Rachford方法及其在Heron问题中的应用

Efficient Douglas-Rachford Methods on Hadamard Manifolds with Applications to the Heron Problems

本文提出在Hadamard流形上高效求解Douglas-Rachford方法，用于解决广义Heron问题。

A convergent relaxation of the Douglas-Rachford algorithm

This paper proposes an algorithm for solving structured optimization problems, which covers both the backward-backward and the Douglas-Rachford algorithms as special cases, and analyzes its convergence. The set of fixed points of the algorithm is characterized in several cases. Convergence criteria of the algorithm in terms of general fixed point operators are established. When applying to nonconvex feasibility including the inconsistent case, we prove local linear convergence results under mild assumptions on regularity of individual sets and of the collection of sets which need not intersect…

伴随算子失配对原始-对偶Douglas-Rachford方法的影响

The Influence of an Adjoint Mismatch on the Primal-Dual Douglas-Rachford Method

本文分析了伴随算子失配下原始-对偶Douglas-Rachford方法的收敛性，给出不动点存在条件、原始解误差上界及强凸情形下保证线性收敛的步长，并提出一种同样收敛的替代方法。