The Alternating Direction Method of Multipliers (ADMM) is widely used for linearly constrained convex problems. It is proven to have an $o(1/\sqrt{K})$ nonergodic convergence rate and a faster $O(1/K)$ ergodic rate after ergodic averaging, which may destroy the sparsity and low-rankness in sparse and low-rank learning, where $K$ is the number of iterations. In this paper, we modify the accelerated ADMM proposed in [Y. Ouyang, Y. Chen, G. Lan, and E. Pasiliao, An Accelerated Linearized Alternating Direction Method of Multipliers, SIAM J. on Imaging Sciences, 2015, 1588-1623] and give an $O(1/K)$ nonergodic convergence rate analysis, which satisfies $|F(x^K)-F(x^*)|\leq O(1/K)$, $\|Ax^K-b\|\leq O(1/K)$ and $x^K$ has a more favorable sparseness and low-rankness than the ergodic result. As far as we know, this is the first $O(1/K)$ nonergodic convergent ADMM type method for general linearly constrained convex problems. Moreover, we show that the lower complexity bound of ADMM type methods for the separable linearly constrained nonsmooth convex problems is $O(1/K)$, which means that our method is optimal.
In this paper, we consider a prototypical convex optimization problem with multi-block variables and separable structures. By adding the Logarithmic Quadratic Proximal (LQP) regularizer with suitable proximal parameter to each of the first grouped subproblems, we develop a partial LQP-based Alternating Direction Method of Multipliers (ADMM-LQP). The dual variable is updated twice with relatively larger stepsizes than the classical region $(0,\frac{1+\sqrt{5}}{2})$. Using a prediction-correction approach to analyze properties of the iterates generated by ADMM-LQP, we establish its global conver…
A method for quasistatic cohesive fracture is introduced that uses an alternating direction method of multipliers (ADMM) to implement an energy approach to cohesive fracture. The ADMM algorithm minimizes a non-smooth, non-convex potential functional at each strain increment to predict the evolution of a cohesive-elastic system. The optimization problem bypasses the explicit stress criterion of force-based (Newtonian) methods, which interferes with Newton iterations impeding convergence. The model is extended with an extrapolation method that significantly reduces the computation time of the se…
We propose two numerical algorithms in the fully nonconvex setting for the minimization of the sum of a smooth function and the composition of a nonsmooth function with a linear operator. The iterative schemes are formulated in the spirit of the proximal alternating direction method of multipliers and its linearized variant, respectively. The proximal terms are introduced via variable metrics, a fact which allows us to derive new proximal splitting algorithms for nonconvex structured optimization problems, as particular instances of the general schemes. Under mild conditions on the sequence of…
Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems
Department of Mathematics, National University of Defense Technology（国防科技大学数学系）；College of Computer, National University of Defense Technology（国防科技大学计算机学院）；The State Key Laboratory for High Performance Computation, National University of Defense Technology（国防科技大学高性能计算国家重点实验室）；Hunan Key Laboratory for Computation and Simulation in Science and Engineering, School of Mathematics and Computational Science, Xiangtan University（湖南计算与模拟科学工程重点实验室，湘潭大学数学与计算科学学院）
We study several versions of the alternating direction method of multipliers (ADMM) for solving the convex problem of finding the distance between two ellipsoids and the nonconvex problem of finding the distance between the boundaries of two ellipsoids. In the convex case we present the ADMM with and without automatic penalty updates and demonstrate via numerical experiments on problems of various dimensions that our methods significantly outperform all other existing methods for finding the distance between ellipsoids. In the nonconvex case we propose a heuristic rule for updating the penalty…
A Dual Symmetric Gauss-Seidel Alternating Direction Method of Multipliers for Hyperspectral Sparse Unmixing
School of Information Science and technology, and the Provincial Key Lab of Information Coding and Trans- mission, Southwest Jiaotong University（信息科学与技术学院，信息编码与传输省重点实验室，西南交通大学）；School of Mathematics, Southwest Jiaotong University（数学学院，西南交通大学）；School of Computer and Computing Science, Zhejiang University City College（计算机与计算科学学院，浙江大学城市学院）
