A note on alternating minimization algorithms: Bregman frame
National University of Defense Technology（国防科技大学）；The State Key Laboratory for High Performance Computation, National University of Defense Technology（高性能计算国家重点实验室（国防科技大学））
In this paper, we propose a Bregman frame for several classical alternating minimization algorithms. In the frame, these algorithms have uniform mathematical formulation. We also present convergence analysis for the frame algorithm. Under the Kurdyka-Lojasiewicz property, stronger convergence is obtained.
We introduce and analyze BPALM and A-BPALM, two multi-block proximal alternating linearized minimization algorithms using Bregman distances for solving structured nonconvex problems. The objective function is the sum of a multi-block relatively smooth function (i.e., relatively smooth by fixing all the blocks except one) and block separable (nonsmooth) nonconvex functions. It turns out that the sequences generated by our algorithms are subsequentially convergent to critical points of the objective function, while they are globally convergent under KL inequality assumption. Further, the rate of…
In this paper, we consider a class of nonsmooth nonconvex optimization problems whose objective is the sum of a block relative smooth function and a proper and lower semicontinuous block separable function. Although the analysis of block proximal gradient (BPG) methods for the class of block $L$-smooth functions have been successfully extended to Bregman BPG methods that deal with the class of block relative smooth functions, accelerated Bregman BPG methods are scarce and challenging to design. Taking our inspiration from Nesterov-type acceleration and the majorization-minimization scheme, we …
In this paper, we compute the stationary states of the multicomponent phase-field crystal model by formulating it as a block constrained minimization problem. The original infinite-dimensional non-convex minimization problem is approximated by a finite-dimensional constrained non-convex minimization problem after an appropriate spatial discretization. To efficiently solve the above optimization problem, we propose a so-called adaptive block Bregman proximal gradient (AB-BPG) algorithm that fully exploits the problem's block structure. The proposed method updates each order parameter alternativ…
Focus of this work is solving a non-smooth constraint minimization problem by a primal-dual splitting algorithm involving proximity operators. The problem is penalized by the Bregman divergence associated with the non-smooth total variation (TV) functional. We analyse two aspects: Firstly, the convergence of the regularized solution of the minimization problem to the minimum norm solution. Second, the convergence of the iteratively regularized minimizer to the minimum norm solution by a primal-dual algorithm. For both aspects, we use the assumption of a variational source condition (VSC). This…
University of Tübingen（图宾根大学）；Normandie Univ（诺曼底大学）；ENSICAEN（卡昂高等工程师学院）；CNRS（法国国家科学研究中心）；GREYC（卡昂格雷克计算机、图像与智能系统实验室）
Lipschitz continuity of the gradient mapping of a continuously differentiable function plays a crucial role in designing various optimization algorithms. However, many functions arising in practical applications such as low rank matrix factorization or deep neural network problems do not have a Lipschitz continuous gradient. This led to the development of a generalized notion known as the $L$-smad property, which is based on generalized proximity measures called Bregman distances. However, the $L$-smad property cannot handle nonsmooth functions, for example, simple nonsmooth functions like $\a…
We develop computational methods for approximating the solution of a linear multi-term matrix equation in low rank. We follow an alternating minimization framework, where the solution is represented as a product of two matrices, and approximations to each matrix are sought by solving certain minimization problems repeatedly. The solution methods we present are based on a rank-adaptive variant of alternating energy minimization methods that builds an approximation iteratively by successively computing a rank-one solution component at each step. We also develop efficient procedures to improve th…
In this work we propose a nonconvex two-stage \underline{s}tochastic \underline{a}lternating \underline{m}inimizing (SAM) method for sparse phase retrieval. The proposed algorithm is guaranteed to have an exact recovery from $O(s\log n)$ samples if provided the initial guess is in a local neighbour of the ground truth. Thus, the proposed algorithm is two-stage, first we estimate a desired initial guess (e.g. via a spectral method), and then we introduce a randomized alternating minimization strategy for local refinement. Also, the hard-thresholding pursuit algorithm is employed to solve the sp…
For some typical and widely used non-convex half-quadratic regularization models and the Ambrosio-Tortorelli approximate Mumford-Shah model, based on the Kurdyka-Łojasiewicz analysis and the recent nonconvex proximal algorithms, we developed an efficient preconditioned framework aiming at the linear subproblems that appeared in the nonlinear alternating minimization procedure. Solving large-scale linear subproblems is always important and challenging for lots of alternating minimization algorithms. By cooperating the efficient and classical preconditioned iterations into the nonlinear and nonc…
While Computerized Tomography (CT) images can help detect disease such as Covid-19, regular CT machines are large and expensive. Cheaper and more portable machines suffer from errors in geometry acquisition that downgrades CT image quality. The errors in geometry can be represented with parameters in the mathematical model for image reconstruction. To obtain a good image, we formulate a nonlinear least squares problem that simultaneously reconstructs the image and corrects for errors in the geometry parameters. We develop an accelerated alternating minimization scheme to reconstruct the image …
Western Norway University of Applied Sciences（挪威西部应用科技大学）；University of Bergen（卑尔根大学）
The paper presents a fully coupled TV-Stokes model, and propose an algorithm based on alternating minimization of the objective functional whose first iteration is exactly the modified TV-Stokes model proposed earlier. The model is a generalization of the second order Total Generalized Variation model. A convergence analysis is given.
We study alternating minimization for matrix completion in the simplest possible setting: completing a rank-one matrix from a revealed subset of the entries. We bound the asymptotic convergence rate by the variational characterization of eigenvalues of a reversible consensus problem. This leads to a polynomial upper bound on the asymptotic rate in terms of number of nodes as well as the largest degree of the graph of revealed entries.
Affine sum-of-ranks minimization (ASRM) generalizes the affine rank minimization (ARM) problem from matrices to tensors. Here, the interest lies in the ranks of a family $\mathcal{K}$ of different matricizations. Transferring our priorly discussed results on asymptotic log-det rank minimization, we show that iteratively reweighted least squares with weight strength $p = 0$ remains a, theoretically and practically, particularly viable method denoted as $\mathrm{IRLS}$-$0\mathcal{K}$. As in the matrix case, we prove global convergence of asymptotic minimizers of the log-det sum-of-ranks function…
The goal of this paper is to develop a novel numerical method for efficient multiplicative noise removal. The nonlocal self-similarity of natural images implies that the matrices formed by their nonlocal similar patches are low-rank. By exploiting this low-rank prior with application to multiplicative noise removal, we propose a nonlocal low-rank model for this task and develop a proximal alternating reweighted minimization (PARM) algorithm to solve the optimization problem resulting from the model. Specifically, we utilize a generalized nonconvex surrogate of the rank function to regularize t…
The affine rank minimization (ARM) problem is well known for both its applications and the fact that it is NP-hard. One of the most successful approaches, yet arguably underrepresented, is iteratively reweighted least squares (IRLS), more specifically $\mathrm{IRLS}$-$0$. Despite comprehensive empirical evidence that it overall outperforms nuclear norm minimization and related methods, it is still not understood to a satisfying degree. In particular, the significance of a slow decrease of the therein appearing regularization parameter denoted $γ$ poses interesting questions. While commonly equ…
Alternating minimization algorithms for graph regularized tensor completion
Huawei European Research Institute（华为欧洲研究院）；LISN, INRIA, Université Paris-Saclay（法国国家信息与自动化研究所，巴黎萨克雷大学）；LSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院科学与工程计算国家重点实验室）；Department of INMA, Université catholique de Louvain（鲁汶天主教大学）
