Iteratively re-weighted least squares minimization for sparse recovery
Princeton University（普林斯顿大学）；University of South Carolina（南卡罗来纳大学）；Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences（奥地利科学院约翰·拉登计算与应用数学研究所）；New York University, Courant Institute of Mathematical Sciences（纽约大学柯朗数学科学研究所）
Ingrid Daubechies, Ronald DeVore, Massimo Fornasier, C. Sinan Gunturk
We analyze an Iteratively Re-weighted Least Squares (IRLS) algorithm for promoting l1-minimization in sparse and compressible vector recovery. We prove its convergence and we estimate its local rate. We show how the algorithm can be modified in order to promote lt-minimization for t<1, and how this modification produces superlinear rates of convergence.
In this paper, we theoretically investigate the low-rank matrix recovery problem in the context of the unconstrained regularized nuclear norm minimization (RNNM) framework. Our theoretical findings show that, the RNNM method is able to provide a robust recovery of any matrix $X$ (not necessary to be exactly low-rank) from its few noisy measurements $\textbf{b}=\mathcal{A}(X)+\textbf{n}$ with a bounded constraint $\|\textbf{n}\|_{2}\leqε$, provided that the $tk$-order restricted isometry constant (RIC) of $\mathcal{A}$ satisfies a certain constraint related to $t>0$. Specifically, the obtained …
Affine sum-of-ranks minimization (ASRM) generalizes the affine rank minimization (ARM) problem from matrices to tensors. Here, the interest lies in the ranks of a family $\mathcal{K}$ of different matricizations. Transferring our priorly discussed results on asymptotic log-det rank minimization, we show that iteratively reweighted least squares with weight strength $p = 0$ remains a, theoretically and practically, particularly viable method denoted as $\mathrm{IRLS}$-$0\mathcal{K}$. As in the matrix case, we prove global convergence of asymptotic minimizers of the log-det sum-of-ranks function…
Low-rank matrix recovery via iteratively reweighted least squares minimization
Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences（奥地利科学院约翰·拉东计算与应用数学研究所）；Hausdorff Center for Mathematics & Institute for Numerical Simulation, University of Bonn（波恩大学霍夫多夫数学中心与数值模拟研究所）；Mathematics Department, Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所数学系）
The affine rank minimization (ARM) problem is well known for both its applications and the fact that it is NP-hard. One of the most successful approaches, yet arguably underrepresented, is iteratively reweighted least squares (IRLS), more specifically $\mathrm{IRLS}$-$0$. Despite comprehensive empirical evidence that it overall outperforms nuclear norm minimization and related methods, it is still not understood to a satisfying degree. In particular, the significance of a slow decrease of the therein appearing regularization parameter denoted $γ$ poses interesting questions. While commonly equ…
In this paper, we consider the $L_1/L_2 $ minimization for sparse recovery and study its relationship with the $L_1$-$ αL_2 $ model. Based on this relationship, we propose three numerical algorithms to minimize this ratio model, two of which work as adaptive schemes and greatly reduce the computation time. Focusing on two adaptive schemes, we discuss their connection to existing approaches and analyze their convergence. The experimental results demonstrate the proposed approaches are comparable to the state-of-the-art methods in sparse recovery and work particularly well when the ground-truth …
Signals and images with discontinuities appear in many problems in such diverse areas as biology, medicine, mechanics, and electrical engineering. The concrete data are often discrete, indirect and noisy measurements of some quantities describing the signal under consideration. A frequent task is to find the segments of the signal or image which corresponds to finding the discontinuities or jumps in the data. Methods based on minimizing the piecewise constant Mumford-Shah functional -- whose discretized version is known as Potts functional -- are advantageous in this scenario, in particular, i…
Sparse recovery is widely applied in many fields, since many signals or vectors can be sparsely represented under some frames or dictionaries. Most of fast algorithms at present are based on solving $l^0$ or $l^1$ minimization problems and they are efficient in sparse recovery. However, compared with the practical results, the theoretical sufficient conditions on the sparsity of the signal for $l^0$ or $l^1$ minimization problems and algorithms are too strict. \par In many applications, there are signals with certain structures as piecewise sparsity. Piecewise sparsity means that the sparse si…
We recover jump-sparse and sparse signals from blurred incomplete data corrupted by (possibly non-Gaussian) noise using inverse Potts energy functionals. We obtain analytical results (existence of minimizers, complexity) on inverse Potts functionals and provide relations to sparsity problems. We then propose a new optimization method for these functionals which is based on dynamic programming and the alternating direction method of multipliers (ADMM). A series of experiments shows that the proposed method yields very satisfactory jump-sparse and sparse reconstructions, respectively. We highlig…
Coupled Tensor-Matrix Recovery via Proximal Alternating Linearized Minimization, with an Application to Workforce Skill and Small-Business Health Estimation
As a new color image representation tool, quaternion has achieved excellent results in color image processing problems. In this paper, we propose a novel low-rank quaternion matrix completion algorithm to recover missing data of color image. Motivated by two kinds of low-rank approximation approaches (low-rank decomposition and nuclear norm minimization) in traditional matrix-based methods, we combine the two approaches in our quaternion matrix-based model. Furthermore, the nuclear norm of the quaternion matrix is replaced by the sum of Frobenius norm of its two low-rank factor quaternion matr…
Johns Hopkins University（约翰斯·霍普金斯大学）；Technical University of Munich（慕尼黑工业大学）；KU Eichstätt–Ingolstadt（埃希施泰特-英戈尔施塔特天主教大学）
The recovery of sparse data is at the core of many applications in machine learning and signal processing. While such problems can be tackled using $\ell_1$-regularization as in the LASSO estimator and in the Basis Pursuit approach, specialized algorithms are typically required to solve the corresponding high-dimensional non-smooth optimization for large instances. Iteratively Reweighted Least Squares (IRLS) is a widely used algorithm for this purpose due its excellent numerical performance. However, while existing theory is able to guarantee convergence of this algorithm to the minimizer, it …
