In this paper we study the compressed sensing problem of recovering a sparse signal from a system of underdetermined linear equations when we have prior information about the probability of each entry of the unknown signal being nonzero. In particular, we focus on a model where the entries of the unknown vector fall into two sets, each with a different probability of being nonzero. We propose a weighted $\ell_1$ minimization recovery algorithm and analyze its performance using a Grassman angle approach. We compute explicitly the relationship between the system parameters (the weights, the number of measurements, the size of the two sets, the probabilities of being non-zero) so that an iid random Gaussian measurement matrix along with weighted $\ell_1$ minimization recovers almost all such sparse signals with overwhelming probability as the problem dimension increases. This allows us to compute the optimal weights. We also provide simulations to demonstrate the advantages of the method over conventional $\ell_1$ optimization.
In this paper, we consider the matrix recovery from rank-one projection measurements proposed in [Cai and Zhang, Ann. Statist., 43(2015), 102-138], via nonconvex minimization. We establish a sufficient identifiability condition, which can guarantee the exact recovery of low-rank matrix via Schatten-$p$ minimization $\min_{X}\|X\|_{S_p}^p$ for $0<p<1$ under affine constraint, and stable recovery of low-rank matrix under $\ell_q$ constraint and Dantzig selector constraint. Our condition is also sufficient to guarantee low-rank matrix recovery via least $q$ minimization $\min_{X}\|\mathcal{A}(X)-…
In this paper, we propose a general scale invariant approach for sparse signal recovery via the minimization of the $q$-ratio sparsity. When $1 < q \leq \infty$, both the theoretical analysis based on $q$-ratio constrained minimal singular values (CMSV) and the practical algorithms via nonlinear fractional programming are presented. Numerical experiments are conducted to demonstrate the advantageous performance of the proposed approaches over the state-of-the-art sparse recovery methods.
We study the recovery conditions of weighted mixed $\ell_2/\ell_p\,(0<p\leq 1)$ minimization for block sparse signal reconstruction from compressed measurements when partial block support information is available. We show that the block $p$-restricted isometry property (RIP) can ensure the robust recovery. Moreover, we present the sufficient and necessary condition for the recovery by using weighted block $p$-null space property. The relationship between the block $p$-RIP and the weighted block $p$-null space property has been established. Finally, we illustrate our results with a series of nu…
In this paper, we introduce a weighted $\ell_2/\ell_1$ minimization to recover block sparse signals with arbitrary prior support information. When partial prior support information is available, a sufficient condition based on the high order block RIP is derived to guarantee stable and robust recovery of block sparse signals via the weighted $\ell_2/\ell_1$ minimization. We then show if the accuracy of arbitrary prior block support estimate is at least $50\%$, the sufficient recovery condition by the weighted $\ell_2/\ell_{1}$ minimization is weaker than that by the $\ell_2/\ell_{1}$ minimizat…
Weighted $\ell_1$-minimization has been studied as a technique for the reconstruction of a sparse signal from compressively sampled measurements when prior information about the signal, in the form of a support estimate, is available. In this work, we study the recovery conditions and the associated recovery guarantees of weighted $\ell_1$-minimization when arbitrarily many distinct weights are permitted. For example, such a setup might be used when one has multiple estimates for the support of a signal, and these estimates have varying degrees of accuracy. Our analysis yields an extension to …
In this paper, we consider the use of Total Variation (TV) minimization for compressive imaging; that is, image reconstruction from subsampled measurements. Focusing on two important imaging modalities -- namely, Fourier imaging and structured binary imaging via the Walsh--Hadamard transform -- we derive uniform recovery guarantees asserting stable and robust recovery for arbitrary random sampling strategies. Using this, we then derive a class of theoretically-optimal sampling strategies. For Fourier sampling, we show recovery of an image with approximately $s$-sparse gradient from $m \gtrsim_…
Weighted nuclear norm minimization has been recently recognized as a technique for reconstruction of a low-rank matrix from compressively sampled measurements when some prior information about the column and row subspaces of the matrix is available. In this work, we study the recovery conditions and the associated recovery guarantees of weighted nuclear norm minimization when multiple weights are allowed. This setup might be used when one has access to prior subspaces forming multiple angles with the column and row subspaces of the ground-truth matrix. While existing works in this field use a …
This paper studies the problem of recovering a signal vector and the corrupted noise vector from a collection of corrupted linear measurements through the solution of a l1 minimization, where the sensing matrix is a partial Fourier matrix whose rows are selected randomly and uniformly from rows of a full Fourier matrix. After choosing the parameter in the l1 minimization appropriately, we show that the recovery can be successful even when a constant fraction of the measurements are arbitrarily corrupted, moreover, the proportion of corrupted measurement can grows arbitrarily close to 1, provid…
Compressive sensing relies on the sparse prior imposed on the signal of interest to solve the ill-posed recovery problem in an under-determined linear system. The objective function used to enforce the sparse prior information should be both effective and easily optimizable. Motivated by the entropy concept from information theory, in this paper we propose the generalized Shannon entropy function and Rényi entropy function of the signal as the sparsity promoting regularizers. Both entropy functions are nonconvex, non-separable. Their local minimums only occur on the boundaries of the orthants …
Analyzing Weighted $\ell_1$ Minimization for Sparse Recovery with Nonuniform Sparse Models\footnote{The results of this paper were presented in part at the International Symposium on Information Theory, ISIT 2009}
In this paper we investigate the reconstruction conditions of nuclear norm minimization for low-rank matrix recovery. We obtain sufficient conditions $δ_{tr}<t/(4-t)$ with $0<t<4/3$ to guarantee the robust reconstruction $(z\neq0)$ or exact reconstruction $(z=0)$ of all rank $r$ matrices $X\in\mathbb{R}^{m\times n}$ from $b=\mathcal{A}(X)+z$ via nuclear norm minimization. Furthermore, we not only show that when $t=1$, the upper bound of $δ_r<1/3$ is the same as the result of Cai and Zhang \cite{Cai and Zhang}, but also demonstrate that the gained upper bounds concerning the recovery error are …
In this paper, we bring forward a completely perturbed nonconvex Schatten $p$-minimization to address a model of completely perturbed low-rank matrix recovery. The paper that based on the restricted isometry property generalizes the investigation to a complete perturbation model thinking over not only noise but also perturbation, gives the restricted isometry property condition that guarantees the recovery of low-rank matrix and the corresponding reconstruction error bound. In particular, the analysis of the result reveals that in the case that $p$ decreases $0$ and $a>1$ for the complete pert…
Sparse subspace clustering (SSC) relies on sparse regression for accurate neighbor identification. Inspired by recent progress in compressive sensing, this paper proposes a new sparse regression scheme for SSC via two-step reweighted $\ell_1$-minimization, which also generalizes a two-step $\ell_1$-minimization algorithm introduced by E. J. Candès et al in [The Annals of Statistics, vol. 42, no. 2, pp. 669-699, 2014] without incurring extra algorithmic complexity. To fully exploit the prior information offered by the computed sparse representation vector in the first step, our approach places …
Affine phase retrieval is the problem of recovering signals from the magnitude-only measurements with a priori information. In this paper, we use the $\ell_1$ minimization to exploit the sparsity of signals for affine phase retrieval, showing that $O(k\log(en/k))$ Gaussian random measurements are sufficient to recover all $k$-sparse signals by solving a natural $\ell_1$ minimization program, where $n$ is the dimension of signals. For the case where measurements are corrupted by noises, the reconstruction error bounds are given for both real-valued and complex-valued signals. Our results demons…
In the paper, we proposed the Dantzig selector based on the $\ell_{1}-α\ell_{2}$~$(0< α\leq1)$ minimization for the signal recovery. In the Dantzig selector, the constraint $\|{\bf A}^{\top}({\bf b}-{\bf A}{\bf x})\|_\infty \leq η$ for some small constant $η>0$ means the columns of ${\bf A}$ has very weakly correlated with the error vector ${\bf e}={\bf A}{\bf x}-{\bf b}$. First, recovery guarantees based on the restricted isometry property (RIP) are established for signals. Next, we propose the effective algorithm to solve the proposed Dantzig selector. Last, we illustrate the proposed model …
In the paper, we introduce an unconstrained analysis model based on the $\ell_{1}-α\ell_{2}$ $(0< α\leq1)$ minimization for the signal and image reconstruction. We develop some new technology lemmas for tight frame, and the recovery guarantees based on the restricted isometry property adapted to frames. The effective algorithm is established for the proposed nonconvex analysis model. We illustrate the performance of the proposed model and algorithm for the signal and compressed sensing MRI reconstruction via extensive numerical experiments. And their performance is better than that of the exis…
In this paper we theoretically study exact recovery of sparse vectors from compressed measurements by minimizing a general nonconvex function that can be decomposed into the sum of single variable functions belonging to a class of smooth nonconvex sparsity promoting functions. Null space property (NSP) and restricted isometry property (RIP) are used as key theoretical tools. The notion of \emph{scale function} associated to a sparsity promoting function is introduced to generalize the state-of-the-art analysis technique of the $l_p$ minimization problem. The analysis is used to derive an upper…
This paper considers solving the unconstrained $\ell_q$-norm ($0\leq q<1$) regularized least squares ($\ell_q$-LS) problem for recovering sparse signals in compressive sensing. We propose two highly efficient first-order algorithms via incorporating the proximity operator for nonconvex $\ell_q$-norm functions into the fast iterative shrinkage/thresholding (FISTA) and the alternative direction method of multipliers (ADMM) frameworks, respectively. Furthermore, in solving the nonconvex $\ell_q$-LS problem, a sequential minimization strategy is adopted in the new algorithms to gain better global …
This paper investigates the problem of signal estimation from undersampled noisy sub-Gaussian measurements under the assumption of a cosparse model. Based on generalized notions of sparsity, we derive novel recovery guarantees for the $\ell^1$-analysis basis pursuit, enabling accurate predictions of its sample complexity. The corresponding bounds on the number of required measurements do explicitly depend on the Gram matrix of the analysis operator and therefore particularly account for its mutual coherence structure. Our findings defy conventional wisdom which promotes the sparsity of analysi…
Low-rank matrix recovery has found many applications in science and engineering such as machine learning, signal processing, collaborative filtering, system identification, and Euclidean embedding. But the low-rank matrix recovery problem is an NP hard problem and thus challenging. A commonly used heuristic approach is the nuclear norm minimization. In [12,14,15], the authors established the necessary and sufficient null space conditions for nuclear norm minimization to recover every possible low-rank matrix with rank at most r (the strong null space condition). In addition, in [12], Oymak et …
We study the recovery results of $\ell_p$-constrained compressive sensing (CS) with $p\geq 1$ via robust width property and determine conditions on the number of measurements for standard Gaussian matrices under which the property holds with high probability. Our paper extends the existing results in Cahill and Mixon (2014) from $\ell_2$-constrained CS to $\ell_p$-constrained case with $p\geq 1$ and complements the recovery analysis for robust CS with $\ell_p$ loss function.
In this paper we study the recovery conditions of weighted $l_{1}$ minimization for signal reconstruction from incomplete linear measurements when partial prior support information is available. We obtain that a high order RIP condition can guarantee stable and robust recovery of signals in bounded $l_{2}$ and Dantzig selector noise settings. Meanwhile, we not only prove that the sufficient recovery condition of weighted $l_{1}$ minimization method is weaker than that of standard $l_{1}$ minimization method, but also prove that weighted $l_{1}$ minimization method provides better upper bounds …
As one of the most plausible convex optimization methods for sparse data reconstruction, $\ell_1$-minimization plays a fundamental role in the development of sparse optimization theory. The stability of this method has been addressed in the literature under various assumptions such as restricted isometry property (RIP), null space property (NSP), and mutual coherence. In this paper, we propose a unified means to develop the so-called weak stability theory for $\ell_1$-minimization methods under the condition called weak range space property of a transposed design matrix, which turns out to be …
We study the recovery conditions of weighted $\ell_1$ minimization for real-valued signal reconstruction from phaseless compressive sensing measurements when partial support information is available. A strong restricted isometry property condition is provided to ensure the stable recovery. Moreover, we present the weighted null space property as the sufficient and necessary condition for the success of $k$-sparse phaseless recovery via weighted $\ell_1$ minimization. Numerical experiments are conducted to illustrate our results.
The goal of phaseless compressed sensing is to recover an unknown sparse or approximately sparse signal from the magnitude of its measurements. However, it does not take advantage of any support information of the original signal. Therefore, our main contribution in this paper is to extend the theoretical framework for phaseless compressed sensing to incorporate with prior knowledge of the support structure of the signal. Specifically, we investigate two conditions that guarantee stable recovery of a weighted $k$-sparse signal via weighted l1 minimization without any phase information. We firs…
In this paper we study the $\ell_p$-analysis optimization ($0<p\leq1$) problem for cosparse signal recovery. We establish a bound for recovery error via the restricted $p$-isometry property over any subspace. We further prove that the nonconvex $\ell_q$-analysis optimization can do recovery with a lower sample complexity and in a wider range of cosparsity than its convex counterpart. In addition, we develop an iteratively reweighted method to solve the optimization problem under a variational framework. Empirical results of preliminary computational experiments illustrate that the nonconvex me…
The definition of the graph Fourier transform is a fundamental issue in graph signal processing. Conventional graph Fourier transform is defined through the eigenvectors of the graph Laplacian matrix, which minimize the $\ell_2$ norm signal variation. However, the computation of Laplacian eigenvectors is expensive when the graph is large. In this paper, we propose an alternative definition of graph Fourier transform based on the $\ell_1$ norm variation minimization. We obtain a necessary condition satisfied by the $\ell_1$ Fourier basis, and provide a fast greedy algorithm to approximate the $…
In compressed sensing sparse solutions are usually obtained by solving an $\ell^1$-minimization problem. Furthermore, the sparsity of the signal does need not be directly given. In fact, it is sufficient to have a signal that is sparse after an application of a suitable transform. In this paper we consider the stability of solutions obtained from $\ell^p$-minimization for arbitrary $0<p \leq1$. Further we suppose that the signals are sparse with respect to general redundant transforms associated to not necessarily tight frames. Since we are considering general frames the role of the dual frame…
We study extensions of compressive sensing and low rank matrix recovery to the recovery of low rank tensors from incomplete linear information. While the reconstruction of low rank matrices via nuclear norm minimization is rather well-understand by now, almost no theory is available for the extension to higher order tensors due to various theoretical and computational difficulties arising for tensor decompositions. In fact, nuclear norm minimization for matrix recovery is a tractable convex relaxation approach, but the extension of the nuclear norm to tensors is in general NP-hard to compute. …
The joint sparse recovery problem is a generalization of the single measurement vector problem which is widely studied in Compressed Sensing and it aims to recovery a set of jointly sparse vectors. i.e. have nonzero entries concentrated at common location. Meanwhile l_p-minimization subject to matrices is widely used in a large number of algorithms designed for this problem. Therefore the main contribution in this paper is two theoretical results about this technique. The first one is to prove that in every multiple systems of linear equation, there exists a constant p* such that the original …
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；KU Eichstätt-Ingolstadt（艾希施泰特-英戈尔施塔特天主教大学）
In this paper, we study the problem of recovering two unknown signals from their convolution, which is commonly referred to as blind deconvolution. Reformulation of blind deconvolution as a low-rank recovery problem has led to multiple theoretical recovery guarantees in the past decade due to the success of the nuclear norm minimization heuristic. In particular, in the absence of noise, exact recovery has been established for sufficiently incoherent signals contained in lower-dimensional subspaces. However, if the convolution is corrupted by additive bounded noise, the stability of the recover…
