We consider compressed sensing of block-sparse signals, i.e., sparse signals that have nonzero coefficients occuring in clusters. Based on an uncertainty relation for block-sparse signals, we define a block-coherence measure and we show that a block-version of the orthogonal matching pursuit algorithm recovers block k-sparse signals in no more than k steps if the block-coherence is sufficiently small. The same condition on block-sparsity is shown to guarantee successful recovery through a mixed l2/l1 optimization approach. The significance of the results lies in the fact that making explicit use of block-sparsity can yield better reconstruction properties than treating the signal as being sparse in the conventional sense thereby ignoring the additional structure in the problem.
To solve the problem of joint sparsity pattern recovery in a decen-tralized network, we propose an algorithm named decentralized and collaborative subspace pursuit (DCSP). The basic idea of DCSP is to embed collaboration among nodes and fusion strategy into each iteration of the standard subspace pursuit (SP) algorithm. In DCSP, each node collaborates with several of its neighbors by sharing high-dimensional coefficient estimates and communicates with other remote nodes by exchanging low-dimensional support set estimates. Experimental evaluations show that, compared with several existing algor…
In this paper, we propose a general scale invariant approach for sparse signal recovery via the minimization of the $q$-ratio sparsity. When $1 < q \leq \infty$, both the theoretical analysis based on $q$-ratio constrained minimal singular values (CMSV) and the practical algorithms via nonlinear fractional programming are presented. Numerical experiments are conducted to demonstrate the advantageous performance of the proposed approaches over the state-of-the-art sparse recovery methods.
We address the problem of sparse recovery using greedy compressed sensing recovery algorithms, without explicit knowledge of the sparsity. Estimating the sparsity order is a crucial problem in many practical scenarios, e.g., wireless communications, where exact value of the sparsity order of the unknown channel may be unavailable a priori. In this paper we have proposed a new greedy algorithm, referred to as the Multiple Choice Hard Thresholding Pursuit (MCHTP), which modifies the popular hard thresholding pursuit (HTP) suitably to iteratively recover the unknown sparse vector along with the s…
In compressive sensing, sparse signals are recovered from underdetermined noisy linear observations. One of the interesting problems which attracted a lot of attention in recent times is the support recovery or sparsity pattern recovery problem. The aim is to identify the non-zero elements in the original sparse signal. In this article we consider the sparsity pattern recovery problem under a probabilistic signal model where the sparse support follows a Bernoulli distribution and the signal restricted to this support follows a Gaussian distribution. We show that the energy in the original sign…
Consider the $n$-dimensional vector $y=X\be+\e$, where $\be \in \R^p$ has only $k$ nonzero entries and $\e \in \R^n$ is a Gaussian noise. This can be viewed as a linear system with sparsity constraints, corrupted by noise. We find a non-asymptotic upper bound on the probability that the optimal decoder for $β$ declares a wrong sparsity pattern, given any generic perturbation matrix $X$. In the case when $X$ is randomly drawn from a Gaussian ensemble, we obtain asymptotically sharp sufficient conditions for exact recovery, which agree with the known necessary conditions previously established.
In this paper, we consider the problem of joint sparsity pattern recovery in a distributed sensor network. The sparse multiple measurement vector signals (MMVs) observed by all the nodes are assumed to have a common (but unknown) sparsity pattern. To accurately recover the common sparsity pattern in a decentralized manner with a low communication overhead of the network, we develop an algorithm named decentralized and collaborative subspace pursuit (DCSP). In DCSP, each node is required to perform three kinds of operations per iteration: 1) estimate the local sparsity pattern by finding the su…
A noisy underdetermined system of linear equations is considered in which a sparse vector (a vector with a few nonzero elements) is subject to measurement. The measurement matrix elements are drawn from a Gaussian distribution. We study the information-theoretic constraints on exact support recovery of a sparse vector from the measurement vector and matrix. We compute a tight, sufficient condition that is applied to ergodic wide-sense stationary sparse vectors. We compare our results with the existing bounds and recovery conditions. Finally, we extend our results to approximately sparse signal…
We study the problem of recovering the sparsity pattern of block-sparse signals from noise-corrupted measurements. A simple, efficient recovery method, namely, a block-version of the orthogonal matching pursuit (OMP) method, is considered in this paper and its behavior for recovering the block-sparsity pattern is analyzed. We provide sufficient conditions under which the block-version of the OMP can successfully recover the block-sparse representations in the presence of noise. Our analysis reveals that exploiting block-sparsity can improve the recovery ability and lead to a guaranteed recover…
This paper considers signal recovery in the framework of cumulative coherence. First, we show that the Lasso estimator and the Dantzig selector exhibit similar behavior under the cumulative coherence. Then we estimate the approximation equivalence between the Lasso and the Dantzig selector by calculating prediction loss difference under the condition of cumulative coherence. And we also prove that the cumulative coherence implies the restricted eigenvalue condition. Last, we illustrate the advantages of cumulative coherence condition for three class matrices, in terms of the recovery performan…
In oversampled adaptive sensing (OAS), noisy measurements are collected in multiple subframes. The sensing basis in each subframe is adapted according to some posterior information exploited from previous measurements. The framework is shown to significantly outperform the classic non-adaptive compressive sensing approach. This paper extends the notion of OAS to signals with structured sparsity. We develop a low-complexity OAS algorithm based on structured orthogonal sensing. Our investigations depict that the proposed algorithm outperforms the conventional non-adaptive compressive sensing fra…
Recovery of the sparsity pattern (or support) of an unknown sparse vector from a small number of noisy linear measurements is an important problem in compressed sensing. In this paper, the high-dimensional setting is considered. It is shown that if the measurement rate and per-sample signal-to-noise ratio (SNR) are finite constants independent of the length of the vector, then the optimal sparsity pattern estimate will have a constant fraction of errors. Lower bounds on the measurement rate needed to attain a desired fraction of errors are given in terms of the SNR and various key parameters o…
We study the problem of jointly sparse support recovery with 1-bit compressive measurements in a sensor network. Sensors are assumed to observe sparse signals having the same but unknown sparse support. Each sensor quantizes its measurement vector element-wise to 1-bit and transmits the quantized observations to a fusion center. We develop a computationally tractable support recovery algorithm which minimizes a cost function defined in terms of the likelihood function and the $l_{1,\infty}$ norm. We observe that even with noisy 1-bit measurements, jointly sparse support can be recovered accura…
Compressed Sensing (CS) is an effective approach to reduce the required number of samples for reconstructing a sparse signal in an a priori basis, but may suffer severely from the issue of basis mismatch. In this paper we study the problem of simultaneously recovering multiple spectrally-sparse signals that are supported on the same frequencies lying arbitrarily on the unit circle. We propose an atomic norm minimization problem, which can be regarded as a continuous counterpart of the discrete CS formulation and be solved efficiently via semidefinite programming. Through numerical experiments,…
Modern mobile terminals produce massive small data packets. For these short-length packets, it is inefficient to follow the current multiple access schemes to allocate transmission resources due to heavy signaling overhead. We propose a non-orthogonal many-access scheme that is well suited for the future communication systems equipped with many receive antennas. The system is modeled as having a block-sparsity pattern with unknown sparsity level (i.e., unknown number of transmitted messages). Block precoding is employed at each single-antenna transmitter to enable the simultaneous transmission…
We address the problem of joint sparsity pattern recovery based on low dimensional multiple measurement vectors (MMVs) in resource constrained distributed networks. We assume that distributed nodes observe sparse signals which share the same sparsity pattern and each node obtains measurements via a low dimensional linear operator. When the measurements are collected at distributed nodes in a communication network, it is often required that joint sparse recovery be performed under inherent resource constraints such as communication bandwidth and transmit/processing power. We present two approac…
Recovery of the sparsity pattern (or support) of an unknown sparse vector from a limited number of noisy linear measurements is an important problem in compressed sensing. In the high-dimensional setting, it is known that recovery with a vanishing fraction of errors is impossible if the measurement rate and the per-sample signal-to-noise ratio (SNR) are finite constants, independent of the vector length. In this paper, it is shown that recovery with an arbitrarily small but constant fraction of errors is, however, possible, and that in some cases computationally simple estimators are near-opti…
Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to understand individually sparsity or entropy from specific research interests. This paper proposes a mathematical formalism, a joint axiomatic characterization, which contributes to comprehend (the beauty of) sparsity and entropy. The paper gathers and introduces inherent and first principles criteria as axioms and attributes that jointly characterize sparsity a…
The algorithms based on the technique of optimal $k$-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal $k$-thresholding model, and thus the computational cos…
In this paper, we propose a method for block sparse signal recovery that minimizes the block $q$-ratio sparsity $\left(\lVert z\rVert_{2,1}/\lVert z\rVert_{2,q}\right)^{\frac{q}{q-1}}$ with $q\in[0,\infty]$. For the case of $1<q\leq\infty$, we present the theoretical analyses and the computing algorithms for both cases of the $\ell_2$-bounded and $\ell_{2,\infty}$-bounded noises. The corresponding unconstrained model is also investigated. Its superior performance in block sparse signal reconstruction is demonstrated by numerical experiments.
In this article, we discuss a novel greedy algorithm for the recovery of compressive sampled signals under noisy conditions. Most of the greedy recovery algorithms proposed in the literature require sparsity of the signal to be known or they estimate sparsity, for a known representation basis, from the number of measurements. These algorithms recover signals when noise level is significantly low. We propose Entropy minimization based Matching Pursuit (EMP) which has the capability to reject noise even when noise level is comparable to that of signal level. The proposed algorithm can cater to c…
This paper investigates the problem of signal estimation from undersampled noisy sub-Gaussian measurements under the assumption of a cosparse model. Based on generalized notions of sparsity, we derive novel recovery guarantees for the $\ell^1$-analysis basis pursuit, enabling accurate predictions of its sample complexity. The corresponding bounds on the number of required measurements do explicitly depend on the Gram matrix of the analysis operator and therefore particularly account for its mutual coherence structure. Our findings defy conventional wisdom which promotes the sparsity of analysi…
Sparse signal recoveries from multiple measurement vectors (MMV) with joint sparsity property have many applications in signal, image, and video processing. The problem becomes much more involved when snapshots of the signal matrix are temporally correlated. With signal's temporal correlation in mind, we provide a framework of iterative MMV algorithms based on thresholding, functional feedback and null space tuning. Convergence analysis for exact recovery is established. Unlike most of iterative greedy algorithms that select indices in a measurement/solution space, we determine indices based o…
Sparse Bayesian learning (SBL) is a popular approach to sparse signal recovery in compressed sensing (CS). In SBL, the signal sparsity information is exploited by assuming a sparsity-inducing prior for the signal that is then estimated using Bayesian inference. In this paper, a new sparsity-inducing prior is introduced and efficient algorithms are developed for signal recovery. The main algorithm is shown to produce a sparser solution than existing SBL methods while preserving their desirable properties. Numerical simulations with one-dimensional synthetic signals and two-dimensional images ve…
In this paper we theoretically study exact recovery of sparse vectors from compressed measurements by minimizing a general nonconvex function that can be decomposed into the sum of single variable functions belonging to a class of smooth nonconvex sparsity promoting functions. Null space property (NSP) and restricted isometry property (RIP) are used as key theoretical tools. The notion of \emph{scale function} associated to a sparsity promoting function is introduced to generalize the state-of-the-art analysis technique of the $l_p$ minimization problem. The analysis is used to derive an upper…
The problem of recovering the sparsity pattern of a fixed but unknown vector $β^* \in \real^p based on a set of $n$ noisy observations arises in a variety of settings, including subset selection in regression, graphical model selection, signal denoising, compressive sensing, and constructive approximation. Of interest are conditions on the model dimension $p$, the sparsity index $s$ (number of non-zero entries in $β^*$), and the number of observations $n$ that are necessary and/or sufficient to ensure asymptotically perfect recovery of the sparsity pattern. This paper focuses on the informatio…
Known sparsity thresholds for basis pursuit to deliver the maximally sparse solution of the compressed sensing recovery problem typically depend on the dictionary's coherence. While the coherence is easy to compute, it can lead to rather pessimistic thresholds as it captures only limited information about the dictionary. In this paper, we show that viewing the dictionary as the concatenation of two general sub-dictionaries leads to provably better sparsity thresholds--that are explicit in the coherence parameters of the dictionary and of the individual sub-dictionaries. Equivalently, our resul…
We propose extended coherence-based conditions for exact sparse support recovery using orthogonal matching pursuit (OMP) and orthogonal least squares (OLS). Unlike standard uniform guarantees, we embed some information about the decay of the sparse vector coefficients in our conditions. As a result, the standard condition $μ<1/(2k-1)$ (where $μ$ denotes the mutual coherence and $k$ the sparsity level) can be weakened as soon as the non-zero coefficients obey some decay, both in the noiseless and the bounded-noise scenarios. Furthermore, the resulting condition is approaching $μ<1/k$ for strong…
Common ISAR radar images and signals can be reconstructed from much fewer samples than the sampling theorem requires since they are usually sparse. Unavailable randomly positioned samples can result from heavily corrupted parts of the signal. Since these samples can be omitted and declared as unavailable, the application of the compressive sensing methods in the recovery of heavily corrupted signal and radar images is possible. A\ simple direct method for the recovery of unavailable signal samples and the calculation of the restored ISAR image is reviewed. An analysis of the noise influence is…
The problem of detecting the sparsity pattern of a k-sparse vector in R^n from m random noisy measurements is of interest in many areas such as system identification, denoising, pattern recognition, and compressed sensing. This paper addresses the scaling of the number of measurements m, with signal dimension n and sparsity-level nonzeros k, for asymptotically-reliable detection. We show a necessary condition for perfect recovery at any given SNR for all algorithms, regardless of complexity, is m = Omega(k log(n-k)) measurements. Conversely, it is shown that this scaling of Omega(k log(n-k)) m…
The most frequently used condition for sampling matrices employed in compressive sampling is the restricted isometry (RIP) property of the matrix when restricted to sparse signals. At the same time, imposing this condition makes it difficult to find explicit matrices that support recovery of signals from sketches of the optimal (smallest possible)dimension. A number of attempts have been made to relax or replace the RIP property in sparse recovery algorithms. We focus on the relaxation under which the near-isometry property holds for most rather than for all submatrices of the sampling matrix,…
We describe a probabilistic, {\it sublinear} runtime, measurement-optimal system for model-based sparse recovery problems through dimensionality reducing, {\em dense} random matrices. Specifically, we obtain a linear sketch $u\in \R^M$ of a vector $\bestsignal\in \R^N$ in high-dimensions through a matrix $Φ\in \R^{M\times N}$ $(M<N)$. We assume this vector can be well approximated by $K$ non-zero coefficients (i.e., it is $K$-sparse). In addition, the nonzero coefficients of $\bestsignal$ can obey additional structure constraints such as matroid, totally unimodular, or knapsack constraints, wh…
We present a detailed analysis of the unconstrained $\ell_1$-weighted LASSO method for recovery of sparse data from its observation by randomly generated matrices, satisfying the Restricted Isometry Property (RIP) with constant $δ<1$, and subject to negligible measurement and compressibility errors. We prove that if the data is $k$-sparse, then the size of support of the LASSO minimizer, $s$, maintains a comparable sparsity, $s\leq C_δk$. For example, if $δ=0.7$ then $s< 11k$ and a slightly smaller $δ=0.4$ yields $s< 4k$. We also derive new $\ell_2/\ell_1$ error bounds which highlight precise …
We discuss a general notion of "sparsity structure" and associated recoveries of a sparse signal from its linear image of reduced dimension possibly corrupted with noise. Our approach allows for unified treatment of (a) the "usual sparsity" and "usual $\ell_1$ recovery," (b) block-sparsity with possibly overlapping blocks and associated block-$\ell_1$ recovery, and (c) low-rank-oriented recovery by nuclear norm minimization. The proposed recovery routines are natural extensions of the usual $\ell_1$ minimization used in Compressed Sensing. Specifically we present nullspace-type sufficient cond…
Lower dimensional signal representation schemes frequently assume that the signal of interest lies in a single vector space. In the context of the recently developed theory of compressive sensing (CS), it is often assumed that the signal of interest is sparse in an orthonormal basis. However, in many practical applications, this requirement may be too restrictive. A generalization of the standard sparsity assumption is that the signal lies in a union of subspaces. Recovery of such signals from a small number of samples has been studied recently in several works. Here, we consider the problem o…
In this paper we present a new coherence-based performance guarantee for the Orthogonal Matching Pursuit (OMP) algorithm. A lower bound for the probability of correctly identifying the support of a sparse signal with additive white Gaussian noise is derived. Compared to previous work, the new bound takes into account the signal parameters such as dynamic range, noise variance, and sparsity. Numerical simulations show significant improvements over previous work and a closer match to empirically obtained results of the OMP algorithm.
In this paper, we consider the recovery of block sparse signals, whose nonzero entries appear in blocks (or clusters) rather than spread arbitrarily throughout the signal, from incomplete linear measurement. A high order sufficient condition based on block RIP is obtained to guarantee the stable recovery of all block sparse signals in the presence of noise, and robust recovery when signals are not exactly block sparse via mixed $l_{2}/l_{1}$ minimization. Moreover, a concrete example is established to ensure the condition is sharp. The significance of the results presented in this paper lies i…
The problem of recovering signals of high complexity from low quality sensing devices is analyzed via a combination of tools from signal processing and harmonic analysis. By using the rich structure offered by the recent development in fusion frames, we introduce a compressed sensing framework in which we split the dense information into sub-channel or local pieces and then fuse the local estimations. Each piece of information is measured by potentially low quality sensors, modeled by linear matrices and recovered via compressed sensing -- when necessary. Finally, by a fusion process within th…
In this paper, {the goal is to design deterministic sampling patterns on the sphere and the rotation group} and, thereby, construct sensing matrices for sparse recovery of band-limited functions. It is first shown that random sensing matrices, which consists of random samples of Wigner D-functions, satisfy the Restricted Isometry Property (RIP) with proper preconditioning and can be used for sparse recovery on the rotation group. The mutual coherence, however, is used to assess the performance of deterministic and regular sensing matrices. We show that many of widely used regular sampling patt…
Fusion frames are collection of subspaces which provide a redundant representation of signal spaces. They generalize classical frames by replacing frame vectors with frame subspaces. This paper considers the sparse recovery of a signal from a fusion frame. We use a block sparsity model for fusion frames and then show that sparse signals under this model can be compressively sampled and reconstructed in ways similar to standard Compressed Sensing (CS). In particular we invoke a mixed l1/l2 norm minimization in order to reconstruct sparse signals. In our work, we show that assuming a certain inc…
We consider a generalization of the multiple measurement vector (MMV) problem, where the measurement matrices are allowed to differ across measurements. This problem arises naturally when multiple measurements are taken over time, e.g., and the measurement modality (matrix) is time-varying. We derive probabilistic recovery guarantees showing that---under certain (mild) conditions on the measurement matrices---l2/l1-norm minimization and a variant of orthogonal matching pursuit fail with a probability that decays exponentially in the number of measurements. This allows us to conclude that, perh…
We investigate the problem of estimating the unknown degree of sparsity from compressive measurements without the need to carry out a sparse recovery step. While the sparsity order can be directly inferred from the effective rank of the observation matrix in the multiple snapshot case, this appears to be impossible in the more challenging single snapshot case. We show that specially designed measurement matrices allow to rearrange the measurement vector into a matrix such that its effective rank coincides with the effective sparsity order. In fact, we prove that matrices which are composed of …
The sparsity in levels model recently inspired a new generation of effective acquisition and reconstruction modalities for compressive imaging. Moreover, it naturally arises in various areas of signal processing such as parallel acquisition, radar, and the sparse corruptions problem. Reconstruction strategies for sparse in levels signals usually rely on a suitable convex optimization program. Notably, although iterative and greedy algorithms can outperform convex optimization in terms of computational efficiency and have been studied extensively in the case of standard sparsity, little is know…
Compressed sensing provided a data-acquisition paradigm for sparse signals. Remarkably, it has been shown that practical algorithms provide robust recovery from noisy linear measurements acquired at a near optimal sampling rate. In many real-world applications, a signal of interest is typically sparse not in the canonical basis but in a certain transform domain, such as wavelets or the finite difference. The theory of compressed sensing was extended to the analysis sparsity model but known extensions are limited to specific choices of sensing matrix and sparsifying transform. In this paper, we…
Compressed sensing deals with the reconstruction of sparse signals using a small number of linear measurements. One of the main challenges in compressed sensing is to find the support of a sparse signal. In the literature, several bounds on the scaling law of the number of measurements for successful support recovery have been derived where the main focus is on random Gaussian measurement matrices. In this paper, we investigate the noisy support recovery problem from an estimation theoretic point of view, where no specific assumption is made on the underlying measurement matrix. The linear mea…
The problem of consistently estimating the sparsity pattern of a vector $\betastar \in \real^\mdim$ based on observations contaminated by noise arises in various contexts, including subset selection in regression, structure estimation in graphical models, sparse approximation, and signal denoising. We analyze the behavior of $\ell_1$-constrained quadratic programming (QP), also referred to as the Lasso, for recovering the sparsity pattern. Our main result is to establish a sharp relation between the problem dimension $\mdim$, the number $\spindex$ of non-zero elements in $\betastar$, and the n…
University of Electronic Science and Technology of China（电子科技大学）；China Academy of Engineering Physics（中国工程物理研究院）
In this paper, we consider the block-sparse signals recovery problem in the context of multiple measurement vectors (MMV) with common row sparsity patterns. We develop a new method for recovery of common row sparsity MMV signals, where a pattern-coupled hierarchical Gaussian prior model is introduced to characterize both the block-sparsity of the coefficients and the statistical dependency between neighboring coefficients of the common row sparsity MMV signals. Unlike many other methods, the proposed method is able to automatically capture the block sparse structure of the unknown signal. Our …
In this paper we introduce a nonuniform sparsity model and analyze the performance of an optimized weighted $\ell_1$ minimization over that sparsity model. In particular, we focus on a model where the entries of the unknown vector fall into two sets, with entries of each set having a specific probability of being nonzero. We propose a weighted $\ell_1$ minimization recovery algorithm and analyze its performance using a Grassmann angle approach. We compute explicitly the relationship between the system parameters-the weights, the number of measurements, the size of the two sets, the probabiliti…
In sparse recovery, the unique sparsest solution to an under-determined system of linear equations is of main interest. This scheme is commonly proposed to be applied to signal acquisition. In most cases, the signals are not sparse themselves, and therefore, they need to be sparsely represented with the help of a so-called dictionary being specific to the corresponding signal family. The dictionaries cannot be used for optimization of the resulting under-determined system because they are fixed by the given signal family. However, the measurement matrix is available for optimization and can be…
Subsampled blind deconvolution is the recovery of two unknown signals from samples of their convolution. To overcome the ill-posedness of this problem, solutions based on priors tailored to specific application have been developed in practical applications. In particular, sparsity models have provided promising priors. However, in spite of empirical success of these methods in many applications, existing analyses are rather limited in two main ways: by disparity between the theoretical assumptions on the signal and/or measurement model versus practical setups; or by failure to provide a perfor…
A trend in compressed sensing (CS) is to exploit structure for improved reconstruction performance. In the basic CS model, exploiting the clustering structure among nonzero elements in the solution vector has drawn much attention, and many algorithms have been proposed. However, few algorithms explicitly consider correlation within a cluster. Meanwhile, in the multiple measurement vector (MMV) model correlation among multiple solution vectors is largely ignored. Although several recently developed algorithms consider the exploitation of the correlation, these algorithms need to know a priori t…
Since compressive sensing deals with a signal reconstruction using a reduced set of measurements, the existence of a unique solution is of crucial importance. The most important approach to this problem is based on the restricted isometry property which is computationally unfeasible. The coherence index-based uniqueness criteria are computationally efficient, however, they are pessimistic. An approach to alleviate this problem has been recently introduced by relaxing the coherence index condition for the unique signal reconstruction using the orthogonal matching pursuit approach. This approach…
We present an uncertainty relation for the representation of signals in two different general (possibly redundant or incomplete) signal sets. This uncertainty relation is relevant for the analysis of signals containing two distinct features each of which can be described sparsely in a suitable general signal set. Furthermore, the new uncertainty relation is shown to lead to improved sparsity thresholds for recovery of signals that are sparse in general dictionaries. Specifically, our results improve on the well-known $(1+1/d)/2$-threshold for dictionaries with coherence $d$ by up to a factor o…
