In many compressed sensing applications, linear programming (LP) has been used to reconstruct a sparse signal. When observation is noisy, the LP formulation is extended to allow an inequality constraint and the solution is dependent on a parameter $δ$, related to the observation noise level. Recently, some researchers also considered quadratic programming (QP) for compressed sensing signal reconstruction and the solution in this case is dependent on a Lagrange multiplier $β$. In this work, we investigated the relation between $δ$ and $β$ and derived an upper and a lower bound on $β$ in terms of $δ$. For a given $δ$, these bounds can be used to approximate $β$. Since $δ$ is a physically related quantity and easy to determine for an application while there is no easy way in general to determine $β$, our results can be used to set $β$ when the QP is used for compressed sensing. Our results and experimental verification also provide some insight into the solutions generated by compressed sensing.
We discuss the applicability of compressed sensing theory. We take a genuine look at both experimental results and theoretical works. We answer the following questions: 1) What can compressed sensing really do? 2) More importantly, why?
In this paper the authors describe the problem of acquisition of interfered signals and formulate a filtering problem. A frequency-selective compressed sensing technique is proposed as a solution to this problem. Signal acquisition is critical in facilitating frequency-selective compressed sensing. The authors propose a filtering compressed sensing parameter, which allows to assess if a given acquisition process makes frequency-selective compressed sensing possible for a given filtering problem. A numerical experiment which shows how the described method works in practice is conducted.
The restricted isometry property (RIP) has become well-known in the compressed sensing community. Recently, a weaken version of RIP was proposed for exact sparse recovery under weak moment assumptions. In this note, we prove that the weaken RIP is also sufficient for \textsl{stable and robust} sparse recovery by linking it with a recently introduced robust width property in compressed sensing. Moreover, we show that it can be widely apply to other compressed sensing instances as well.
In this letter, we propose a turbo compressed sensing algorithm with partial discrete Fourier transform (DFT) sensing matrices. Interestingly, the state evolution of the proposed algorithm is shown to be consistent with that derived using the replica method. Numerical results demonstrate that the proposed algorithm outperforms the well-known approximate message passing (AMP) algorithm when a partial DFT sensing matrix is involved.
Compressed sensing is a novel technique where one can recover sparse signals from the undersampled measurements. In this correspondence, a $K \times N$ measurement matrix for compressed sensing is deterministically constructed via additive character sequences. The Weil bound is then used to show that the matrix has asymptotically optimal coherence for $N=K^2$, and to present a sufficient condition on the sparsity level for unique sparse recovery. Also, the restricted isometry property (RIP) is statistically studied for the deterministic matrix. Using additive character sequences with small alp…
Compressed sensing is a signal processing technique whereby the limits imposed by the Shannon--Nyquist theorem can be exceeded provided certain conditions are imposed on the signal. Such conditions occur in many real-world scenarios, and compressed sensing has emerging applications in medical imaging, big data, and statistics. Finding practical matrix constructions and computationally efficient recovery algorithms for compressed sensing is an area of intense research interest. Many probabilistic matrix constructions have been proposed, and it is now well known that matrices with entries drawn …
Compressed sensing is now established as an effective method for dimension reduction when the underlying signals are sparse or compressible with respect to some suitable basis or frame. One important, yet under-addressed problem regarding the compressive acquisition of analog signals is how to perform quantization. This is directly related to the important issues of how "compressed" compressed sensing is (in terms of the total number of bits one ends up using after acquiring the signal) and ultimately whether compressed sensing can be used to obtain compressed representations of suitable signa…
In this paper, we analyze the information theoretic lower bound on the necessary number of samples needed for recovering a sparse signal under different compressed sensing settings. We focus on the weighted graph model, a model-based framework proposed by Hegde et al. (2015), for standard compressed sensing as well as for one-bit compressed sensing. We study both the noisy and noiseless regimes. Our analysis is general in the sense that it applies to any algorithm used to recover the signal. We carefully construct restricted ensembles for different settings and then apply Fano's inequality to …
Compressed sensing (CS) enables people to acquire the compressed measurements directly and recover sparse or compressible signals faithfully even when the sampling rate is much lower than the Nyquist rate. However, the pure random sensing matrices usually require huge memory for storage and high computational cost for signal reconstruction. Many structured sensing matrices have been proposed recently to simplify the sensing scheme and the hardware implementation in practice. Based on the restricted isometry property and coherence, couples of existing structured sensing matrices are reviewed in…
In the present note we develop some theoretical results in the theory of anisotropic compressed sensing that allow to take structured sparsity and variable density structured sampling into account. We expect that the obtained results will be useful to derive explicit expressions for optimal sampling strategies in the non-Cartesian (radial, spiral, etc.) setting in MRI.
This letter proposes a novel distributed compressed estimation scheme for sparse signals and systems based on compressive sensing techniques. The proposed scheme consists of compression and decompression modules inspired by compressive sensing to perform distributed compressed estimation. A design procedure is also presented and an algorithm is developed to optimize measurement matrices, which can further improve the performance of the proposed distributed compressed estimation scheme. Simulations for a wireless sensor network illustrate the advantages of the proposed scheme and algorithm in t…
This chapter gives an overview over recovery guarantees for total variation minimization in compressed sensing for different measurement scenarios. In addition to summarizing the results in the area, we illustrate why an approach that is common for synthesis sparse signals fails and different techniques are necessary. Lastly, we discuss a generalizations of recent results for Gaussian measurements to the subgaussian case.
In this paper, we provide a new approach to estimating the error of reconstruction from $ΣΔ$ quantized compressed sensing measurements. Our method is based on the restricted isometry property (RIP) of a certain projection of the measurement matrix. Our result yields simple proofs and a slight generalization of the best-known reconstruction error bounds for Gaussian and subgaussian measurement matrices.
This paper surveys recent work in applying ideas from graphical models and message passing algorithms to solve large scale regularized regression problems. In particular, the focus is on compressed sensing reconstruction via ell_1 penalized least-squares (known as LASSO or BPDN). We discuss how to derive fast approximate message passing algorithms to solve this problem. Surprisingly, the analysis of such algorithms allows to prove exact high-dimensional limit results for the LASSO risk. This paper will appear as a chapter in a book on `Compressed Sensing' edited by Yonina Eldar and Gitta Kutyn…
The field of compressed sensing has shown that a sparse but otherwise arbitrary vector can be recovered exactly from a small number of randomly constructed linear projections (or samples). The question addressed in this paper is whether an even smaller number of samples is sufficient when there exists prior knowledge about the distribution of the unknown vector, or when only partial recovery is needed. An information-theoretic lower bound with connections to free probability theory and an upper bound corresponding to a computationally simple thresholding estimator are derived. It is shown that…
In one-bit compressed sensing, previous results state that sparse signals may be robustly recovered when the measurements are taken using Gaussian random vectors. In contrast to standard compressed sensing, these results are not extendable to natural non-Gaussian distributions without further assumptions, as can be demonstrated by simple counter-examples. We show that approximately sparse signals that are not extremely sparse can be accurately reconstructed from single-bit measurements sampled according to a sub-gaussian distribution, and the reconstruction comes as the solution to a convex pr…
The weak-$\ell^p$ norm can be used to define a measure $s$ of sparsity. When we compute $s$ for the discrete cosine transform coefficients of a signal, the value of $s$ is related to the information content of said signal. We use this value of $s$ to define a reference-free index $\mathcal{E}$, called the sparsity index, that we can use to predict with high accuracy the quality of signal reconstruction in the setting of compressed sensing imaging. That way, when compressed sensing is framed in the context of sampling theory, we can use $\mathcal{E}$ to decide when to further partition the samp…
This paper addresses the topic of robust Bayesian compressed sensing over finite fields. For stationary and ergodic sources, it provides asymptotic (with the size of the vector to estimate) necessary and sufficient conditions on the number of required measurements to achieve vanishing reconstruction error, in presence of sensing and communication noise. In all considered cases, the necessary and sufficient conditions asymptotically coincide. Conditions on the sparsity of the sensing matrix are established in presence of communication noise. Several previously published results are generalized …
This paper describes performance bounds for compressed sensing in the presence of Poisson noise when the underlying signal, a vector of Poisson intensities, is sparse or compressible (admits a sparse approximation). The signal-independent and bounded noise models used in the literature to analyze the performance of compressed sensing do not accurately model the effects of Poisson noise. However, Poisson noise is an appropriate noise model for a variety of applications, including low-light imaging, where sensing hardware is large or expensive, and limiting the number of measurements collected i…
Sign truncated matching pursuit (STrMP) algorithm is presented in this paper. STrMP is a new greedy algorithm for the recovery of sparse signals from the sign measurement, which combines the principle of consistent reconstruction with orthogonal matching pursuit (OMP). The main part of STrMP is as concise as OMP and hence STrMP is simple to implement. In contrast to previous greedy algorithms for one-bit compressed sensing, STrMP only need to solve a convex and unconstraint subproblem at each iteration. Numerical experiments show that STrMP is fast and accurate for one-bit compressed sensing c…
Most of compressed sensing (CS) theory to date is focused on incoherent sensing, that is, columns from the sensing matrix are highly uncorrelated. However, sensing systems with naturally occurring correlations arise in many applications, such as signal detection, motion detection and radar. Moreover, in these applications it is often not necessary to know the support of the signal exactly, but instead small errors in the support and signal are tolerable. Despite the abundance of work utilizing incoherent sensing matrices, for this type of tolerant recovery we suggest that coherence is actually…
The analysis sparsity model is a very effective approach in modern Compressed Sensing applications. Specifically, redundant analysis operators can lead to fewer measurements needed for reconstruction when employing the analysis $l_1$-minimization in Compressed Sensing. In this paper, we pick an eigenvector of the Zauner unitary matrix and -- under certain assumptions on the ambient dimension -- we build a spark deficient Gabor frame. The analysis operator associated with such a spark deficient Gabor frame, is a new (highly) redundant Gabor transform, which we use as a sparsifying transform in …
In this letter, a binary sparse Bayesian learning (BSBL) algorithm is proposed to slove the one-bit compressed sensing (CS) problem in both single measurement vector (SMV) and multiple measurement vectors (MMVs). By utilising the Bussgang-like decomposition, the one-bit CS problem can be approximated as a standard linear model. Consequently, the standard SBL algorithm can be naturally incorporated. Numerical results demonstrate the effectiveness of the BSBL algorithm.
Compressed sensing is a novel technique where one can recover sparse signals from the undersampled measurements. In this paper, a $K \times N$ measurement matrix for compressed sensing is deterministically constructed via multiplicative character sequences. Precisely, a constant multiple of a cyclic shift of an $M$-ary power residue or Sidelnikov sequence is arranged as a column vector of the matrix, through modulating a primitive $M$-th root of unity. The Weil bound is then used to show that the matrix has asymptotically optimal coherence for large $K$ and $M$, and to present a sufficient con…
A signature result in compressed sensing is that Gaussian random sampling achieves stable and robust recovery of sparse vectors under optimal conditions on the number of measurements. However, in the context of image reconstruction, it has been extensively documented that sampling strategies based on Fourier measurements outperform this purportedly optimal approach. Motivated by this seeming paradox, we investigate the problem of optimal sampling for compressed sensing. Rigorously combining the theories of wavelet approximation and infinite-dimensional compressed sensing, our analysis leads to…
This paper proposes an efficient method to construct the bipartite graph with as many edges as possible while without introducing the shortest cycles of length equal to 4. The binary matrix associated with the bipartite graph described above presents comparable and even better phase transitions than Gaussian random matrices.
We propose an innovative meteorological radar, which uses reduced number of spatiotemporal samples without compromising the accuracy of target information. Our approach extends recent research on compressed sensing (CS) for radar remote sensing of hard point scatterers to volumetric targets. The previously published CS-based radar techniques are not applicable for sampling weather since the precipitation echoes lack sparsity in both range-time and Doppler domains. We propose an alternative approach by adopting the latest advances in matrix completion algorithms to demonstrate the sparse sensin…
We consider vector-quantized (VQ) transmission of compressed sensing (CS) measurements over noisy channels. Adopting mean-square error (MSE) criterion to measure the distortion between a sparse vector and its reconstruction, we derive channel-optimized quantization principles for encoding CS measurement vector and reconstructing sparse source vector. The resulting necessary optimal conditions are used to develop an algorithm for training channel-optimized vector quantization (COVQ) of CS measurements by taking the end-to-end distortion measure into account.
Compressed sensing (CS) demonstrates that a sparse, or compressible signal can be acquired using a low rate acquisition process below the Nyquist rate, which projects the signal onto a small set of vectors incoherent with the sparsity basis. In this paper, we propose a new framework for compressed sensing recovery problem using iterative approximation method via L0 minimization. Instead of directly solving the unconstrained L0 norm optimization problem, we use the linearization and proximal points techniques to approximate the penalty function at each iteration. The proposed algorithm is very …
Traditional compressed sensing considers sampling a 1D signal. For a multidimensional signal, if reshaped into a vector, the required size of the sensing matrix becomes dramatically large, which increases the storage and computational complexity significantly. To solve this problem, we propose to reshape the multidimensional signal into a 2D signal and sample the 2D signal using compressed sensing column by column with the same sensing matrix. It is referred to as parallel compressed sensing, and it has much lower storage and computational complexity. For a given reconstruction performance of …
We consider a resource-constrained scenario where a compressed sensing- (CS) based sensor has a low number of measurements which are quantized at a low rate followed by transmission or storage. Applying this scenario, we develop a new quantizer design which aims to attain a high-quality reconstruction performance of a sparse source signal based on analysis-by-synthesis framework. Through simulations, we compare the performance of the proposed quantization algorithm vis-a-vis existing quantization methods.
The achievable and converse regions for sparse representation of white Gaussian noise based on an overcomplete dictionary are derived in the limit of large systems. Furthermore, the marginal distribution of such sparse representations is also inferred. The results are obtained via the Replica method which stems from statistical mechanics. A direct outcome of these results is the introduction of sharp threshold for $\ell_{0}$-norm decoding in noisy compressed sensing, and its mean-square error for underdetermined Gaussian vector channels.
This paper considers the fundamental limit of compressed sensing for i.i.d. signal distributions and i.i.d. Gaussian measurement matrices. Its main contribution is a rigorous characterization of the asymptotic mutual information (MI) and minimum mean-square error (MMSE) in this setting. Under mild technical conditions, our results show that the limiting MI and MMSE are equal to the values predicted by the replica method from statistical physics. This resolves a well-known problem that has remained open for over a decade.
Construction on the measurement matrix $A$ is a central problem in compressed sensing. Although using random matrices is proven optimal and successful in both theory and applications. A deterministic construction on the measurement matrix is still very important and interesting. In fact, it is still an open problem proposed by T. Tao. In this paper, we shall provide a new deterministic construction method and prove it is optimal with regard to the mutual incoherence.
The application of Compresses Sensing is a promising physical layer technology for the joint activity and data detection of signals. Detecting the activity pattern correctly has severe impact on the system performance and is therefore of major concern. In contrast to previous work, in this paper we optimize joint activity and data detection in under-determined systems by minimizing the Bayes-Risk for erroneous activity detection. We formulate a new Compressed Sensing Bayes-Risk detector which directly allows to influence error rates at the activity detection dynamically by a parameter that can…
A new variant of the Compressed Sensing problem is investigated when the number of measurements corrupted by errors is upper bounded by some value l but there are no more restrictions on errors. We prove that in this case it is enough to make 2(t+l) measurements, where t is the sparsity of original data. Moreover for this case a rather simple recovery algorithm is proposed. An analog of the Singleton bound from coding theory is derived what proves optimality of the corresponding measurement matrices.
Modern image and video compression codes employ elaborate structures existing in such signals to encode them into few number of bits. Compressed sensing recovery algorithms on the other hand use such signals' structures to recover them from few linear observations. Despite the steady progress in the field of compressed sensing, structures that are often used for signal recovery are still much simpler than those employed by state-of-the-art compression codes. The main goal of this paper is to bridge this gap through answering the following question: Can one employ a given compression code to bu…
As an alternative to the traditional sampling theory, compressed sensing allows acquiring much smaller amount of data, still estimating the spectra of frequency-sparse signals accurately. However, compressed sensing usually requires random sampling in data acquisition, which is difficult to implement in hardware. In this paper, we propose a deterministic and simple sampling scheme, that is, sampling at three sub-Nyquist rates which have coprime undersampled ratios. This sampling method turns out to be valid through numerical experiments. A complex-valued multitask algorithm based on variationa…
This paper provides performance bounds for compressed sensing in the presence of Poisson noise using expander graphs. The Poisson noise model is appropriate for a variety of applications, including low-light imaging and digital streaming, where the signal-independent and/or bounded noise models used in the compressed sensing literature are no longer applicable. In this paper, we develop a novel sensing paradigm based on expander graphs and propose a MAP algorithm for recovering sparse or compressible signals from Poisson observations. The geometry of the expander graphs and the positivity of t…
Two-part reconstruction is a framework for signal recovery in compressed sensing (CS), in which the advantages of two different algorithms are combined. Our framework allows to accelerate the reconstruction procedure without compromising the reconstruction quality. To illustrate the efficacy of our two-part approach, we extend the author's previous Sudocodes algorithm and make it robust to measurement noise. In a 1-bit CS setting, promising numerical results indicate that our algorithm offers both a reduction in run-time and improvement in reconstruction quality.
Compressed sensing (CS) and 1-bit CS cannot directly recover quantized signals and require time consuming recovery. In this paper, we introduce \textit{Hamming compressed sensing} (HCS) that directly recovers a k-bit quantized signal of dimensional $n$ from its 1-bit measurements via invoking $n$ times of Kullback-Leibler divergence based nearest neighbor search. Compared with CS and 1-bit CS, HCS allows the signal to be dense, takes considerably less (linear) recovery time and requires substantially less measurements ($\mathcal O(\log n)$). Moreover, HCS recovery can accelerate the subsequent…
Compressed sensing is a promising technique that attempts to faithfully recover sparse signal with as few linear and nonadaptive measurements as possible. Its performance is largely determined by the characteristic of sensing matrix. Recently several zero-one binary sensing matrices have been deterministically constructed for their relative low complexity and competitive performance. Considering the complexity of implementation, it is of great practical interest if one could further improve the sparsity of binary matrix without performance loss. Based on the study of restricted isometry proper…
Binary matrix and ternary matrix are two types of popular sensing matrices in compressed sensing for their competitive performance and low computation. However, to the best of our knowledge, there seems no literature aiming at evaluating their performances if they hold the same sparisty, though it is of practical importance. Based on both RIP analysis and numerical simulations, this paper, for the first time, discloses that {0, 1} binary matrix holds better overall performance over {0, +1, -1} ternary matrix, if they share the same distribution on nonzero positions.
There have been many matching pursuit algorithms (MPAs) which handle the sparse signal recovery problem a.k.a. compressed sensing (CS). In the MPAs, the correlation computation step has a dominant computational complexity. In this letter, we propose a new fast correlation computation method when we use some classes of partial unitary matrices as the sensing matrix. Those partial unitary matrices include partial Fourier matrices and partial Hadamard matrices which are popular sensing matrices. The proposed correlation computation method can be applied to almost all MPAs without causing any degr…
Wireless tomography is a technique for inferring a physical environment within a monitored region by analyzing RF signals traversed across the region. In this paper, we consider wireless tomography in a two and higher dimensionally structured monitored region, and propose a multi-dimensional wireless tomography scheme based on compressed sensing to estimate a spatial distribution of shadowing loss in the monitored region. In order to estimate the spatial distribution, we consider two compressed sensing frameworks: vector-based compressed sensing and tensor-based compressed sensing. When the sh…
We introduce a theoretical approach for designing generalizations of the approximate message passing (AMP) algorithm for compressed sensing which are valid for large observation matrices that are drawn from an invariant random matrix ensemble. By design, the fixed points of the algorithm obey the Thouless-Anderson-Palmer (TAP) equations corresponding to the ensemble. Using a dynamical functional approach we are able to derive an effective stochastic process for the marginal statistics of a single component of the dynamics. This allows us to design memory terms in the algorithm in such a way th…
This paper studies the problem of recovering a signal from one-bit compressed sensing measurements under a manifold model; that is, assuming that the signal lies on or near a manifold of low intrinsic dimension. We provide a convex recovery method based on the Geometric Multi-Resolution Analysis and prove recovery guarantees with a near-optimal scaling in the intrinsic manifold dimension. Our method is the first tractable algorithm with such guarantees for this setting. The results are complemented by numerical experiments confirming the validity of our approach.
The recently proposed Magnetic Resonance Fingerprinting (MRF) technique can simultaneously estimate multiple parameters through dictionary matching. It has promising potentials in a wide range of applications. However, MRF introduces errors due to undersampling during the data acquisition process and the limit of dictionary resolution. In this paper, we investigate the error source of MRF and propose the technologies of improving the quality of MRF with compressed sensing, error prediction by decision trees, and adaptive filtering. Experimental results support our observations and show signifi…
The central idea of compressed sensing is to exploit the fact that most signals of interest are sparse in some domain and use this to reduce the number of measurements to encode. However, if the sparsity of the input signal is not precisely known, but known to lie within a specified range, compressed sensing as such cannot exploit this fact and would need to use the same number of measurements even for a very sparse signal. In this paper, we propose a novel method called Compressed Shattering to adapt compressed sensing to the specified sparsity range, without changing the sensing matrix by cr…
The sparse signal recovery in the standard compressed sensing (CS) problem requires that the sensing matrix be known a priori. Such an ideal assumption may not be met in practical applications where various errors and fluctuations exist in the sensing instruments. This paper considers the problem of compressed sensing subject to a structured perturbation in the sensing matrix. Under mild conditions, it is shown that a sparse signal can be recovered by $\ell_1$ minimization and the recovery error is at most proportional to the measurement noise level, which is similar to the standard CS result.…
Distributed compressed sensing is concerned with representing an ensemble of jointly sparse signals using as few linear measurements as possible. Two novel joint reconstruction algorithms for distributed compressed sensing are presented in this paper. These algorithms are based on the idea of using one of the signals as side information; this allows to exploit joint sparsity in a more effective way with respect to existing schemes. They provide gains in reconstruction quality, especially when the nodes acquire few measurements, so that the system is able to operate with fewer measurements than…
A multiplicative Gaussian wire-tap channel inspired by compressed sensing is studied. Lower and upper bounds on the secrecy capacity are derived, and shown to be relatively tight in the large system limit for a large class of compressed sensing matrices. Surprisingly, it is shown that the secrecy capacity of this channel is nearly equal to the capacity without any secrecy constraint provided that the channel of the eavesdropper is strictly worse than the channel of the intended receiver. In other words, the eavesdropper can see almost everything and yet learn almost nothing. This behavior, whi…
Compressed sensing is a novel technique where one can recover sparse signals from the undersampled measurements. This paper studies a $K \times N$ partial Fourier measurement matrix for compressed sensing which is deterministically constructed via cyclic difference sets (CDS). Precisely, the matrix is constructed by $K$ rows of the $N\times N$ inverse discrete Fourier transform (IDFT) matrix, where each row index is from a $(N, K, λ)$ cyclic difference set. The restricted isometry property (RIP) is statistically studied for the deterministic matrix to guarantee the recovery of sparse signals. …
This letter investigates the joint recovery of a frequency-sparse signal ensemble sharing a common frequency-sparse component from the collection of their compressed measurements. Unlike conventional arts in compressed sensing, the frequencies follow an off-the-grid formulation and are continuously valued in $\left\lbrack 0,1 \right\rbrack$. As an extension of atomic norm, the concatenated atomic norm minimization approach is proposed to handle the exact recovery of signals, which is reformulated as a computationally tractable semidefinite program. The optimality of the proposed approach is ch…
Although largely different concepts, echo state networks and compressed sensing models both rely on collections of random weights; as the reservoir dynamics for echo state networks, and the sensing coefficients in compressed sensing. Several methods for generating the random matrices and metrics to indicate desirable performance are well-studied in compressed sensing, but less so for echo state networks. This work explores any overlap in these compressed sensing methods and metrics for application to echo state networks. Several methods for generating the random reservoir weights are considere…
The fundamental principle underlying compressed sensing is that a signal, which is sparse under some basis representation, can be recovered from a small number of linear measurements. However, prior knowledge of the sparsity basis is essential for the recovery process. This work introduces the concept of blind compressed sensing, which avoids the need to know the sparsity basis in both the sampling and the recovery process. We suggest three possible constraints on the sparsity basis that can be added to the problem in order to make its solution unique. For each constraint we prove conditions f…
We study two approaches to distributed compressed sensing for in-network data compression and signal reconstruction at a sink in a wireless sensor network where sensors are placed on a straight line. Communication to the sink is considered to be bandwidth-constrained due to the large number of devices. By using distributed compressed sensing for compression of the data in the network, the communication cost (bandwith usage) to the sink can be decreased at the expense of delay induced by the local communication necessary for compression. We investigate the relation between cost and delay given …
