Designing Incoherent Dictionaries for Compressed Sensing: Algorithm Comparison

Optimized Compressed Sensing via Incoherent Frames Designed by Convex Optimization

The construction of highly incoherent frames, sequences of vectors placed on the unit hyper sphere of a finite dimensional Hilbert space with low correlation between them, has proven very difficult. Algorithms proposed in the past have focused in minimizing the absolute value off-diagonal entries of the Gram matrix of these structures. Recently, a method based on convex optimization that operates directly on the vectors of the frame has been shown to produce promising results. This paper gives a detailed analysis of the optimization problem at the heart of this approach and, based on these ins…

Sparsity based sub-wavelength imaging with partially incoherent light via quadratic compressed sensing

We demonstrate that sub-wavelength optical images borne on partially-spatially-incoherent light can be recovered, from their far-field or from the blurred image, given the prior knowledge that the image is sparse, and only that. The reconstruction method relies on the recently demonstrated sparsity-based sub-wavelength imaging. However, for partially-spatially-incoherent light, the relation between the measurements and the image is quadratic, yielding non-convex measurement equations that do not conform to previously used techniques. Consequently, we demonstrate new algorithmic methodology, re…

On some deterministic dictionaries supporting sparsity

We describe a new construction of an incoherent dictionary, referred to as the oscillator dictionary, which is based on considerations in the representation theory of finite groups. The oscillator dictionary consists of order of p^5 unit vectors in a Hilbert space of dimension p, where p is an odd prime, whose pairwise inner products have magnitude of at most 4/sqrt(p). An explicit algorithm to construct a large portion of the oscillator dictionary is presented.

New Guarantees for Blind Compressed Sensing

Blind Compressed Sensing (BCS) is an extension of Compressed Sensing (CS) where the optimal sparsifying dictionary is assumed to be unknown and subject to estimation (in addition to the CS sparse coefficients). Since the emergence of BCS, dictionary learning, a.k.a. sparse coding, has been studied as a matrix factorization problem where its sample complexity, uniqueness and identifiability have been addressed thoroughly. However, in spite of the strong connections between BCS and sparse coding, recent results from the sparse coding problem area have not been exploited within the context of BCS…

Tolerant Compressed Sensing With Partially Coherent Sensing Matrices

Most of compressed sensing (CS) theory to date is focused on incoherent sensing, that is, columns from the sensing matrix are highly uncorrelated. However, sensing systems with naturally occurring correlations arise in many applications, such as signal detection, motion detection and radar. Moreover, in these applications it is often not necessary to know the support of the signal exactly, but instead small errors in the support and signal are tolerable. Despite the abundance of work utilizing incoherent sensing matrices, for this type of tolerant recovery we suggest that coherence is actually…

Infinite dimensional compressed sensing from anisotropic measurements and applications to inverse problems in PDE

We consider a compressed sensing problem in which both the measurement and the sparsifying systems are assumed to be frames (not necessarily tight) of the underlying Hilbert space of signals, which may be finite or infinite dimensional. The main result gives explicit bounds on the number of measurements in order to achieve stable recovery, which depends on the mutual coherence of the two systems. As a simple corollary, we prove the efficiency of nonuniform sampling strategies in cases when the two systems are not incoherent, but only asymptotically incoherent, as with the recovery of wavelet c…

Optimized projections for compressed sensing via rank-constrained nearest correlation matrix

Optimizing the acquisition matrix is useful for compressed sensing of signals that are sparse in overcomplete dictionaries, because the acquisition matrix can be adapted to the particular correlations of the dictionary atoms. In this paper a novel formulation of the optimization problem is proposed, in the form of a rank-constrained nearest correlation matrix problem. Furthermore, improvements for three existing optimization algorithms are introduced, which are shown to be particular instances of the proposed formulation. Simulation results show notable improvements and superior robustness in …

On the Error in Phase Transition Computations for Compressed Sensing

Evaluating the statistical dimension is a common tool to determine the asymptotic phase transition in compressed sensing problems with Gaussian ensemble. Unfortunately, the exact evaluation of the statistical dimension is very difficult and it has become standard to replace it with an upper-bound. To ensure that this technique is suitable, [1] has introduced an upper-bound on the gap between the statistical dimension and its approximation. In this work, we first show that the error bound in [1] in some low-dimensional models such as total variation and $\ell_1$ analysis minimization becomes po…

压缩感知与冗余字典及结构化测量

Compressive Sensing with Redundant Dictionaries and Structured Measurements

本文研究了在冗余字典和结构化测量下信号恢复问题，提出通过子采样固定正交矩阵保证D-RIP条件，从而为结构化测量和冗余字典的压缩感知提供理论保障。

A null space property approach to compressed sensing with frames

An interesting topic in compressive sensing concerns problems of sensing and recovering signals with sparse representations in a dictionary. In this note, we study conditions of sensing matrices A for the L1-synthesis method to accurately recover sparse, or nearly sparse signals in a given dictionary D. In particular, we propose a dictionary based null space property (D-NSP) which, to the best of our knowledge, is the first sufficient and necessary condition for the success of the L1 recovery. This new property is then utilized to detect some of those dictionaries whose sparse families cannot …

Signal Space CoSaMP for Sparse Recovery with Redundant Dictionaries

Compressive sensing (CS) has recently emerged as a powerful framework for acquiring sparse signals. The bulk of the CS literature has focused on the case where the acquired signal has a sparse or compressible representation in an orthonormal basis. In practice, however, there are many signals that cannot be sparsely represented or approximated using an orthonormal basis, but that do have sparse representations in a redundant dictionary. Standard results in CS can sometimes be extended to handle this case provided that the dictionary is sufficiently incoherent or well-conditioned, but these app…

Algorithms for the Construction of Incoherent Frames Under Various Design Constraints

Unit norm finite frames are generalizations of orthonormal bases with many applications in signal processing. An important property of a frame is its coherence, a measure of how close any two vectors of the frame are to each other. Low coherence frames are useful in compressed sensing applications. When used as measurement matrices, they successfully recover highly sparse solutions to linear inverse problems. This paper describes algorithms for the design of various low coherence frame types: real, complex, unital (constant magnitude) complex, sparse real and complex, nonnegative real and comp…

A null space analysis of the L1 synthesis method in dictionary-based compressed sensing

An interesting topic in compressed sensing aims to recover signals with sparse representations in a dictionary. Recently the performance of the L1-analysis method has been a focus, while some fundamental problems for the L1-synthesis method are still unsolved. For example, what are the conditions for it to stably recover compressible signals under noise? Whether coherent dictionaries allow the existence of sensing matrices that guarantee good performances of the L1-synthesis method? To answer these questions, we build up a framework for the L1-synthesis method. In particular, we propose a dict…

A new inexact iterative hard thresholding algorithm for compressed sensing

Compressed sensing (CS) demonstrates that a sparse, or compressible signal can be acquired using a low rate acquisition process below the Nyquist rate, which projects the signal onto a small set of vectors incoherent with the sparsity basis. In this paper, we propose a new framework for compressed sensing recovery problem using iterative approximation method via L0 minimization. Instead of directly solving the unconstrained L0 norm optimization problem, we use the linearization and proximal points techniques to approximate the penalty function at each iteration. The proposed algorithm is very …

Efficient Adaptive Compressive Sensing Using Sparse Hierarchical Learned Dictionaries

Recent breakthrough results in compressed sensing (CS) have established that many high dimensional objects can be accurately recovered from a relatively small number of non- adaptive linear projection observations, provided that the objects possess a sparse representation in some basis. Subsequent efforts have shown that the performance of CS can be improved by exploiting the structure in the location of the non-zero signal coefficients (structured sparsity) or using some form of online measurement focusing (adaptivity) in the sensing process. In this paper we examine a powerful hybrid of thes…

Spectral Compressive Sensing with Model Selection

The performance of existing approaches to the recovery of frequency-sparse signals from compressed measurements is limited by the coherence of required sparsity dictionaries and the discretization of frequency parameter space. In this paper, we adopt a parametric joint recovery-estimation method based on model selection in spectral compressive sensing. Numerical experiments show that our approach outperforms most state-of-the-art spectral CS recovery approaches in fidelity, tolerance to noise and computation efficiency.

Compressed Sensing for Implantable Neural Recordings Using Co-sparse Analysis Model and Weighted $\ell_1$-Optimization

Reliable and energy-efficient wireless data transmission remains a major challenge in resource-constrained wireless neural recording tasks, where data compression is generally adopted to relax the burdens on the wireless data link. Recently, Compressed Sensing (CS) theory has successfully demonstrated its potential in neural recording application. The main limitation of CS, however, is that the neural signals have no good sparse representation with commonly used dictionaries and learning a reliable dictionary is often data dependent and computationally demanding. In this paper, a novel CS appr…

Analyzing Least Squares and Kalman Filtered Compressed Sensing

In recent work, we studied the problem of causally reconstructing time sequences of spatially sparse signals, with unknown and slow time-varying sparsity patterns, from a limited number of linear "incoherent" measurements. We proposed a solution called Kalman Filtered Compressed Sensing (KF-CS). The key idea is to run a reduced order KF only for the current signal's estimated nonzero coefficients' set, while performing CS on the Kalman filtering error to estimate new additions, if any, to the set. KF may be replaced by Least Squares (LS) estimation and we call the resulting algorithm LS-CS. In…

High Resolution Compressed Sensing Radar using Difference Set Codes

In this paper, we consider compressive sensing (CS)-based recovery of delays and Doppler frequencies of targets in high resolution radars. We propose a novel sub-Nyquist sampling method in the Fourier domain based on difference sets (DS), called DS-sampling, to create dictionaries with highly incoherent atoms. The coherence of the dictionary reaches the Welch minimum bound if the DS-sampling is employed. This property let us to implement sub-Nyquist high resolution radars with minimum number of samples. We also develop a low complexity recovery method, based on structured CS and propose a new …

基于压缩感知的高分辨率雷达

High-Resolution Radar via Compressed Sensing

本文提出一种将时频平面离散化为N×N网格的压缩感知雷达模型，利用目标稀疏性（K远小于N²）通过非相干脉冲和压缩感知技术重构目标场景，理论给出稀疏度K的上界，数值模拟验证了实际性能更优，相比传统雷达具有更高分辨率潜力。

Efficient adaptation of complex-valued noiselet sensing matrices for compressed single-pixel imaging

Minimal mutual coherence of discrete noiselets and Haar wavelets makes this pair of bases an essential choice for the measurement and compression matrices in compressed-sensing-based single-pixel detectors. In this paper we propose an efficient way of using complex-valued and non-binary noiselet functions for object sampling in single-pixel cameras with binary spatial light modulators and incoherent illumination. The proposed method allows to determine m complex noiselet coefficients from m+1 binary sampling measurements. Further, we introduce a modification to the complex fast noiselet transf…

Bilinear Compressed Sensing under known Signs via Convex Programming

We consider the bilinear inverse problem of recovering two vectors, $\boldsymbol{x} \in\mathbb{R}^L$ and $\boldsymbol{w} \in\mathbb{R}^L$, from their entrywise product. We consider the case where $\boldsymbol{x}$ and $\boldsymbol{w}$ have known signs and are sparse with respect to known dictionaries of size $K$ and $N$, respectively. Here, $K$ and $N$ may be larger than, smaller than, or equal to $L$. We introduce $\ell_1$-BranchHull, which is a convex program posed in the natural parameter space and does not require an approximate solution or initialization in order to be stated or solved. Un…

Kalman Filtered Compressed Sensing

We consider the problem of reconstructing time sequences of spatially sparse signals (with unknown and time-varying sparsity patterns) from a limited number of linear "incoherent" measurements, in real-time. The signals are sparse in some transform domain referred to as the sparsity basis. For a single spatial signal, the solution is provided by Compressed Sensing (CS). The question that we address is, for a sequence of sparse signals, can we do better than CS, if (a) the sparsity pattern of the signal's transform coefficients' vector changes slowly over time, and (b) a simple prior model on t…

The dynamics of message passing on dense graphs, with applications to compressed sensing

Approximate message passing algorithms proved to be extremely effective in reconstructing sparse signals from a small number of incoherent linear measurements. Extensive numerical experiments further showed that their dynamics is accurately tracked by a simple one-dimensional iteration termed state evolution. In this paper we provide the first rigorous foundation to state evolution. We prove that indeed it holds asymptotically in the large system limit for sensing matrices with independent and identically distributed gaussian entries. While our focus is on message passing algorithms for compre…

Compressed Sensing SAR Imaging with Multilook Processing

Multilook processing is a widely used speckle reduction approach in synthetic aperture radar (SAR) imaging. Conventionally, it is achieved by incoherently summing of some independent low-resolution images formulated from overlapping subbands of the SAR signal. However, in the context of compressive sensing (CS) SAR imaging, where the samples are collected at sub-Nyquist rate, the data spectrum is highly aliased that hinders the direct application of the existing multilook techniques. In this letter, we propose a new CS-SAR imaging method that can realize multilook processing simultaneously dur…

Minimization of Transformed $L_1$ Penalty: Closed Form Representation and Iterative Thresholding Algorithms

The transformed $l_1$ penalty (TL1) functions are a one parameter family of bilinear transformations composed with the absolute value function. When acting on vectors, the TL1 penalty interpolates $l_0$ and $l_1$ similar to $l_p$ norm ($p \in (0,1)$). In our companion paper, we showed that TL1 is a robust sparsity promoting penalty in compressed sensing (CS) problems for a broad range of incoherent and coherent sensing matrices. Here we develop an explicit fixed point representation for the TL1 regularized minimization problem. The TL1 thresholding functions are in closed form for all paramete…

稀疏二进制矩阵的压缩感知：近最优时间下的实例最优误差保证

Compressed Sensing with Sparse Binary Matrices: Instance Optimal Error Guarantees in Near-Optimal Time

提出一种使用近最优大小的稀疏二进制矩阵和亚线性时间恢复算法的压缩感知方法，在O((k log k) log N)时间内实现实例最优误差界。