In many compressive sensing problems today, the relationship between the measurements and the unknowns could be nonlinear. Traditional treatment of such nonlinear relationships have been to approximate the nonlinearity via a linear model and the subsequent un-modeled dynamics as noise. The ability to more accurately characterize nonlinear models has the potential to improve the results in both existing compressive sensing applications and those where a linear approximation does not suffice, e.g., phase retrieval. In this paper, we extend the classical compressive sensing framework to a second-order Taylor expansion of the nonlinearity. Using a lifting technique and a method we call quadratic basis pursuit, we show that the sparse signal can be recovered exactly when the sampling rate is sufficiently high. We further present efficient numerical algorithms to recover sparse signals in second-order nonlinear systems, which are considerably more difficult to solve than their linear counterparts in sparse optimization.
In this correspondence, we obtain exact recovery conditions for regularized modified basis pursuit (reg-mod-BP) and discuss when the obtained conditions are weaker than those for modified-CS or for basis pursuit (BP). The discussion is also supported by simulation comparisons. Reg-mod-BP provides a solution to the sparse recovery problem when both an erroneous estimate of the signal's support, denoted by $T$, and an erroneous estimate of the signal values on $T$ are available.
The number of available algorithms for the so-called Basis Pursuit Denoising problem (or the related LASSO-problem) is large and keeps growing. Similarly, the number of experiments to evaluate and compare these algorithms on different instances is growing. In this note, we present a method to produce instances with exact solutions which is based on a simple observation which is related to the so called source condition from sparse regularization.
Inspired by alternating direction method of multipliers and the idea of operator splitting, we propose a efficient algorithm for solving large-scale quadratically constrained basis pursuit. Experimental results show that the proposed algorithm can achieve 50~~100 times speedup when compared with the baseline interior point algorithm implemented in CVX.
In compressive sensing, the basis pursuit algorithm aims to find the sparsest solution to an underdetermined linear equation system. In this paper, we generalize basis pursuit to finding the sparsest solution to higher order nonlinear systems of equations, called nonlinear basis pursuit. In contrast to the existing nonlinear compressive sensing methods, the new algorithm that solves the nonlinear basis pursuit problem is convex and not greedy. The novel algorithm enables the compressive sensing approach to be used for a broader range of applications where there are nonlinear relationships betw…
In compressive sensing (CS) theory, as the number of samples is decreased below a minimum threshold, the average error of the recovery increases. Sufficient sampling is either required for quality reconstruction or the error is resignedly accepted. However, most CS work has not taken advantage of the inherent structure in a variety of signals relevant to engineering applications. Hence, this paper proposes a new method of recovery built on basis pursuit (BP), called Structure-Constrained Basis Pursuit (SCBP), that constrains signals based on known structure rather than through extra sampling. …
Signal processing tasks as fundamental as sampling, reconstruction, minimum mean-square error interpolation and prediction can be viewed under the prism of reproducing kernel Hilbert spaces. Endowing this vantage point with contemporary advances in sparsity-aware modeling and processing, promotes the nonparametric basis pursuit advocated in this paper as the overarching framework for the confluence of kernel-based learning (KBL) approaches leveraging sparse linear regression, nuclear-norm regularization, and dictionary learning. The novel sparse KBL toolbox goes beyond translating sparse param…
In this work, we study the problem of reconstructing a sparse signal from a limited number of linear 'incoherent' noisy measurements, when a part of its support is known. The known part of the support may be available from prior knowledge or from the previous time instant (in applications requiring recursive reconstruction of a time sequence of sparse signals, e.g. dynamic MRI). We study a modification of Basis Pursuit Denoising (BPDN) and bound its reconstruction error. A key feature of our work is that the bounds that we obtain are computable. Hence, we are able to use Monte Carlo to study t…
This paper focuses on the estimation of low-complexity signals when they are observed through $M$ uniformly quantized compressive observations. Among such signals, we consider 1-D sparse vectors, low-rank matrices, or compressible signals that are well approximated by one of these two models. In this context, we prove the estimation efficiency of a variant of Basis Pursuit Denoise, called Consistent Basis Pursuit (CoBP), enforcing consistency between the observations and the re-observed estimate, while promoting its low-complexity nature. We show that the reconstruction error of CoBP decays li…
We consider the recovery of a (real- or complex-valued) signal from magnitude-only measurements, known as phase retrieval. We formulate phase retrieval as a convex optimization problem, which we call PhaseMax. Unlike other convex methods that use semidefinite relaxation and lift the phase retrieval problem to a higher dimension, PhaseMax is a "non-lifting" relaxation that operates in the original signal dimension. We show that the dual problem to PhaseMax is Basis Pursuit, which implies that phase retrieval can be performed using algorithms initially designed for sparse signal recovery. We dev…
This work considers recovery of signals that are sparse over two bases. For instance, a signal might be sparse in both time and frequency, or a matrix can be low rank and sparse simultaneously. To facilitate recovery, we consider minimizing the sum of the $\ell_1$-norms that correspond to each basis, which is a tractable convex approach. We find novel optimality conditions which indicates a gain over traditional approaches where $\ell_1$ minimization is done over only one basis. Next, we analyze these optimality conditions for the particular case of time-frequency bases. Denoting sparsity in t…
For massive multiple-input multiple-output (MIMO) systems operating in frequency-division duplex mode, downlink channel state information (CSI) acquisition will incur large overhead. This overhead is substantially reduced when sparse channel estimation techniques are employed, owing to the channel sparsity in the angular domain. When a sparse channel estimation method is implemented, the measurement matrix, which is related to the pilot matrix, is essential to the channel estimation performance. Existing sparse channel estimation schemes widely adopt random measurement matrices, which have bee…
We present a theoretical analysis of the average performance of OMP for sparse approximation. For signals that are generated from a dictionary with $K$ atoms and coherence $μ$ and coefficients corresponding to a geometric sequence with parameter $α<1$, we show that OMP is successful with high probability as long as the sparsity level $S$ scales as $Sμ^2 \log K \lesssim 1-α$. This improves by an order of magnitude over worst case results and shows that OMP and its famous competitor Basis Pursuit outperform each other depending on the setting.
It is well known that $\ell_1$ minimization can be used to recover sufficiently sparse unknown signals from compressed linear measurements. In fact, exact thresholds on the sparsity, as a function of the ratio between the system dimensions, so that with high probability almost all sparse signals can be recovered from i.i.d. Gaussian measurements, have been computed and are referred to as "weak thresholds" \cite{D}. In this paper, we introduce a reweighted $\ell_1$ recovery algorithm composed of two steps: a standard $\ell_1$ minimization step to identify a set of entries where the signal is li…
Sampling rate is the bottleneck for spectrum sensing over multi-GHz bandwidth. Recent progress in compressed sensing (CS) initialized several sub-Nyquist rate approaches to overcome the problem. However, efforts to design CS reconstruction algorithms for wideband spectrum sensing are very limited. It is possible to further reduce the sampling rate requirement and improve reconstruction performance via algorithms considering prior knowledge of cognitive radio spectrum usages. In this paper, we group the usages of cognitive radio spectrum into three categories and propose a modified orthogonal m…
Johns Hopkins University（约翰斯·霍普金斯大学）；Technical University of Munich（慕尼黑工业大学）；KU Eichstätt–Ingolstadt（埃希施泰特-英戈尔施塔特天主教大学）
The recovery of sparse data is at the core of many applications in machine learning and signal processing. While such problems can be tackled using $\ell_1$-regularization as in the LASSO estimator and in the Basis Pursuit approach, specialized algorithms are typically required to solve the corresponding high-dimensional non-smooth optimization for large instances. Iteratively Reweighted Least Squares (IRLS) is a widely used algorithm for this purpose due its excellent numerical performance. However, while existing theory is able to guarantee convergence of this algorithm to the minimizer, it …
Instituto de Sistemas e Robótica (ISR), Instituto Superior Técnico (IST), Technical University of Lisbon, Portugal（里斯本理工大学高等技术学院系统机器人研究所）；Department of Electrical and Computer Engineering at Carnegie Mellon University, USA（卡内基梅隆大学电气与计算机工程系）；Department of Computer Science at ETH Zurich, Switzerland（苏黎世联邦理工学院计算机科学系）
Stanford University（斯坦福大学）；The Chinese University of Hong Kong（香港中文大学）
Motivated by the philosophy and phenomenal success of compressed sensing, the problem of reconstructing a matrix from a sampling of its entries has attracted much attention recently. Such a problem can be viewed as an information-theoretic variant of the well-studied matrix completion problem, and the main objective is to design an efficient algorithm that can reconstruct a matrix by inspecting only a small number of its entries. Although this is an impossible task in general, Candès and co-authors have recently shown that under a so-called incoherence assumption, a rank $r$ $n\times n$ matrix…
This paper is about iteratively reweighted basis-pursuit algorithms for compressed sensing and matrix completion problems. In a first part, we give a theoretical explanation of the fact that reweighted basis pursuit can improve a lot upon basis pursuit for exact recovery in compressed sensing. We exhibit a condition that links the accuracy of the weights to the RIP and incoherency constants, which ensures exact recovery. In a second part, we introduce a new algorithm for matrix completion, based on the idea of iterative reweighting. Since a weighted nuclear "norm" is typically non-convex, it c…
In this paper we present a linear programming solution for sign pattern recovery of a sparse signal from noisy random projections of the signal. We consider two types of noise models, input noise, where noise enters before the random projection; and output noise, where noise enters after the random projection. Sign pattern recovery involves the estimation of sign pattern of a sparse signal. Our idea is to pretend that no noise exists and solve the noiseless $\ell_1$ problem, namely, $\min \|β\|_1 ~ s.t. ~ y=G β$ and quantizing the resulting solution. We show that the quantized solution perfect…
Sparse signal recovery from a small number of random measurements is a well known NP-hard to solve combinatorial optimization problem, with important applications in signal and image processing. The standard approach to the sparse signal recovery problem is based on the basis pursuit method. This approach requires the solution of a large convex optimization problem, and therefore suffers from high computational complexity. Here, we discuss a stochastic optimization method, as a low-complexity alternative to the basis pursuit approach.
Compressed sensing is a developing field aiming at reconstruction of sparse signals acquired in reduced dimensions, which make the recovery process under-determined. The required solution is the one with minimum $\ell_0$ norm due to sparsity, however it is not practical to solve the $\ell_0$ minimization problem. Commonly used techniques include $\ell_1$ minimization, such as Basis Pursuit (BP) and greedy pursuit algorithms such as Orthogonal Matching Pursuit (OMP) and Subspace Pursuit (SP). This manuscript proposes a novel semi-greedy recovery approach, namely A* Orthogonal Matching Pursuit (…
We discuss a strategy of sparse approximation that is based on the use of an overcomplete basis, and evaluate its performance when a random matrix is used as this basis. A small combination of basis vectors is chosen from a given overcomplete basis, according to a given compression rate, such that they compactly represent the target data with as small a distortion as possible. As a selection method, we study the $\ell_0$- and $\ell_1$-based methods, which employ the exhaustive search and $\ell_1$-norm regularization techniques, respectively. The performance is assessed in terms of the trade-of…
Recently, the 1-bit compressive sensing (1-bit CS) has been studied in the field of sparse signal recovery. Since the amplitude information of sparse signals in 1-bit CS is not available, it is often the support or the sign of a signal that can be exactly recovered with a decoding method. In this paper, we first show that a necessary assumption (that has been overlooked in the literature) should be made for some existing theories and discussions for 1-bit CS. Without such an assumption, the found solution by some existing decoding algorithms might be inconsistent with 1-bit measurements. This …
The convolutional sparse model has recently gained increasing attention in the signal and image processing communities, and several methods have been proposed for solving the pursuit problem emerging from it -- in particular its convex relaxation, Basis Pursuit. In the first of this two-part work, we have provided a theoretical back-bone for this model, providing guarantees for the uniqueness of the sparsest solution and for the success of pursuit algorithms by introducing the notion of stripe sparsity and other related measures. Herein, we extend the analysis to a noisy regime, thereby consid…
An l0-regularized linear regression for a sparse signal reconstruction is implemented based on the quadratic unconstrained binary optimization (QUBO) formulation. In this method, the signal values are quantized and expressed as bit sequences. By transforming l0-norm to a quadratic form of these bits, the fully quadratic objective function is provided and optimized by the solver specialized for QUBO, such as the quantum annealer. Numerical experiments with a commercial quantum annealer show that the proposed method performs slightly better than conventional methods based on orthogonal matching …
The theory of Compressed Sensing, the emerging sampling paradigm 'that goes against the common wisdom', asserts that 'one can recover signals in Rn from far fewer samples or measurements, if the signal has a sparse representation in some orthonormal basis', from m = O(klogn), k<< n nonadaptive measurements . The accuracy of the recovered signal is 'as good as that attainable with direct knowledge of the k most important coefficients and its locations'. Moreover, a good approximation to those important coefficients is extracted from the measurements by solving a L1 minimization problem viz. Bas…
Intel Labs（英特尔实验室）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
A reliable support detection is essential for a greedy algorithm to reconstruct a sparse signal accurately from compressed and noisy measurements. This paper proposes a novel support detection method for greedy algorithms, which is referred to as "\textit{maximum a posteriori (MAP) support detection}". Unlike existing support detection methods that identify support indices with the largest correlation value in magnitude per iteration, the proposed method selects them with the largest likelihood ratios computed under the true and null support hypotheses by simultaneously exploiting the distribu…
An algorithmic framework, based on the difference of convex functions algorithm (DCA), is proposed for minimizing a class of concave sparse metrics for compressed sensing problems. The resulting algorithm iterates a sequence of $\ell_1$ minimization problems. An exact sparse recovery theory is established to show that the proposed framework always improves on the basis pursuit ($\ell_1$ minimization) and inherits robustness from it. Numerical examples on success rates of sparse solution recovery illustrate further that, unlike most existing non-convex compressed sensing solvers in the literatu…
Motivated by the question of optimal functional approximation via compressed sensing, we propose generalizations of the Iterative Hard Thresholding and the Compressive Sampling Matching Pursuit algorithms able to promote sparse in levels signals. We show, by means of numerical experiments, that the proposed algorithms are successfully able to outperform their unstructured variants when the signal exhibits the sparsity structure of interest. Moreover, in the context of piecewise smooth function approximation, we numerically demonstrate that the structure promoting decoders outperform their unst…
In this work we address the problem of recovering sparse solutions to non linear inverse problems. We look at two variants of the basic problem, the synthesis prior problem when the solution is sparse and the analysis prior problem where the solution is cosparse in some linear basis. For the first problem, we propose non linear variants of the Orthogonal Matching Pursuit (OMP) and CoSamp algorithms; for the second problem we propose a non linear variant of the Greedy Analysis Pursuit (GAP) algorithm. We empirically test the success rates of our algorithms on exponential and logarithmic functio…
Orthogonal Matching Pursuit (OMP) is a simple, yet empirically competitive algorithm for sparse recovery. Recent developments have shown that OMP guarantees exact recovery of K-sparse signals with K or more than K iterations if the observation matrix satisfies the restricted isometry property (RIP) with some conditions. We develop RIP-based online guarantees for recovery of a K-sparse signal with more than K OMP iterations. Though these guarantees cannot be generalized to all sparse signals a priori, we show that they can still hold online when the state-of-the-art K-step recovery guarantees f…
In this paper, we continue our previous work on the reduction of algebraic lattices over imaginary quadratic fields for the special case when the lattice is spanned over a two dimensional basis. In particular, we show that the algebraicvariant of Gauss algorithm returns a basis that corresponds to the successive minima of the lattice in polynomial time if the chosen ring is Euclidean.
This short note proves the $\ell_2-\ell_1$ instance optimality of a $\ell_1/\ell_1$ solver, i.e a variant of \emph{basis pursuit denoising} with a $\ell_1$ fidelity constraint, when applied to the estimation of sparse (or compressible) signals observed by sparsely corrupted compressive measurements. The approach simply combines two known results due to Y. Plan, R. Vershynin and E. Candès.
This article extends the concept of compressed sensing to signals that are not sparse in an orthonormal basis but rather in a redundant dictionary. It is shown that a matrix, which is a composition of a random matrix of certain type and a deterministic dictionary, has small restricted isometry constants. Thus, signals that are sparse with respect to the dictionary can be recovered via Basis Pursuit from a small number of random measurements. Further, thresholding is investigated as recovery algorithm for compressed sensing and conditions are provided that guarantee reconstruction with high pro…
In this paper, we discuss the generalized Hamming weights of a class of linear codes associated with non-degenerate quadratic forms. In order to do so, we study the quadratic forms over subspaces of finite field and obtain some interesting results about subspaces and their dual spaces. On this basis, we solve all the generalized Hamming weights of these linear codes.
