Sparse signal recoveries from multiple measurement vectors (MMV) with joint sparsity property have many applications in signal, image, and video processing. The problem becomes much more involved when snapshots of the signal matrix are temporally correlated. With signal's temporal correlation in mind, we provide a framework of iterative MMV algorithms based on thresholding, functional feedback and null space tuning. Convergence analysis for exact recovery is established. Unlike most of iterative greedy algorithms that select indices in a measurement/solution space, we determine indices based on an orthogonal subspace spanned by the iterative sequence. In addition, a functional feedback that controls the amount of energy relocation from the "tails" is implemented and analyzed. It is seen that the principle of functional feedback is capable to lower the number of iteration and speed up the convergence of the algorithm. Numerical experiments demonstrate that the proposed algorithm has a clearly advantageous balance of efficiency, adaptivity and accuracy compared with other state-of-the-art algorithms.
We address the problem of sparse recovery using greedy compressed sensing recovery algorithms, without explicit knowledge of the sparsity. Estimating the sparsity order is a crucial problem in many practical scenarios, e.g., wireless communications, where exact value of the sparsity order of the unknown channel may be unavailable a priori. In this paper we have proposed a new greedy algorithm, referred to as the Multiple Choice Hard Thresholding Pursuit (MCHTP), which modifies the popular hard thresholding pursuit (HTP) suitably to iteratively recover the unknown sparse vector along with the s…
The algorithms based on the technique of optimal $k$-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal $k$-thresholding model, and thus the computational cos…
The hard thresholding technique plays a vital role in the development of algorithms for sparse signal recovery. By merging this technique and heavy-ball acceleration method which is a multi-step extension of the traditional gradient descent method, we propose the so-called heavy-ball-based hard thresholding (HBHT) and heavy-ball-based hard thresholding pursuit (HBHTP) algorithms for signal recovery. It turns out that the HBHT and HBHTP can successfully recover a $k$-sparse signal if the restricted isometry constant of the measurement matrix satisfies $δ_{3k}<0.618 $ and $δ_{3k}<0.577,$ respect…
We study here sparse recovery problems in the presence of additive noise. We analyze a thresholding version of the CoSaMP algorithm, named Thresholding Greedy Pursuit (TGP). We demonstrate that an appropriate choice of thresholding parameter, even without the knowledge of sparsity level of the signal and strength of the noise, can result in exact recovery with no false discoveries as the dimension of the data increases to infinity.
Newton-Direction-Based ReLU-Thresholding Methods for Nonnegative Sparse Signal Recovery
School of Mathematics and Statistics, Shandong University of Technology（山东理工大学数学与统计学院）；Shenzhen International Center for Industrial and Applied Mathematics, SRIBD, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院深圳市工业与应用数学中心）；Department of Mathematical Sciences, University of Nottingham Ningbo China（宁波诺丁汉大学数学系）
Optimal $k$-thresholding algorithms are a class of $k$-sparse signal recovery algorithms that overcome the shortcomings of traditional hard thresholding algorithms caused by the oscillation of the residual function. In this paper, a novel convergence analysis for optimal $k$-thresholding algorithms is established, which reveals the data-time tradeoffs of these algorithms. Both the analysis and numerical results demonstrate that when the number of measurements is small, the algorithms cannot converge; when the number of measurements is suitably large, the number of iterations required for succe…
This paper deals with the problem of sparse recovery often found in compressive sensing applications exploiting a priori knowledge. In particular, we present a knowledge-aided normalized iterative hard thresholding (KA-NIHT) algorithm that exploits information about the probabilities of nonzero entries. We also develop a strategy to update the probabilities using a recursive KA-NIHT (RKA-NIHT) algorithm, which results in improved recovery. Simulation results illustrate and compare the performance of the proposed and existing algorithms.
In this paper, we propose a general scale invariant approach for sparse signal recovery via the minimization of the $q$-ratio sparsity. When $1 < q \leq \infty$, both the theoretical analysis based on $q$-ratio constrained minimal singular values (CMSV) and the practical algorithms via nonlinear fractional programming are presented. Numerical experiments are conducted to demonstrate the advantageous performance of the proposed approaches over the state-of-the-art sparse recovery methods.
Samsung Research Institute（三星研究院）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Simultaneous orthogonal matching pursuit (SOMP) and block OMP (BOMP) are two widely used techniques for sparse support recovery in multiple measurement vector (MMV) and block sparse (BS) models respectively. For optimal performance, both SOMP and BOMP require \textit{a priori} knowledge of signal sparsity or noise variance. However, sparsity and noise variance are unavailable in most practical applications. This letter presents a novel technique called generalized residual ratio thresholding (GRRT) for operating SOMP and BOMP without the \textit{a priori} knowledge of signal sparsity and noise…
Indian Institute of Technology, Madras（印度理工学院马德拉斯分校）
In this paper we consider the problem of exact recovery of a fixed sparse vector with the measurement matrices sequentially arriving along with corresponding measurements. We propose an extension of the iterative hard thresholding (IHT) algorithm, termed as sequential IHT (SIHT) which breaks the total time horizon into several phases such that IHT is executed in each of these phases using a fixed measurement matrix obtained at the beginning of that phase. We consider a stochastic setting where the measurement matrices obtained at each phase are independent samples of a sub Gaussian random matr…
In this paper, we propose the family of Iterative Methods with Adaptive Thresholding (IMAT) for sparsity promoting reconstruction of Delta Modulated (DM) voice signals. We suggest a novel missing sampling approach to delta modulation that facilitates sparsity promoting reconstruction of the original signal from a subset of DM samples with less quantization noise. Utilizing our proposed missing sampling approach to delta modulation, we provide an analytical discussion on the convergence of IMAT for DM coding technique. We also modify the basic IMAT algorithm and propose the Iterative Method wit…
A simple hard-thresholding operation is shown to be able to recover $L$ signals $\mathbf{x}_1,...,\mathbf{x}_L \in \mathbb{R}^n$ that share a common support of size $s$ from $m = \mathcal{O}(s)$ one-bit measurements per signal if $L \ge \log(en/s)$. This result improves the single signal recovery bounds with $m = \mathcal{O}(s\log(en/s))$ measurements in the sense that asymptotically fewer measurements per non-zero entry are needed. Numerical evidence supports the theoretical considerations.
In oversampled adaptive sensing (OAS), noisy measurements are collected in multiple subframes. The sensing basis in each subframe is adapted according to some posterior information exploited from previous measurements. The framework is shown to significantly outperform the classic non-adaptive compressive sensing approach. This paper extends the notion of OAS to signals with structured sparsity. We develop a low-complexity OAS algorithm based on structured orthogonal sensing. Our investigations depict that the proposed algorithm outperforms the conventional non-adaptive compressive sensing fra…
We propose a simple modification to the iterative hard thresholding (IHT) algorithm, which recovers asymptotically sparser solutions as a function of the condition number. When aiming to minimize a convex function $f(x)$ with condition number $κ$ subject to $x$ being an $s$-sparse vector, the standard IHT guarantee is a solution with relaxed sparsity $O(sκ^2)$, while our proposed algorithm, regularized IHT, returns a solution with sparsity $O(sκ)$. Our algorithm significantly improves over ARHT which also finds a solution of sparsity $O(sκ)$, as it does not require re-optimization in each iter…
This paper investigates the problem of signal estimation from undersampled noisy sub-Gaussian measurements under the assumption of a cosparse model. Based on generalized notions of sparsity, we derive novel recovery guarantees for the $\ell^1$-analysis basis pursuit, enabling accurate predictions of its sample complexity. The corresponding bounds on the number of required measurements do explicitly depend on the Gram matrix of the analysis operator and therefore particularly account for its mutual coherence structure. Our findings defy conventional wisdom which promotes the sparsity of analysi…
In this paper, we propose a method for block sparse signal recovery that minimizes the block $q$-ratio sparsity $\left(\lVert z\rVert_{2,1}/\lVert z\rVert_{2,q}\right)^{\frac{q}{q-1}}$ with $q\in[0,\infty]$. For the case of $1<q\leq\infty$, we present the theoretical analyses and the computing algorithms for both cases of the $\ell_2$-bounded and $\ell_{2,\infty}$-bounded noises. The corresponding unconstrained model is also investigated. Its superior performance in block sparse signal reconstruction is demonstrated by numerical experiments.
In this paper we theoretically study exact recovery of sparse vectors from compressed measurements by minimizing a general nonconvex function that can be decomposed into the sum of single variable functions belonging to a class of smooth nonconvex sparsity promoting functions. Null space property (NSP) and restricted isometry property (RIP) are used as key theoretical tools. The notion of \emph{scale function} associated to a sparsity promoting function is introduced to generalize the state-of-the-art analysis technique of the $l_p$ minimization problem. The analysis is used to derive an upper…
