RIP-based performance guarantee for low-tubal-rank tensor recovery
School of Mathematics and Statistics, Southwest University（西南大学数学与统计学院）；Xi’an Jiaotong University（西安交通大学）；College of Artificial Intelligence, Southwest University（西南大学人工智能学院）
Feng Zhang, Wendong Wang, Jianwen Huang, Yao Wang, Jianjun Wang
The essential task of tensor data analysis focuses on the tensor decomposition and the corresponding notion of rank. In this paper, by introducing the notion of tensor Singular Value Decomposition (t-SVD), we establish a Regularized Tensor Nuclear Norm Minimization (RTNNM) model for low-tubal-rank tensor recovery. As we know that many variants of the Restricted Isometry Property (RIP) have proven to be crucial analysis tools for sparse recovery. In the t-SVD framework, we initiatively define a novel tensor Restricted Isometry Property (t-RIP). Furthermore, we show that any third-order tensor $\boldsymbol{\mathcal{X}}$ can stably be recovered from few linear noise measurements under some certain t-RIP conditions via the RTNNM model. We note that, as far as the authors are aware, such kind of result has not previously been reported in the literature.
Robust Low-Tubal-Rank Tensor Completion under Cross-Concentrated Sampling
University of Central Florida（中佛罗里达大学）；Michigan State University（密歇根州立大学）；University of Kentucky（肯塔基大学）；University of Texas MD Anderson Cancer Center（德克萨斯大学 MD 安德森癌症中心）
The low-tubal-rank tensor model has been recently proposed for real-world multidimensional data. In this paper, we study the low-tubal-rank tensor completion problem, i.e., to recover a third-order tensor by observing a subset of its elements selected uniformly at random. We propose a fast iterative algorithm, called {\em Tubal-Alt-Min}, that is inspired by a similar approach for low-rank matrix completion. The unknown low-tubal-rank tensor is represented as the product of two much smaller tensors with the low-tubal-rank property being automatically incorporated, and Tubal-Alt-Min alternates b…
This paper establishes a sharp condition on the restricted isometry property (RIP) for both the sparse signal recovery and low-rank matrix recovery. It is shown that if the measurement matrix $A$ satisfies the RIP condition $δ_k^A<1/3$, then all $k$-sparse signals $β$ can be recovered exactly via the constrained $\ell_1$ minimization based on $y=Aβ$. Similarly, if the linear map $\cal M$ satisfies the RIP condition $δ_r^{\cal M}<1/3$, then all matrices $X$ of rank at most $r$ can be recovered exactly via the constrained nuclear norm minimization based on $b={\cal M}(X)$. Furthermore, in both c…
In this paper, we consider the recovery of block sparse signals, whose nonzero entries appear in blocks (or clusters) rather than spread arbitrarily throughout the signal, from incomplete linear measurement. A high order sufficient condition based on block RIP is obtained to guarantee the stable recovery of all block sparse signals in the presence of noise, and robust recovery when signals are not exactly block sparse via mixed $l_{2}/l_{1}$ minimization. Moreover, a concrete example is established to ensure the condition is sharp. The significance of the results presented in this paper lies i…
This paper conducts a rigorous analysis for provable estimation of multidimensional arrays, in particular third-order tensors, from a random subset of its corrupted entries. Our study rests heavily on a recently proposed tensor algebraic framework in which we can obtain tensor singular value decomposition (t-SVD) that is similar to the SVD for matrices, and define a new notion of tensor rank referred to as the tubal rank. We prove that by simply solving a convex program, which minimizes a weighted combination of tubal nuclear norm, a convex surrogate for the tubal rank, and the $\ell_1$-norm, …
Many inverse problems in signal processing deal with the robust estimation of unknown data from underdetermined linear observations. Low dimensional models, when combined with appropriate regularizers, have been shown to be efficient at performing this task. Sparse models with the 1-norm or low rank models with the nuclear norm are examples of such successful combinations. Stable recovery guarantees in these settings have been established using a common tool adapted to each case: the notion of restricted isometry property (RIP). In this paper, we establish generic RIP-based guarantees for the …
We make a trivial modification to the elegant analysis of Garg and Khandekar (\emph{Gradient Descent with Sparsification} ICML 2009) that replaces the standard Restricted Isometry Property (RIP), with another RIP-type property (which could be simpler than the RIP, but we are not sure; it could be as hard as the RIP to check, thereby rendering this little writeup totally worthless).
This paper presents a new analysis for the orthogonal matching pursuit (OMP) algorithm. It is shown that if the restricted isometry property (RIP) is satisfied at sparsity level $O(\bar{k})$, then OMP can recover a $\bar{k}$-sparse signal in 2-norm. For compressed sensing applications, this result implies that in order to uniformly recover a $\bar{k}$-sparse signal in $\Real^d$, only $O(\bar{k} \ln d)$ random projections are needed. This analysis improves earlier results on OMP that depend on stronger conditions such as mutual incoherence that can only be satisfied with $Ω(\bar{k}^2 \ln d)$ ra…
In this paper we study the recovery conditions of weighted $l_{1}$ minimization for signal reconstruction from incomplete linear measurements when partial prior support information is available. We obtain that a high order RIP condition can guarantee stable and robust recovery of signals in bounded $l_{2}$ and Dantzig selector noise settings. Meanwhile, we not only prove that the sufficient recovery condition of weighted $l_{1}$ minimization method is weaker than that of standard $l_{1}$ minimization method, but also prove that weighted $l_{1}$ minimization method provides better upper bounds …
Restricted Isometry Property (RIP) is of fundamental importance in the theory of compressed sensing and forms the base of many exact and robust recovery guarantees in this field. A quantitative description of RIP involves bounding the so-called RIP constants of measurement matrices. In this respect, it is noteworthy that most results in the literature concerning RIP are upper bounds of RIP constants, which can be interpreted as a theoretical guarantee of successful sparse recovery. On the contrary, the land of lower bounds for RIP constants remains uncultivated. Lower bounds of RIP constants, …
The most frequently used condition for sampling matrices employed in compressive sampling is the restricted isometry (RIP) property of the matrix when restricted to sparse signals. At the same time, imposing this condition makes it difficult to find explicit matrices that support recovery of signals from sketches of the optimal (smallest possible)dimension. A number of attempts have been made to relax or replace the RIP property in sparse recovery algorithms. We focus on the relaxation under which the near-isometry property holds for most rather than for all submatrices of the sampling matrix,…
This paper concentrates on the recovery of block-sparse signals, which is not only sparse but also nonzero elements are arrayed into some blocks (clusters) rather than being arbitrary distributed all over the vector, from linear measurements. We establish high-order sufficient conditions based on block RIP to ensure the exact recovery of every block $s$-sparse signal in the noiseless case via mixed $l_2/l_p$ minimization method, and the stable and robust recovery in the case that signals are not accurately block-sparse in the presence of noise. Additionally, a lower bound on necessary number o…
A sufficient condition reported very recently for perfect recovery of a K-sparse vector via orthogonal matching pursuit (OMP) in K iterations is that the restricted isometry constant of the sensing matrix satisfies delta_K+1<1/(sqrt(delta_K+1)+1). By exploiting an approximate orthogonality condition characterized via the achievable angles between two orthogonal sparse vectors upon compression, this paper shows that the upper bound on delta can be further relaxed to delta_K+1<(sqrt(1+4*delta_K+1)-1)/(2K).This result thus narrows the gap between the so far best known bound and the ultimate perfo…
We consider the problem of recovering sparse vectors from underdetermined linear measurements via $\ell_p$-constrained basis pursuit. Previous analyses of this problem based on generalized restricted isometry properties have suggested that two phenomena occur if $p\neq 2$. First, one may need substantially more than $s \log(en/s)$ measurements (optimal for $p=2$) for uniform recovery of all $s$-sparse vectors. Second, the matrix that achieves recovery with the optimal number of measurements may not be Gaussian (as for $p=2$). We present a new, direct analysis which shows that in fact neither o…
We study the recovery conditions of weighted mixed $\ell_2/\ell_p\,(0<p\leq 1)$ minimization for block sparse signal reconstruction from compressed measurements when partial block support information is available. We show that the block $p$-restricted isometry property (RIP) can ensure the robust recovery. Moreover, we present the sufficient and necessary condition for the recovery by using weighted block $p$-null space property. The relationship between the block $p$-RIP and the weighted block $p$-null space property has been established. Finally, we illustrate our results with a series of nu…
In this paper, we use the block orthogonal matching pursuit (BOMP) algorithm to recover block sparse signals $\x$ from measurements $\y=\A\x+\v$, where $\v$ is an $\ell_2$-bounded noise vector (i.e., $\|\v\|_2\leq ε$ for some constant $ε$). We investigate some sufficient conditions based on the block restricted isometry property (block-RIP) for exact (when $\v=\0$) and stable (when $\v\neq\0$) recovery of block sparse signals $\x$. First, on the one hand, we show that if $\A$ satisfies the block-RIP with $δ_{K+1}<1/\sqrt{K+1}$, then every block $K$-sparse signal $\x$ can be exactly or stably r…
The restricted isometry property (RIP) has become well-known in the compressed sensing community. Recently, a weaken version of RIP was proposed for exact sparse recovery under weak moment assumptions. In this note, we prove that the weaken RIP is also sufficient for \textsl{stable and robust} sparse recovery by linking it with a recently introduced robust width property in compressed sensing. Moreover, we show that it can be widely apply to other compressed sensing instances as well.
Direction of Arrival (DOA) estimation of mixed uncorrelated and coherent sources is a long existing challenge in array signal processing. Application of compressive sensing to array signal processing has opened up an exciting class of algorithms. The authors investigated the application of orthogonal matching pursuit (OMP) for direction of Arrival (DOA) estimation for different scenarios, especially to tackle the case of coherent sources and observed inconsistencies in the results. In this paper, a modified OMP algorithm is proposed to overcome these deficiencies by exploiting maximum variance…
We address some theoretical guarantees for Schatten-$p$ quasi-norm minimization ($p \in (0,1]$) in recovering low-rank matrices from compressed linear measurements. Firstly, using null space properties of the measurement operator, we provide a sufficient condition for exact recovery of low-rank matrices. This condition guarantees unique recovery of matrices of ranks equal or larger than what is guaranteed by nuclear norm minimization. Secondly, this sufficient condition leads to a theorem proving that all restricted isometry property (RIP) based sufficient conditions for $\ell_p$ quasi-norm mi…
Matrix recovery is the problem of recovering a low-rank matrix from a few linear measurements. Recently, this problem has gained a lot of attention as it is employed in many applications such as Netflix prize problem, seismic data interpolation and collaborative filtering. In these applications, one might access to additional prior information about the column and row spaces of the matrix. These extra information can potentially enhance the matrix recovery performance. In this paper, we propose an efficient greedy algorithm that exploits prior information in the recovery procedure. The perform…
Orthogonal Matching Pursuit (OMP) is a simple, yet empirically competitive algorithm for sparse recovery. Recent developments have shown that OMP guarantees exact recovery of K-sparse signals with K or more than K iterations if the observation matrix satisfies the restricted isometry property (RIP) with some conditions. We develop RIP-based online guarantees for recovery of a K-sparse signal with more than K OMP iterations. Though these guarantees cannot be generalized to all sparse signals a priori, we show that they can still hold online when the state-of-the-art K-step recovery guarantees f…
In this paper, we put forth a new joint sparse recovery algorithm called signal space matching pursuit (SSMP). The key idea of the proposed SSMP algorithm is to sequentially investigate the support of jointly sparse vectors to minimize the subspace distance to the residual space. Our performance guarantee analysis indicates that SSMP accurately reconstructs any row $K$-sparse matrix of rank $r$ in the full row rank scenario if the sampling matrix $\mathbf{A}$ satisfies $\text{krank}(\mathbf{A}) \ge K+1$, which meets the fundamental minimum requirement on $\mathbf{A}$ to ensure exact recovery. …
Compressed sensing (CS) theory considers the restricted isometry property (RIP) as a sufficient condition for measurement matrix which guarantees the recovery of any sparse signal from its compressed measurements. The RIP condition also preserves enough information for classification of sparse symbols, even with fewer measurements. In this work, we utilize RIP bound as the cost function for training a simple neural network in order to exploit the near optimal measurements or equivalently near optimal features for classification of a known set of sparse symbols. As an example, we consider demod…
Inspired by an interesting idea of Cai and Zhang, we formulate and prove the convex $k$-sparse decomposition of vectors which is invariant with respect to $\ell_1$ norm. This result fits well in discussing compressed sensing problems under RIP, but we believe it also has independent interest. As an application, a simple derivation of the RIP recovery condition $δ_k+θ_{k,k} < 1$ is presented.
Orthogonal least squares (OLS) is a classic algorithm for sparse recovery, function approximation, and subset selection. In this paper, we analyze the performance guarantee of the OLS algorithm. Specifically, we show that OLS guarantees the exact reconstruction of any $K$-sparse vector in $K$ iterations, provided that a sensing matrix has unit $\ell_{2}$-norm columns and satisfies the restricted isometry property (RIP) of order $K+1$ with \begin{align*} δ_{K+1} &<C_{K} = \begin{cases} \frac{1}{\sqrt{K}}, & K=1, \\ \frac{1}{\sqrt{K+\frac{1}{4}}}, & K=2, \\ \frac{1}{\sqrt{K+\frac{1}{16}}}, & K=3…
We formulate a unified framework for the separation of signals that are sparse in "morphologically" different redundant dictionaries. This formulation incorporates the so-called "analysis" and "synthesis" approaches as special cases and contains novel hybrid setups. We find corresponding coherence-based recovery guarantees for an l1-norm based separation algorithm. Our results recover those reported in Studer and Baraniuk, ACHA, submitted, for the synthesis setting, provide new recovery guarantees for the analysis setting, and form a basis for comparing performance in the analysis and synthesi…
In this paper, we study joint network coding and distributed source coding of inter-node dependent messages, with the perspective of compressed sensing. Specifically, the theoretical guarantees for robust $\ell_1$-min recovery of an under-determined set of linear network coded sparse messages are investigated. We discuss the guarantees for $\ell_1$-min decoding of quantized network coded messages, using the proposed local network coding coefficients in \cite{naba}, based on Restricted Isometry Property (RIP) of the resulting measurement matrix. Moreover, the relation between tail probability o…
This paper presents several novel theoretical results regarding the recovery of a low-rank matrix from just a few measurements consisting of linear combinations of the matrix entries. We show that properly constrained nuclear-norm minimization stably recovers a low-rank matrix from a constant number of noisy measurements per degree of freedom; this seems to be the first result of this nature. Further, the recovery error from noisy data is within a constant of three targets: 1) the minimax risk, 2) an oracle error that would be available if the column space of the matrix were known, and 3) a mo…
In this paper we consider the problem of recovering a high dimensional data matrix from a set of incomplete and noisy linear measurements. We introduce a new model that can efficiently restrict the degrees of freedom of the problem and is generic enough to find a lot of applications, for instance in multichannel signal compressed sensing (e.g. sensor networks, hyperspectral imaging) and compressive sparse principal component analysis (s-PCA). We assume data matrices have a simultaneous low-rank and joint sparse structure, and we propose a novel approach for efficient compressed sensing (CS) of…
We study the problem of reconstructing an unknown matrix M of rank r and dimension d using O(rd poly log d) Pauli measurements. This has applications in quantum state tomography, and is a non-commutative analogue of a well-known problem in compressed sensing: recovering a sparse vector from a few of its Fourier coefficients. We show that almost all sets of O(rd log^6 d) Pauli measurements satisfy the rank-r restricted isometry property (RIP). This implies that M can be recovered from a fixed ("universal") set of Pauli measurements, using nuclear-norm minimization (e.g., the matrix Lasso), with…
Indoor localization is a supporting technology for a broadening range of pervasive wireless applications. One promis- ing approach is to locate users with radio frequency fingerprints. However, its wide adoption in real-world systems is challenged by the time- and manpower-consuming site survey process, which builds a fingerprint database a priori for localization. To address this problem, we visualize the 3-D RF fingerprint data as a function of locations (x-y) and indices of access points (fingerprint), as a tensor and use tensor algebraic methods for an adaptive tubal-sampling of this finge…
