This paper studies the problem of completing a low-rank matrix from a few of its random entries with the aid of prior information. We suggest a strategy to incorporate prior information into the standard matrix completion procedure by maximizing the correlation between the original signal and the prior information. We also establish performance guarantees for the proposed method, which show that with suitable prior information, the proposed procedure can reduce the sample complexity of the standard matrix completion by a logarithmic factor. To illustrate the theory, we further analyze an important practical application where the prior subspace information is available. Both synthetic and real-world experiments are provided to verify the validity of the theory.
Matrix completion refers to completing a low-rank matrix from a few observed elements of its entries and has been known as one of the significant and widely-used problems in recent years. The required number of observations for exact completion is directly proportional to rank and the coherency parameter of the matrix. In many applications, there might exist additional information about the low-rank matrix of interest. For example, in collaborative filtering, Netflix and dynamic channel estimation in communications, extra subspace information is available. More precisely in these applications,…
We describe a new algorithm, termed subspace evolution and transfer (SET), for solving low-rank matrix completion problems. The algorithm takes as its input a subset of entries of a low-rank matrix, and outputs one low-rank matrix consistent with the given observations. The completion task is accomplished by searching for a column space on the Grassmann manifold that matches the incomplete observations. The SET algorithm consists of two parts -- subspace evolution and subspace transfer. In the evolution part, we use a gradient descent method on the Grassmann manifold to refine our estimate of …
An incoherent low-rank matrix can be efficiently reconstructed after observing a few of its entries at random, and then solving a convex program that minimizes the nuclear norm. In many applications, in addition to these entries, potentially valuable prior knowledge about the column and row spaces of the matrix is also available to the practitioner. In this paper, we incorporate this prior knowledge in matrix completion---by minimizing a weighted nuclear norm---and precisely quantify any improvements. In particular, we find in theory that reliable prior knowledge reduces the sample complexity …
In this short note we extend some of the recent results on matrix completion under the assumption that the columns of the matrix can be grouped (clustered) into subspaces (not necessarily disjoint or independent). This model deviates from the typical assumption prevalent in the literature dealing with compression and recovery for big-data applications. The results have a direct bearing on the problem of subspace clustering under missing or incomplete information.
Boston University（波士顿大学）；University of Wisconsin - Madison（威斯康星大学麦迪逊分校）
This paper considers the problem of completing a matrix with many missing entries under the assumption that the columns of the matrix belong to a union of multiple low-rank subspaces. This generalizes the standard low-rank matrix completion problem to situations in which the matrix rank can be quite high or even full rank. Since the columns belong to a union of subspaces, this problem may also be viewed as a missing-data version of the subspace clustering problem. Let X be an n x N matrix whose (complete) columns lie in a union of at most k subspaces, each of rank <= r < n, and assume N >> kn.…
Ptychography, a special case of the phase retrieval problem, is a popular method in modern imaging. Its measurements are based on the shifts of a locally supported window function. In general, direct recovery of an object from such measurements is known to be an ill-posed problem. Although for some windows the conditioning can be controlled, for a number of important cases it is not possible, for instance for Gaussian windows. In this paper we develop a subspace completion algorithm, which enables stable reconstruction for a much wider choice of windows, including Gaussian windows. The combina…
A new algorithm, termed subspace evolution and transfer (SET), is proposed for solving the consistent matrix completion problem. In this setting, one is given a subset of the entries of a low-rank matrix, and asked to find one low-rank matrix consistent with the given observations. We show that this problem can be solved by searching for a column space that matches the observations. The corresponding algorithm consists of two parts -- subspace evolution and subspace transfer. In the evolution part, we use a line search procedure to refine the column space. However, line search is not guarantee…
We study online robust matrix completion on graphs. At each iteration a vector with some entries missing is revealed and our goal is to reconstruct it by identifying the underlying low-dimensional subspace from which the vectors are drawn. We assume there is an underlying graph structure to the data, that is, the components of each vector correspond to nodes of a certain (known) graph, and their values are related accordingly. We give algorithms that exploit the graph to reconstruct the incomplete data, even in the presence of outlier noise. The theoretical properties of the algorithms are stu…
Many real world data sets exhibit an embedding of low-dimensional structure in a high-dimensional manifold. Examples include images, videos and internet traffic data. It is of great significance to reduce the storage requirements and computational complexity when the data dimension is high. Therefore we consider the problem of reconstructing a data stream from a small subset of its entries, where the data is assumed to lie in a low-dimensional linear subspace, possibly corrupted by noise. We further consider tracking the change of the underlying subspace, which can be applied to applications s…
This work studies two interrelated problems - online robust PCA (RPCA) and online low-rank matrix completion (MC). In recent work by Candès et al., RPCA has been defined as a problem of separating a low-rank matrix (true data), $L:=[\ell_1, \ell_2, \dots \ell_{t}, \dots , \ell_{t_{\max}}]$ and a sparse matrix (outliers), $S:=[x_1, x_2, \dots x_{t}, \dots, x_{t_{\max}}]$ from their sum, $M:=L+S$. Our work uses this definition of RPCA. An important application where both these problems occur is in video analytics in trying to separate sparse foregrounds (e.g., moving objects) and slowly changing…
In this paper, we study the problem of estimating the direction of arrival (DOA) using a sparsely sampled uniform linear array (ULA). Based on an initial incomplete ULA measurement, our strategy is to choose a sparse subset of array elements for measuring the next snapshot. Then, we use a Hankel-structured matrix completion to interpolate for the missing ULA measurements. Finally, the source DOAs are estimated using a subspace method such as Prony on the fully recovered ULA. We theoretically provide a sufficient bound for the number of required samples (array elements) for perfect recovery. Th…
Linear subspace models are pervasive in computational sciences and particularly used for large datasets which are often incomplete due to privacy issues or sampling constraints. Therefore, a critical problem is developing an efficient algorithm for detecting low-dimensional linear structure from incomplete data efficiently, in terms of both computational complexity and storage. In this paper we propose a streaming subspace estimation algorithm called Subspace Navigation via Interpolation from Partial Entries (SNIPE) that efficiently processes blocks of incomplete data to estimate the underlyin…
Principal Components Analysis (PCA) is one of the most widely used dimension reduction techniques. Robust PCA (RPCA) refers to the problem of PCA when the data may be corrupted by outliers. Recent work by Cand{è}s, Wright, Li, and Ma defined RPCA as a problem of decomposing a given data matrix into the sum of a low-rank matrix (true data) and a sparse matrix (outliers). The column space of the low-rank matrix then gives the PCA solution. This simple definition has lead to a large amount of interesting new work on provably correct, fast, and practical solutions to RPCA. More recently, the dynam…
In this work, we present a novel robust distributed beamforming (RDB) approach to mitigate the effects of channel errors on wireless networks equipped with relays based on the exploitation of the cross-correlation between the received data from the relays at the destination and the system output. The proposed RDB method, denoted cross-correlation and subspace projection (CCSP) RDB, considers a total relay transmit power constraint in the system and the objective of maximizing the output signal-to-interference-plus-noise ratio (SINR). The relay nodes are equipped with an amplify-and-forward (AF…
A well-known method for completing low-rank matrices based on convex optimization has been established by Cand{è}s and Recht. Although theoretically complete, the method may not entirely solve the low-rank matrix completion problem. This is because the method captures only the low-rankness property which gives merely a rough constraint that the data points locate on some low-dimensional subspace, but generally ignores the extra structures which specify in more detail how the data points locate on the subspace. Whenever the geometric distribution of the data points is not uniform, the coherence…
Adaptive thresholding methods have proved to yield high SNRs and fast convergence in finding the solution to the Compressed Sensing (CS) problems. Recently, it was observed that the robustness of a class of iterative sparse recovery algorithms such as Iterative Method with Adaptive Thresholding (IMAT) has outperformed the well-known LASSO algorithm in terms of reconstruction quality, convergence speed, and the sensitivity to the noise. In this paper, we introduce a new method towards solving the CS problem. The logic of this method is based on iterative projections of the thresholded signal on…
Online Robust Subspace Tracking from Partial Information
Nanjing University of Information Science and Technology（南京信息工程大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；The Chinese University of Hong Kong（香港中文大学）
This paper examines a general class of noisy matrix completion tasks where the goal is to estimate a matrix from observations obtained at a subset of its entries, each of which is subject to random noise or corruption. Our specific focus is on settings where the matrix to be estimated is well-approximated by a product of two (a priori unknown) matrices, one of which is sparse. Such structural models - referred to here as "sparse factor models" - have been widely used, for example, in subspace clustering applications, as well as in contemporary sparse modeling and dictionary learning tasks. Our…
This paper is concerned with estimating the column space of an unknown low-rank matrix $\boldsymbol{A}^{\star}\in\mathbb{R}^{d_{1}\times d_{2}}$, given noisy and partial observations of its entries. There is no shortage of scenarios where the observations -- while being too noisy to support faithful recovery of the entire matrix -- still convey sufficient information to enable reliable estimation of the column space of interest. This is particularly evident and crucial for the highly unbalanced case where the column dimension $d_{2}$ far exceeds the row dimension $d_{1}$, which is the focal po…
