Matrix rank minimization (RM) problems recently gained extensive attention due to numerous applications in machine learning, system identification and graphical models. In RM problem, one aims to find the matrix with the lowest rank that satisfies a set of linear constraints. The existing algorithms include nuclear norm minimization (NNM) and singular value thresholding. Thus far, most of the attention has been on i.i.d. Gaussian measurement operators. In this work, we introduce a new class of measurement operators, and a novel recovery algorithm, which is notably faster than NNM. The proposed operators are based on what we refer to as subspace expanders, which are inspired by the well known expander graphs based measurement matrices in compressed sensing. We show that given an $n\times n$ PSD matrix of rank $r$, it can be uniquely recovered from a minimal sampling of $O(nr)$ measurements using the proposed structures, and the recovery algorithm can be cast as matrix inversion after a few initial processing steps.
Weighted nuclear norm minimization has been recently recognized as a technique for reconstruction of a low-rank matrix from compressively sampled measurements when some prior information about the column and row subspaces of the matrix is available. In this work, we study the recovery conditions and the associated recovery guarantees of weighted nuclear norm minimization when multiple weights are allowed. This setup might be used when one has access to prior subspaces forming multiple angles with the column and row subspaces of the ground-truth matrix. While existing works in this field use a …
Sparse subspace clustering (SSC) relies on sparse regression for accurate neighbor identification. Inspired by recent progress in compressive sensing, this paper proposes a new sparse regression scheme for SSC via two-step reweighted $\ell_1$-minimization, which also generalizes a two-step $\ell_1$-minimization algorithm introduced by E. J. Candès et al in [The Annals of Statistics, vol. 42, no. 2, pp. 669-699, 2014] without incurring extra algorithmic complexity. To fully exploit the prior information offered by the computed sparse representation vector in the first step, our approach places …
An emerging trend of next generation communication systems is to provide network edges with additional capabilities such as additional storage resources in the form of caches to reduce file delivery latency. To investigate this aspect, we study the fundamental limits of a cache-aided wireless network consisting of one central base station, $M$ transceivers and $K$ receivers from a latency-centric perspective. We use the normalized delivery time (NDT) to capture the per-bit latency for the worst-case file request pattern at high signal-to-noise ratios (SNR), normalized with respect to a referen…
In large-scale problems, standard reinforcement learning algorithms suffer from slow learning speed. In this paper, we follow the framework of using subspaces to tackle this problem. We propose a free-energy minimization framework for selecting the subspaces and integrate the policy of the state-space into the subspaces. Our proposed free-energy minimization framework rests upon Thompson sampling policy and behavioral policy of subspaces and the state-space. It is therefore applicable to a variety of tasks, discrete or continuous state space, model-free and model-based tasks. Through a set of …
We propose a new iterative greedy algorithm for reconstructions of sparse signals with or without noisy perturbations in compressed sensing. The proposed algorithm, called \emph{subspace thresholding pursuit} (STP) in this paper, is a simple combination of subspace pursuit and iterative hard thresholding. Firstly, STP has the theoretical guarantee comparable to that of $\ell_1$ minimization in terms of restricted isometry property. Secondly, with a tuned parameter, on the one hand, when reconstructing Gaussian signals, it can outperform other state-of-the-art reconstruction algorithms greatly;…
In this paper, we formulate the interference alignment (IA) problem for a multiuser multiple-input multiple-output (MIMO) system in the presence of an eavesdropper as a rank constrained rank minimization (RCRM) problem. The aim of the proposed rank minimization IA schemes is to find the precoding and receiver subspace matrices to align interference and wiretapped signals into the lowest dimension subspaces while keeping the desired signal subspace spanning full available spatial dimensions. To deal with the nonconvexity of the rank function, we present two convex relaxations of the RCRM proble…
The multiple measurement vector problem (MMV) is a generalization of the compressed sensing problem that addresses the recovery of a set of jointly sparse signal vectors. One of the important contributions of this paper is to reveal that the seemingly least related state-of-art MMV joint sparse recovery algorithms - M-SBL (multiple sparse Bayesian learning) and subspace-based hybrid greedy algorithms - have a very important link. More specifically, we show that replacing the $\log\det(\cdot)$ term in M-SBL by a rank proxy that exploits the spark reduction property discovered in subspace-based …
In this paper, we generalize the well-known index coding problem to exploit the structure in the source-data to improve system throughput. In many applications, the data to be transmitted may lie (or can be well approximated) in a low-dimensional subspace. We exploit this low-dimensional structure of the data using an algebraic framework to solve the index coding problem (referred to as subspace-aware index coding) as opposed to the traditional index coding problem which is subspace-unaware. Also, we propose an efficient algorithm based on the alternating minimization approach to obtain near o…
In this paper, interference alignment (IA) is designed for secure multiuser multiple-input multiple-output systems in the presence of an eavesdropper. The proposed IA technique designs the transmit precoding and receiving subspace matrices to minimize both the total inter-main-link interference and the wiretapped signals. With perfect channel state information is known at the transmitters (Txs), the cost function of the optimization problem is alternatively minimized over the precoding matrices at the Txs and the receiving subspace matrices at the receivers (Rxs) and the eavesdropper. The feas…
We introduce the notion of relevant information loss for the purpose of casting the signal enhancement problem in information-theoretic terms. We show that many algorithms from machine learning can be reformulated using relevant information loss, which allows their application to the aforementioned problem. As a particular example we analyze principle component analysis for dimensionality reduction, discuss its optimality, and show that the relevant information loss can indeed vanish if the relevant information is concentrated on a lower-dimensional subspace of the input space.
Matrix sensing is the problem of reconstructing a low-rank matrix from a few linear measurements. In many applications such as collaborative filtering, the famous Netflix prize problem, and seismic data interpolation, there exists some prior information about the column and row spaces of the ground-truth low-rank matrix. In this paper, we exploit this prior information by proposing a weighted optimization problem where its objective function promotes both rank and prior subspace information. Using the recent results in conic integral geometry, we obtain the unique optimal weights that minimize…
Matrix recovery is the problem of recovering a low-rank matrix from a few linear measurements. Recently, this problem has gained a lot of attention as it is employed in many applications such as Netflix prize problem, seismic data interpolation and collaborative filtering. In these applications, one might access to additional prior information about the column and row spaces of the matrix. These extra information can potentially enhance the matrix recovery performance. In this paper, we propose an efficient greedy algorithm that exploits prior information in the recovery procedure. The perform…
Matrix completion refers to completing a low-rank matrix from a few observed elements of its entries and has been known as one of the significant and widely-used problems in recent years. The required number of observations for exact completion is directly proportional to rank and the coherency parameter of the matrix. In many applications, there might exist additional information about the low-rank matrix of interest. For example, in collaborative filtering, Netflix and dynamic channel estimation in communications, extra subspace information is available. More precisely in these applications,…
We consider the problem of recovering two unknown vectors, $\boldsymbol{w}$ and $\boldsymbol{x}$, of length $L$ from their circular convolution. We make the structural assumption that the two vectors are members of known subspaces, one with dimension $N$ and the other with dimension $K$. Although the observed convolution is nonlinear in both $\boldsymbol{w}$ and $\boldsymbol{x}$, it is linear in the rank-1 matrix formed by their outer product $\boldsymbol{w}\boldsymbol{x}^*$. This observation allows us to recast the deconvolution problem as low-rank matrix recovery problem from linear measurem…
In this paper we present a new algorithm for compressive sensing that makes use of binary measurement matrices and achieves exact recovery of ultra sparse vectors, in a single pass and without any iterations. Due to its noniterative nature, our algorithm is hundreds of times faster than $\ell_1$-norm minimization, and methods based on expander graphs, both of which require multiple iterations. Our algorithm can accommodate nearly sparse vectors, in which case it recovers index set of the largest components, and can also accommodate burst noise measurements. Compared to compressive sensing meth…
Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative $α$-entropies (denoted $\mathscr{I}_α$), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative $α$-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimiz…
We show that the maximization of the sum degrees-of-freedom for the static flat-fading multiple-input multiple-output (MIMO) interference channel is equivalent to a rank constrained rank minimization problem (RCRM), when the signal spaces span all available dimensions. The rank minimization corresponds to maximizing interference alignment (IA) so that interference spans the lowest dimensional subspace possible. The rank constraints account for the useful signal spaces spanning all available spatial dimensions. That way, we reformulate all IA requirements to requirements involving ranks. Then, …
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；KU Eichstätt-Ingolstadt（艾希施泰特-英戈尔施塔特天主教大学）
In this paper, we study the problem of recovering two unknown signals from their convolution, which is commonly referred to as blind deconvolution. Reformulation of blind deconvolution as a low-rank recovery problem has led to multiple theoretical recovery guarantees in the past decade due to the success of the nuclear norm minimization heuristic. In particular, in the absence of noise, exact recovery has been established for sufficiently incoherent signals contained in lower-dimensional subspaces. However, if the convolution is corrupted by additive bounded noise, the stability of the recover…
Extracting latent low-dimensional structure from high-dimensional data is of paramount importance in timely inference tasks encountered with `Big Data' analytics. However, increasingly noisy, heterogeneous, and incomplete datasets as well as the need for {\em real-time} processing of streaming data pose major challenges to this end. In this context, the present paper permeates benefits from rank minimization to scalable imputation of missing data, via tracking low-dimensional subspaces and unraveling latent (possibly multi-way) structure from \emph{incomplete streaming} data. For low-rank matr…
University of California, Berkeley（加州大学伯克利分校）；The Johns Hopkins University（约翰斯·霍普金斯大学）
In many real-world problems, we are dealing with collections of high-dimensional data, such as images, videos, text and web documents, DNA microarray data, and more. Often, high-dimensional data lie close to low-dimensional structures corresponding to several classes or categories the data belongs to. In this paper, we propose and study an algorithm, called Sparse Subspace Clustering (SSC), to cluster data points that lie in a union of low-dimensional subspaces. The key idea is that, among infinitely many possible representations of a data point in terms of other points, a sparse representatio…
