arXiv 2105.080052021-05-17cs.LGcs.DSstat.ML

在输入稀疏时间内学习隐单纯形

Learning a Latent Simplex in Input-Sparsity Time

Carnegie Mellon University（卡内基梅隆大学）；Indian Institute of Science（印度科学研究所）；Microsoft Research India（微软印度研究院）

Ainesh Bakshi, Chiranjib Bhattacharyya, Ravi Kannan, David P. Woodruff, Samson Zhou

备注：: ICLR 2021

摘要：

我们考虑的问题，学习潜在的$k$ -顶点单纯形$K\subset\mathbb{R}^d$，访问$A\in\mathbb{R}^{d\times n}$，这可以被看作是一个数据矩阵与$n$点，通过随机扰动单$K$（潜在的超越$K$）中的潜在点。一大类潜在变量模型，如对抗聚类，混合隶属度随机块模型和主题模型，可以转换为学习一个潜在的单纯形。Bhattacharyya和Kannan（SODA，2020）给出了一种学习这种潜在单纯形的算法，大致为$O(k\cdot\textrm{nnz}(A))$，其中$\textrm{nnz}(A)$是$A$中非零的数量。我们表明，依赖于$k$在运行时间是不必要的给定一个自然的假设的质量的顶部$k$奇异值的$A$，这在许多这些应用程序。此外，我们证明了这个假设是必要的，否则学习潜在单纯形的算法将意味着谱低秩近似的算法突破。Bhattacharyya和Kannan提出了一种自适应算法，它使$k$矩阵-向量积查询成为$A$，每个查询都是它之前所有查询的函数，由于每个矩阵-向量积都需要$\textrm{nnz}(A)$时间，因此它们的总运行时间是不可避免的。相反，我们得到一个低秩近似$A$在输入稀疏时间，并表明，这样得到的列空间有小的$\sinΘ$（角）距离右顶部- $k$奇异空间的$A$。我们的算法，然后选择$k$点的低秩子空间与$k$精心选择的随机向量的最大内积。通过在低秩子空间中工作，我们避免了在每次迭代中读取整个矩阵，从而避免了$Θ(k\cdot\textrm{nnz}(A))$的运行时间。

英文摘要：

We consider the problem of learning a latent $k$-vertex simplex $K\subset\mathbb{R}^d$, given access to $A\in\mathbb{R}^{d\times n}$, which can be viewed as a data matrix with $n$ points that are obtained by randomly perturbing latent points in the simplex $K$ (potentially beyond $K$). A large class of latent variable models, such as adversarial clustering, mixed membership stochastic block models, and topic models can be cast as learning a latent simplex. Bhattacharyya and Kannan (SODA, 2020) give an algorithm for learning such a latent simplex in time roughly $O(k\cdot\textrm{nnz}(A))$, where $\textrm{nnz}(A)$ is the number of non-zeros in $A$. We show that the dependence on $k$ in the running time is unnecessary given a natural assumption about the mass of the top $k$ singular values of $A$, which holds in many of these applications. Further, we show this assumption is necessary, as otherwise an algorithm for learning a latent simplex would imply an algorithmic breakthrough for spectral low rank approximation. At a high level, Bhattacharyya and Kannan provide an adaptive algorithm that makes $k$ matrix-vector product queries to $A$ and each query is a function of all queries preceding it. Since each matrix-vector product requires $\textrm{nnz}(A)$ time, their overall running time appears unavoidable. Instead, we obtain a low-rank approximation to $A$ in input-sparsity time and show that the column space thus obtained has small $\sinΘ$ (angular) distance to the right top-$k$ singular space of $A$. Our algorithm then selects $k$ points in the low-rank subspace with the largest inner product with $k$ carefully chosen random vectors. By working in the low-rank subspace, we avoid reading the entire matrix in each iteration and thus circumvent the $Θ(k\cdot\textrm{nnz}(A))$ running time.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文