压缩谱嵌入：避开SVD
Compressive spectral embedding: sidestepping the SVD
摘要：
基于奇异值分解（SVD）的谱嵌入是许多学习任务中广泛使用的“预处理”步骤，通常通过投影到多个主要奇异向量上并重新缩放坐标轴（通过奇异值的预定义函数）来实现降维。然而，捕获问题结构所需的此类向量的数量随着问题的大小而增长，甚至部分SVD计算也成为瓶颈。在本文中，我们提出了一个低复杂度的压缩频谱嵌入算法，它采用随机投影和有限阶多项式展开计算近似SVD为基础的嵌入。对于一个m × n矩阵，T个非零，其时间复杂度为O（（T+m+n）log（m+n）），嵌入维数为O（log（m+n）），这两个参数都与我们要捕获的奇异向量的个数无关。据我们所知，这是第一个工作，以规避这种依赖于奇异向量的数量一般SVD为基础的嵌入。避开SVD的关键是观察到，对于下游推理任务，如聚类和分类，我们只对使用结果嵌入来评估从欧几里德范数导出的成对相似性度量感兴趣，而不是像部分SVD试图做的那样捕捉基础矩阵对任意向量的影响。我们在网络数据集上的数值结果证明了所提出的方法的有效性，并激励进一步探索其应用于大规模推理任务。
英文摘要：
Spectral embedding based on the Singular Value Decomposition (SVD) is a widely used "preprocessing" step in many learning tasks, typically leading to dimensionality reduction by projecting onto a number of dominant singular vectors and rescaling the coordinate axes (by a predefined function of the singular value). However, the number of such vectors required to capture problem structure grows with problem size, and even partial SVD computation becomes a bottleneck. In this paper, we propose a low-complexity it compressive spectral embedding algorithm, which employs random projections and finite order polynomial expansions to compute approximations to SVD-based embedding. For an m times n matrix with T non-zeros, its time complexity is O((T+m+n)log(m+n)), and the embedding dimension is O(log(m+n)), both of which are independent of the number of singular vectors whose effect we wish to capture. To the best of our knowledge, this is the first work to circumvent this dependence on the number of singular vectors for general SVD-based embeddings. The key to sidestepping the SVD is the observation that, for downstream inference tasks such as clustering and classification, we are only interested in using the resulting embedding to evaluate pairwise similarity metrics derived from the euclidean norm, rather than capturing the effect of the underlying matrix on arbitrary vectors as a partial SVD tries to do. Our numerical results on network datasets demonstrate the efficacy of the proposed method, and motivate further exploration of its application to large-scale inference tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
