矩阵和张量的谱学习
Spectral Learning on Matrices and Tensors
摘要：
谱方法已经成为机器学习和科学计算等领域的主流。它们涉及找到某种谱分解，以获得可以捕获手头问题的重要结构的基函数。最常见的光谱方法是主成分分析（PCA）。它利用数据协方差矩阵的顶部特征向量，例如进行降维。该数据预处理步骤在将信号与噪声分离方面通常是有效的。应用于矩阵的PCA和其他谱技术具有若干限制。通过仅限制成对矩，它们有效地对底层数据进行高斯近似，并在具有导致非高斯性的隐变量的数据上失败。然而，在大多数数据集中，存在无法直接观察到的潜在影响，例如，文档语料库中的主题或疾病的根本原因。通过将谱分解方法扩展到高阶矩，我们证明了有效学习各种潜变量模型的能力。高阶矩可以用张量表示，直观地说，它们可以编码比成对矩矩阵更多的信息。更重要的是，张量分解可以提取矩阵方法遗漏的潜在效应，例如唯一识别非正交分量。利用这些方面是卓有成效的证明无监督学习的范围广泛的潜变量模型。我们还概述了设计高效的张量分解方法的计算技术。我们介绍Tensorly，它有一个简单的Python接口来表达张量操作。它具有灵活的后端系统，支持NumPy，PyTorch，TensorFlow和MXNet等，允许多GPU和CPU操作，并与深度学习功能进行无缝集成。
英文摘要：
Spectral methods have been the mainstay in several domains such as machine learning and scientific computing. They involve finding a certain kind of spectral decomposition to obtain basis functions that can capture important structures for the problem at hand. The most common spectral method is the principal component analysis (PCA). It utilizes the top eigenvectors of the data covariance matrix, e.g. to carry out dimensionality reduction. This data pre-processing step is often effective in separating signal from noise. PCA and other spectral techniques applied to matrices have several limitations. By limiting to only pairwise moments, they are effectively making a Gaussian approximation on the underlying data and fail on data with hidden variables which lead to non-Gaussianity. However, in most data sets, there are latent effects that cannot be directly observed, e.g., topics in a document corpus, or underlying causes of a disease. By extending the spectral decomposition methods to higher order moments, we demonstrate the ability to learn a wide range of latent variable models efficiently. Higher-order moments can be represented by tensors, and intuitively, they can encode more information than just pairwise moment matrices. More crucially, tensor decomposition can pick up latent effects that are missed by matrix methods, e.g. uniquely identify non-orthogonal components. Exploiting these aspects turns out to be fruitful for provable unsupervised learning of a wide range of latent variable models. We also outline the computational techniques to design efficient tensor decomposition methods. We introduce Tensorly, which has a simple python interface for expressing tensor operations. It has a flexible back-end system supporting NumPy, PyTorch, TensorFlow and MXNet amongst others, allowing multi-GPU and CPU operations and seamless integration with deep-learning functionalities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
