从局部最优构造:通过高阶PCA学习子空间Juntas
Structure from Local Optima: Learning Subspace Juntas via Higher Order PCA
摘要:
我们提出了一个众所周知的问题,学习k-juntas在R^n的推广,和一个新的张量算法解开高维分布的结构。我们的算法可以被看作是主成分分析(PCA)的高阶扩展。我们的激励问题是学习R^n中的标记函数,它由未知的k维子空间确定。这个学习k-子空间junta的问题是学习k-junta(R^n中k个坐标的函数)和学习k个半空间的交集的常见推广。在此背景下,我们引入了一个不相关的噪声属性模型,其中的分布在“相关”的k维子空间是独立的(n-k)维的“不相关”的子空间的分布正交于它。我们给出了一个谱张量算法,识别相关的子空间,从而学习k-子空间juntas在一些额外的假设。我们通过利用单位球上高阶矩张量的局部最优值的结构来做到这一点; PCA找到二阶矩张量(协方差矩阵)的全局最优值。我们的主要结果是,当不相关的(n-k)维子空间中的分布为任意高斯分布时,我们的算法的复杂度为T(k,ε)+ \poly(n),其中T是k维概念学习的复杂度,多项式是所学习的k维概念类的函数.这基本上概括了现有的低维概念学习的结果。
英文摘要:
We present a generalization of the well-known problem of learning k-juntas in R^n, and a novel tensor algorithm for unraveling the structure of high-dimensional distributions. Our algorithm can be viewed as a higher-order extension of Principal Component Analysis (PCA). Our motivating problem is learning a labeling function in R^n, which is determined by an unknown k-dimensional subspace. This problem of learning a k-subspace junta is a common generalization of learning a k-junta (a function of k coordinates in R^n) and learning intersections of k halfspaces. In this context, we introduce an irrelevant noisy attributes model where the distribution over the "relevant" k-dimensional subspace is independent of the distribution over the (n-k)-dimensional "irrelevant" subspace orthogonal to it. We give a spectral tensor algorithm which identifies the relevant subspace, and thereby learns k-subspace juntas under some additional assumptions. We do this by exploiting the structure of local optima of higher moment tensors over the unit sphere; PCA finds the global optima of the second moment tensor (covariance matrix). Our main result is that when the distribution in the irrelevant (n-k)-dimensional subspace is any Gaussian, the complexity of our algorithm is T(k,ε) + \poly(n), where T is the complexity of learning the concept in k dimensions, and the polynomial is a function of the k-dimensional concept class being learned. This substantially generalizes existing results on learning low-dimensional concepts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。