arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1210.7559cs.LGcs.NAmath.NAstat.ML
收藏

用于学习潜变量模型的张量分解

Tensor decompositions for learning latent variable models

University of California, Irvine(加州大学欧文分校);Microsoft Research(微软研究院);Columbia University(哥伦比亚大学);Rutgers University(罗格斯大学)

Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, Matus Telgarsky

AI总结:

本文利用低阶可观测矩的张量结构,通过对称张量分解(类似矩阵SVD的推广)实现高斯混合模型、隐马尔可夫模型等潜变量模型的参数估计,并提供了鲁棒张量幂法的详细分析。

摘要:

本文研究了一类广泛的潜变量模型(包括高斯混合模型、隐马尔可夫模型和潜在狄利克雷分配)的计算和统计高效参数估计方法,该方法利用了其低阶可观测矩(通常是二阶和三阶)中的特定张量结构。具体地,参数估计被简化为从矩导出的对称张量中提取某种(正交)分解的问题;这种分解可以看作是矩阵奇异值分解的自然推广。尽管张量分解通常难以计算,但这些特殊结构张量的分解可以通过多种方法高效获得,包括幂迭代和最大化方法(类似于矩阵的情况)。本文提供了鲁棒张量幂方法的详细分析,建立了类似于矩阵奇异向量的Wedin扰动定理的类比。这意味着对于几种流行的潜变量模型,存在一种鲁棒且计算可行的估计方法。

英文摘要:

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin's perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑