一种用于高斯混合学习的傅里叶分析方法
A Fourier analytique approach to Gaussian mixture learning
AI总结:
本文提出了一种基于傅里叶分析的高斯混合学习算法,能够在任意维度和组件数量下,以多项式复杂度学习高斯分布的中心和权重。
摘要:
假设我们有来自最多\(k\)个\(d\)维球形高斯分布\(\mu_1,\cdots,\mu_{k_0}\)的独立同分布随机样本\(x_1,\cdots,x_n\),各坐标方差\(\sigma^2\)相同且已知,两个不同中心\(y_l\)和\(y_j\)之间的最小\(\ell^2\)距离大于\(2\Delta\sigma \min\{\sqrt{d},\sqrt k\}\),其中\(\Delta>C_0\),\(C_0\)为足够大的通用常数。我们开发了一种随机算法,在权重已知为均匀时,能在任意数量的组件和任意维度下,将高斯组件的中心\(y_l\)学习到\(\ell^2\)距离为\(k^{-\tilde C_0}\)。若组件数量\(k = \Omega(2^d)\),对于任意通用常数\(c>0\),即使权重未知,算法也能将中心学习到\(\ell^2\)距离为\(d^{-\tilde C_0}\),权重精度达到\(cw_{min}\),概率大于\(1 - \exp(-k/c)\),前提是权重在\([c/k,1/ck]\),最小间隔为\(2c\sqrt d\)。两种情况下样本数量和计算时间都以\(\mathrm{poly}(k, d)\)为界。这种样本和计算复杂度的界在非恒定维度情况下此前未知,特别是当\(d\)不是\(O(1)\)时。当\(d = O(1)\)时,此复杂度界可由Regev和Vijayaraghavan的工作得出,他们还表明当\(d\)为\(\Theta( \log k)\)时,在\(d\)维中半径为\(o(\sqrt{d})\)的球内学习高斯随机混合的样本复杂度至少在\(k, d\)上是超多项式的,表明我们的结果在这种情况下是紧密的。
英文摘要:
Suppose that we are given independent, identically distributed random samples $x_1,\cdots,x_n$ from a mixture at most $k$ many $d$-dimensional spherical Gaussian distributions $μ_1,\cdots,μ_{k_0}$ of identical and known variance $σ^2$ in each coordinate, such that the minimum $\ell^2$ distance between two distinct centers $y_l$ and $y_j$ is greater than $2Δσ\min\{\sqrt{d},\sqrt k\}$, where $Δ>C_0$, and $C_0$ is a sufficiently large universal constant. We develop a randomized algorithm that learns the centers $y_l$'s of the Gaussian components to within an $\ell^2$ distance of $k^{-\tilde C_0}$ -- in presence of arbitrarily large number of components and in arbitrary dimension, when the weights are known to be uniform. Furthermore, if the number of components is $k= Ω(2^d)$, then for arbitrary universal constant $c>0$, even for unknown weights, the algorithm learns the centers to within an $\ell^2$ distance of $d^{-\tilde C_0}$ and the weights up to an accuracy of $cw_{min}$, with probability greater than $1 - \exp(-k/c)$, provided that the weights lie in $[c/k,1/ck]$, and the minimum separation is just $2c\sqrt d$. The number of samples and the computational time is bounded above by $\mathrm{poly}(k, d)$ in either case. Such a bound on the sample and computational complexity was previously unknown in the regime of non-constant dimension, and in particular, when $d$ is not $O(1)$. When $d = O(1)$, this complexity bound follows from work of Regev and Vijayaraghavan, where it has also been shown that the sample complexity of learning a random mixture of Gaussians in a ball of radius $o(\sqrt{d})$ in $d$ dimensions, when $d$ is $Θ( \log k)$, is at least super-polynomial in $k, d$, showing that our result is tight in this case.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。