arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.02415cs.LGcs.DSstat.ML
收藏

混合学习的傅立叶方法

A Fourier Approach to Mixture Learning

Stanford University(斯坦福大学);Google Research(谷歌研究院);Google DeepMind(谷歌DeepMind)

Mingda Qiao, Guru Guruganesh, Ankit Singh Rawat, Avinava Dubey, Manzil Zaheer

摘要:

我们重新审视学习混合球高斯的问题。给定来自混合物$\frac{1}{k}\sum_{j=1}^{k}\mathcal{N}(μ_j, I_d)$的样本,目标是估计平均值$μ_1, μ_2, \ldots, μ_k \in \mathbb{R}^d$,误差较小。这个学习问题的难度可以通过定义为所有平均值对之间的最小距离的分离$Δ$来衡量。Regev和Vijayaraghavan(2017)表明,通过$Δ= Ω(\sqrt{\log k})$分离,可以使用$\mathrm{poly}(k, d)$样本学习均值,而如果是$Δ= o(\sqrt{\log k})$和$d = Ω(\log k)$,则需要超多项式多个样本。这就打开了低维区域,其中$d = o(\log k)$。在这项工作中,我们给出了一个算法,有效地学习的手段下分离$d/\sqrt{\log k}$(模双对数因子)在$d = O(\log k/\log\log k)$维度。这种分离是严格小于$\sqrt{\log k}$,也被证明是必要的。与Regev和Vijayaraghavan(2017)的结果一起,我们的工作几乎确定了临界分离阈值,在该阈值下,球形高斯混合物的有效参数学习成为可能。更一般地,我们的算法在时间$\mathrm{poly}(k)\cdot f(d, Δ, ε)$中运行,并且因此在参数$d$、$Δ$和$ε$中是固定参数易处理的。我们的方法是基于估计的傅里叶变换的混合物在精心选择的频率,算法和它的分析是简单和基本的。我们的积极的结果可以很容易地扩展到学习混合的非高斯分布,在一个温和的条件下的傅立叶谱的分布。

英文摘要:

We revisit the problem of learning mixtures of spherical Gaussians. Given samples from mixture $\frac{1}{k}\sum_{j=1}^{k}\mathcal{N}(μ_j, I_d)$, the goal is to estimate the means $μ_1, μ_2, \ldots, μ_k \in \mathbb{R}^d$ up to a small error. The hardness of this learning problem can be measured by the separation $Δ$ defined as the minimum distance between all pairs of means. Regev and Vijayaraghavan (2017) showed that with $Δ= Ω(\sqrt{\log k})$ separation, the means can be learned using $\mathrm{poly}(k, d)$ samples, whereas super-polynomially many samples are required if $Δ= o(\sqrt{\log k})$ and $d = Ω(\log k)$. This leaves open the low-dimensional regime where $d = o(\log k)$. In this work, we give an algorithm that efficiently learns the means in $d = O(\log k/\log\log k)$ dimensions under separation $d/\sqrt{\log k}$ (modulo doubly logarithmic factors). This separation is strictly smaller than $\sqrt{\log k}$, and is also shown to be necessary. Along with the results of Regev and Vijayaraghavan (2017), our work almost pins down the critical separation threshold at which efficient parameter learning becomes possible for spherical Gaussian mixtures. More generally, our algorithm runs in time $\mathrm{poly}(k)\cdot f(d, Δ, ε)$, and is thus fixed-parameter tractable in parameters $d$, $Δ$ and $ε$. Our approach is based on estimating the Fourier transform of the mixture at carefully chosen frequencies, and both the algorithm and its analysis are simple and elementary. Our positive results can be easily extended to learning mixtures of non-Gaussian distributions, under a mild condition on the Fourier spectrum of the distribution.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑