arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.18893cs.DScs.LGstat.ML
收藏

利用高效得分匹配学习一般高斯混合模型

Learning general Gaussian mixtures with efficient score matching

Harvard SEAS(哈佛大学工程与应用科学学院);UT Austin(德克萨斯大学奥斯汀分校)

Sitan Chen, Vasilis Kontonis, Kulin Shah

AI总结:

本文提出一种基于扩散模型和得分匹配的高效算法,在无分离假设条件下学习一般高斯混合模型,首次以样本多项式时间实现无监督学习的最优理论保证。

摘要:

我们研究在 $d$ 维空间中学习 $k$ 个高斯分布的混合模型的问题。我们对底层混合分量不做任何分离性假设:仅要求协方差矩阵的条件数有界,且均值和协方差位于一个有界半径的球内。我们给出一个算法,该算法从目标混合分布中抽取 $d^{\mathrm{poly}(k/\varepsilon)}$ 个样本,在样本多项式时间内运行,并构造一个采样器,其输出分布在总变差距离上与未知混合分布相差不超过 $\varepsilon$。此前针对该问题的工作要么(i)需要关于维度 $d$ 的指数运行时间,要么(ii)对实例施加了强假设(例如球形协方差或可聚类性),要么(iii)对分量个数 $k$ 具有双重指数依赖。我们的方法偏离了该问题常用的技术,如矩方法。取而代之的是,我们利用最近发展起来的一种基于扩散模型的归约,将分布学习归约为一个称为得分匹配的监督学习任务。我们通过证明一个结构性结果来给出后者的算法,该结果表明高斯混合模型的得分函数可以由一个分段多项式函数逼近,并且存在一个高效算法来找到它。据我们所知,这是扩散模型在无监督学习任务中取得最先进理论保证的第一个例子。

英文摘要:

We study the problem of learning mixtures of $k$ Gaussians in $d$ dimensions. We make no separation assumptions on the underlying mixture components: we only require that the covariance matrices have bounded condition number and that the means and covariances lie in a ball of bounded radius. We give an algorithm that draws $d^{\mathrm{poly}(k/\varepsilon)}$ samples from the target mixture, runs in sample-polynomial time, and constructs a sampler whose output distribution is $\varepsilon$-far from the unknown mixture in total variation. Prior works for this problem either (i) required exponential runtime in the dimension $d$, (ii) placed strong assumptions on the instance (e.g., spherical covariances or clusterability), or (iii) had doubly exponential dependence on the number of components $k$. Our approach departs from commonly used techniques for this problem like the method of moments. Instead, we leverage a recently developed reduction, based on diffusion models, from distribution learning to a supervised learning task called score matching. We give an algorithm for the latter by proving a structural result showing that the score function of a Gaussian mixture can be approximated by a piecewise-polynomial function, and there is an efficient algorithm for finding it. To our knowledge, this is the first example of diffusion models achieving a state-of-the-art theoretical guarantee for an unsupervised learning task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑