arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.07707cs.ITcs.LGmath.ITmath.STstat.MLstat.TH
收藏

信道容量在高斯混合模型学习中的作用

On the Role of Channel Capacity in Learning Gaussian Mixture Models

The Hebrew University of Jerusalem(耶路撒冷希伯来大学);Tel Aviv University(特拉维夫大学)

Elad Romanov, Tamir Bendory, Or Ordentlich

摘要:

研究了在具有球形协方差矩阵$σ^2\mathbf{I}$的$\mathbb{R}^d$中学习平衡高斯混合模型(GMM)的$k$未知中心的样本复杂度.特别是,我们感兴趣的是以下问题:什么是最大的噪声水平$σ^2$,其中的样本复杂性基本上是相同的,当估计的中心标记的测量?为此,我们将注意力限制在贝叶斯公式的问题,其中的中心是均匀分布的领域$\sqrt{d}\mathcal{S}^{d-1}$。我们的主要研究结果的特点是精确的噪声阈值$σ^2$低于GMM学习问题,在大系统的限制$d,k\to\infty$,是很容易从标记的观察学习,并在此之上,它是实质上更难。阈值出现在$\frac{\log k}{d} = \frac12\log\left( 1+\frac{1}{σ^2} \right)$处,其是加性高斯白噪声(AWGN)信道的容量。将$k$中心的集合视为码,该噪声阈值可以被解释为最大噪声电平,对于该最大噪声电平,码在AWGN信道上的错误概率是小的。GMM学习问题上的先前工作已经确定了中心之间的最小距离作为确定学习相应GMM的统计难度的关键参数。虽然我们的结果只证明GMM的中心是均匀分布在球体上,他们暗示,也许这是解码错误概率与中心星座作为一个信道码,确定学习相应的GMM的统计难度,而不仅仅是最小距离。

英文摘要:

This paper studies the sample complexity of learning the $k$ unknown centers of a balanced Gaussian mixture model (GMM) in $\mathbb{R}^d$ with spherical covariance matrix $σ^2\mathbf{I}$. In particular, we are interested in the following question: what is the maximal noise level $σ^2$, for which the sample complexity is essentially the same as when estimating the centers from labeled measurements? To that end, we restrict attention to a Bayesian formulation of the problem, where the centers are uniformly distributed on the sphere $\sqrt{d}\mathcal{S}^{d-1}$. Our main results characterize the exact noise threshold $σ^2$ below which the GMM learning problem, in the large system limit $d,k\to\infty$, is as easy as learning from labeled observations, and above which it is substantially harder. The threshold occurs at $\frac{\log k}{d} = \frac12\log\left( 1+\frac{1}{σ^2} \right)$, which is the capacity of the additive white Gaussian noise (AWGN) channel. Thinking of the set of $k$ centers as a code, this noise threshold can be interpreted as the largest noise level for which the error probability of the code over the AWGN channel is small. Previous works on the GMM learning problem have identified the minimum distance between the centers as a key parameter in determining the statistical difficulty of learning the corresponding GMM. While our results are only proved for GMMs whose centers are uniformly distributed over the sphere, they hint that perhaps it is the decoding error probability associated with the center constellation as a channel code that determines the statistical difficulty of learning the corresponding GMM, rather than just the minimum distance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑