一元高斯混合模型Jeffreys散度的指数多项式密度快速逼近
Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities
摘要：
Jeffreys发散是对信息科学中广泛应用的定向Kullback-Leibler发散的一种对称化。由于高斯混合模型之间的Jeffreys发散没有封闭形式，文献中提出了各种有利和不利的技术来估计，近似或下限和上限这种发散。在本文中，我们提出了一个简单而快速的启发式近似的Jeffreys散度之间的两个单变量高斯混合任意数量的组件。我们的启发式依赖于转换成双参数化的概率密度属于一个指数家庭的混合物。特别是，我们考虑了通用的多项式指数族密度，并设计了一个发散度来衡量在封闭形式的高斯混合和多项式指数密度近似之间的拟合优度。这种拟合优度发散是Hyvärinen发散的推广，用于估计具有计算上难以处理的归一化器的模型。它允许我们通过选择用于近似混合物的多项式指数密度的阶数来执行模型选择。我们的实验表明，我们的启发式近似杰弗里斯发散提高了几个数量级的随机蒙特卡罗估计的计算时间，同时近似合理以及杰弗里斯发散，特别是当混合物有一个非常小的模式。此外，我们的混合到指数族转换技术可能在其他环境中很有用。
英文摘要：
The Jeffreys divergence is a renown symmetrization of the oriented Kullback-Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with pros and cons have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential family. In particular, we consider the versatile polynomial exponential family densities, and design a divergence to measure in closed-form the goodness of fit between a Gaussian mixture and its polynomial exponential density approximation. This goodness-of-fit divergence is a generalization of the Hyvärinen divergence used to estimate models with computationally intractable normalizers. It allows us to perform model selection by choosing the orders of the polynomial exponential densities used to approximate the mixtures. We demonstrate experimentally that our heuristic to approximate the Jeffreys divergence improves by several orders of magnitude the computational time of stochastic Monte Carlo estimations while approximating reasonably well the Jeffreys divergence, specially when the mixtures have a very small number of modes. Besides, our mixture-to-exponential family conversion techniques may prove useful in other settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
