arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.04478stat.MLcs.LGstat.COstat.ME
收藏

贝叶斯分配模型:非负张量分解和主题模型的序贯蒙特卡罗推断

Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns

Boğaziçi University(博阿齐奇大学);Sabancı University(萨班哲大学);Télécom ParisTech(巴黎高等电信学院)

Ali Taylan Cemgil, Mehmet Burak Kurutmaz, Sinan Yildirim, Melih Barsbey, Umut Simsekli

摘要:

我们介绍了一个动态生成模型,贝叶斯分配模型(BAM),建立显式连接之间的非负张量因子分解(NTF),离散概率分布的图形模型和贝叶斯扩展,和主题模型,如潜在的狄利克雷分配。BAM是基于泊松过程,其事件是通过使用贝叶斯网络标记,其中该网络的条件概率表,然后集成了分析。我们表明,由此产生的边际过程原来是一个波利亚瓮,一个整数值的自我增强过程。这个瓮过程,我们命名为Polya-Bayes过程,服从一定的条件独立性,提供了进一步了解NTF的性质。这些见解也让我们开发空间有效的模拟算法,尊重潜在的稀疏数据:我们提出了一类连续的重要性抽样算法计算NTF和近似其边际似然,这将是有用的模型选择。所得到的方法也可以被看作是一个模型评分方法的主题模型和离散贝叶斯网络的隐变量。新算法具有良好的性能在稀疏的数据制度相比,变分算法,变得更准确时,所观察到的张量的元素的总和趋于无穷大。我们说明了几个例子的性能和数值研究的各种数据制度的算法的行为。

英文摘要:

We introduce a dynamic generative model, Bayesian allocation model (BAM), which establishes explicit connections between nonnegative tensor factorization (NTF), graphical models of discrete probability distributions and their Bayesian extensions, and the topic models such as the latent Dirichlet allocation. BAM is based on a Poisson process, whose events are marked by using a Bayesian network, where the conditional probability tables of this network are then integrated out analytically. We show that the resulting marginal process turns out to be a Polya urn, an integer valued self-reinforcing process. This urn processes, which we name a Polya-Bayes process, obey certain conditional independence properties that provide further insight about the nature of NTF. These insights also let us develop space efficient simulation algorithms that respect the potential sparsity of data: we propose a class of sequential importance sampling algorithms for computing NTF and approximating their marginal likelihood, which would be useful for model selection. The resulting methods can also be viewed as a model scoring method for topic models and discrete Bayesian networks with hidden variables. The new algorithms have favourable properties in the sparse data regime when contrasted with variational algorithms that become more accurate when the total sum of the elements of the observed tensor goes to infinity. We illustrate the performance on several examples and numerically study the behaviour of the algorithms for various data regimes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑