arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.11926cs.LGphysics.comp-phstat.ML
收藏

基于退火重要性采样的RBM配分函数的有效估计

Efficient Evaluation of the Partition Function of RBMs with Annealed Importance Sampling

Universitat Politècnica de Catalunya - BarcelonaTech(加泰罗尼亚理工大学)

Ferran Mazzanti, Enrique Romero

摘要:

基于受限玻尔兹曼机(RBM)的概率模型意味着归一化玻尔兹曼因子的评估,这反过来又需要从配分函数Z的评估。然而,随着系统规模的增加,Z的精确评估成为一项昂贵的任务。当考虑RBM的最常见的学习算法时,这甚至是错误的,其中对数据的经验分布的对数似然的梯度的精确评估包括在每次迭代时对Z的计算。退火重要性采样(AIS)方法提供了一种随机估计系统配分函数的工具。到目前为止,AIS算法在机器学习上下文中的标准使用已经使用大量的Monte Carlo步骤完成。在这项工作中,我们表明,这可能是不需要的,如果一个适当的起始概率分布作为AIS算法的初始化。我们分析了AIS的性能,在小型和大型的问题,并表明,在这两种情况下,一个很好的估计Z可以得到很少的计算成本。

英文摘要:

Probabilistic models based on Restricted Boltzmann Machines (RBMs) imply the evaluation of normalized Boltzmann factors, which in turn require from the evaluation of the partition function Z. The exact evaluation of Z, though, becomes a forbiddingly expensive task as the system size increases. This even worsens when one considers most usual learning algorithms for RBMs, where the exact evaluation of the gradient of the log-likelihood of the empirical distribution of the data includes the computation of Z at each iteration. The Annealed Importance Sampling (AIS) method provides a tool to stochastically estimate the partition function of the system. So far, the standard use of the AIS algorithm in the Machine Learning context has been done using a large number of Monte Carlo steps. In this work we show that this may not be required if a proper starting probability distribution is employed as the initialization of the AIS algorithm. We analyze the performance of AIS in both small- and large-sized problems, and show that in both cases a good estimation of Z can be obtained with little computational cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑