arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.11229physics.comp-phcs.NE
收藏

用于高效评估受限玻尔兹曼机配分函数的退火重要性采样算法的平均场初始化

Mean field initialization of the Annealed Importance Sampling algorithm for an efficient evaluation of the Partition Function of Restricted Boltzmann Machines

A. Prat Pou, E. Romero, J. Martí, F. Mazzanti

AI总结:

针对受限玻尔兹曼机配分函数计算成本高的问题,提出用平均场分布作为退火重要性采样的起始分布,经大小规模问题验证可显著提升估计质量、降低计算成本,并给出两种有效策略。

摘要:

物理学中的概率模型通常需要计算归一化玻尔兹曼因子,而这又意味着需要计算配分函数Z。然而,随着系统规模增大,获取Z的精确值会变成一项计算成本高得难以承受的任务。这一问题也存在于受限玻尔兹曼机(Restricted Boltzmann Machine,RBM)这类概率学习模型中,且情况更糟,因为精确学习规则要求在每次迭代时都计算Z。解决该问题的一种可行方法是使用退火重要性采样(Annealed Importance Sampling,AIS)算法,它提供了一种随机估计系统配分函数的工具。截至目前,AIS算法的标准应用从均匀概率分布出发,需要经过大量蒙特卡洛步,在退火过程后得到可靠的Z估计值。在本研究中我们表明,通过使用恰当选择的平均场起始概率分布,估计的质量和计算成本都能得到显著提升。我们在小规模和大规模问题中都对AIS开展了系统分析,并在已知精确值的问题中将结果与精确值进行了对比。基于系统分析的结果,我们提出了两种有效的策略,它们在所有分析的问题中都表现良好。我们得出结论,这些策略是使用AIS估计配分函数的良好起点,且计算成本相对较低。

英文摘要:

Probabilistic models in physics often require from the evaluation of normalized Boltzmann factors, which in turn implies the computation of the partition function Z. Getting the exact value of Z, though, becomes a forbiddingly expensive task as the system size increases. This problem is also present in probabilistic learning models such as the Restricted Boltzmann Machine (RBM), where the situation is even worse as the exact learning rules implies the computation of Z at each iteration. A possible way to tackle this problem is to use the Annealed Importance Sampling (AIS) algorithm, which provides a tool to stochastically estimate the partition function of the system. So far, the standard application of the AIS algorithm starts from the uniform probability distribution and uses a large number of Monte Carlo steps to obtain reliable estimations of Z following an annealing process. In this work we show that both the quality of the estimation and the cost of the computation can be significantly improved by using a properly selected mean-field starting probability distribution. We perform a systematic analysis of AIS in both small- and large-sized problems, and compare the results to exact values in problems where these are known. As a result of our systematic analysis, we propose two successful strategies that work well in all the problems analyzed. We conclude that these are good starting points to estimate the partition function with AIS with a relatively low computational cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑