用于受限玻尔兹曼机高效学习的非局部转移核

中文摘要 AI 辅助

学习受限玻尔兹曼机（RBM）在计算上具有挑战性，因为它需要计算期望，而这些期望的精确评估通常是难以处理的。这类期望通常采用基于分块吉布斯采样（BGS）的采样近似方法来评估，BGS是一种局部马尔可夫链蒙特卡罗转移核。然而，当RBM存在高能垒时，BGS的局部性会导致采样质量变差，从而降低学习性能。深度回火（DT）会在包括训练RBM在内的一系列可学习RBM上执行并行回火，以此缓解这种局部性问题，但DT算法需要多个步骤在RBM序列中移动以实现非局部转移。本文提出了一种在DT所用RBM序列上定义的转移核，该核在序列上具有往返结构，能够在单次转移中实现非局部移动，同时保持RBM序列不变。数值实验表明，与BGS和DT相比，所提核能更频繁地执行非局部转移，且在更少的转移次数下实现更高的采样质量。我们进一步验证，基于所提核的学习更稳定，能缓解基于BGS和DT的学习中出现的训练失效问题。

英文摘要

Learning restricted Boltzmann machines (RBMs) is computationally challenging because it requires expectations whose exact evaluation is generally intractable. The expectations are typically evaluated using a sampling approximation based on blocked Gibbs sampling (BGS), which is a local Markov chain Monte Carlo transition kernel. However, the locality of BGS can lead to poor sampling quality when the RBM has high energy barriers, thereby degrading learning performance. Deep tempering (DT), which performs parallel tempering over a sequence of learnable RBMs including the training RBM, alleviates this locality issue. However, DT algorithmically requires multiple steps to move through the RBM sequence to achieve a nonlocal transition. In this paper, we propose a transition kernel defined over the RBM sequence used in DT. The proposed kernel has a round-trip structure over the sequence, enabling nonlocal moves within a single transition while leaving the RBM sequence invariant. Numerical experiments show that the proposed kernel performs nonlocal transitions more frequently and achieves higher sampling quality with fewer transitions than BGS and DT. We further verify that learning based on the proposed kernel is more stable and mitigates the training failures observed with BGS- and DT-based learning.

发表机构

• Yamagata University（山形大学）

机构由 AI 辅助整理，请以论文原文为准。