限制Boltzmann机交替Gibbs抽样中的障碍和动力学路径
Barriers and Dynamical Paths in Alternating Gibbs Sampling of Restricted Boltzmann Machines
摘要：
受限玻尔兹曼机（RBM）是一种双层神经网络，用于从数据中无监督地学习模型分布。RBM的二分体系结构自然定义了一个优雅的采样过程，称为交替吉布斯采样（AGS），其中潜变量层的配置被采样为数据变量层的条件，反之亦然。我们在这里研究的性能AGS几个分析听话的模型借用统计力学。我们表明，标准的AGS是不是更有效的比经典的大都会黑斯廷斯（MH）采样的有效能源景观上定义的数据层。然而，RBM可以识别训练数据在其潜在空间中的有意义的表示。此外，使用这些表示，并结合吉布斯采样与MH算法在潜在的空间可以提高RBM的采样性能时，隐藏单元编码弱相关的数据特征。我们在三个数据集上说明了我们的发现：在机器学习中众所周知的Bars and Stripes和MNIST，以及在理论生物学中引入的所谓的Lattice Proteins，用于研究蛋白质中的序列到结构映射。
英文摘要：
Restricted Boltzmann Machines (RBM) are bi-layer neural networks used for the unsupervised learning of model distributions from data. The bipartite architecture of RBM naturally defines an elegant sampling procedure, called Alternating Gibbs Sampling (AGS), where the configurations of the latent-variable layer are sampled conditional to the data-variable layer, and vice versa. We study here the performance of AGS on several analytically tractable models borrowed from statistical mechanics. We show that standard AGS is not more efficient than classical Metropolis-Hastings (MH) sampling of the effective energy landscape defined on the data layer. However, RBM can identify meaningful representations of training data in their latent space. Furthermore, using these representations and combining Gibbs sampling with the MH algorithm in the latent space can enhance the sampling performance of the RBM when the hidden units encode weakly dependent features of the data. We illustrate our findings on three datasets: Bars and Stripes and MNIST, well known in machine learning, and the so-called Lattice Proteins, introduced in theoretical biology to study the sequence-to-structure mapping in proteins.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
