arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1105.2790cond-mat.dis-nncs.AI
收藏

Hopfield网络与Boltzmann机的等价性

On the equivalence of Hopfield Networks and Boltzmann Machines

Sapienza Università di Roma(罗马第一大学);Yale University(耶鲁大学);Alma Mater Studiorum Università di Bologna(博洛尼亚大学)

Adriano Barra, Alberto Bernacchia, Enrica Santucci, Pierluigi Contucci

摘要:

一种特定类型的神经网络,受限玻尔兹曼机(RBM),用于机器学习中的分类和特征检测。RBM的特点是可见和隐藏单元的单独层,能够有效地学习观察数据的生成模型。我们研究了一个“混合”版本的RBM的,其中隐藏的单位是模拟和可见的单位是二进制的,我们表明,热力学的可见单位是等效的Hopfield网络,其中N个可见的单位是神经元和P隐藏的单位是学习模式。我们应用随机稳定性的方法来推导模型的热力学,通过考虑这种技术的正式扩展到多组存储模式的情况下,这可能会作为相关集的研究基准。我们的研究结果表明,模拟Hopfield网络的动力学,需要更新N个神经元和存储N(N-1)/2突触,可以通过一个混合玻尔兹曼机完成,需要更新N+P神经元,但只有NP突触的存储。此外,众所周知的Hopfield网络的玻璃化转变在玻尔兹曼机中也有对应的:它对应于选择隐藏层和可见层的相对大小的最佳标准,解决了模型的灵活性和通用性之间的权衡。Hopfield模型的低存储阶段对应于很少的隐藏单元,因此是过度约束的RBM,而自旋玻璃阶段(太多的隐藏单元)对应于无约束的RBM,易于对观测数据进行过拟合。

英文摘要:

A specific type of neural network, the Restricted Boltzmann Machine (RBM), is implemented for classification and feature detection in machine learning. RBM is characterized by separate layers of visible and hidden units, which are able to learn efficiently a generative model of the observed data. We study a "hybrid" version of RBM's, in which hidden units are analog and visible units are binary, and we show that thermodynamics of visible units are equivalent to those of a Hopfield network, in which the N visible units are the neurons and the P hidden units are the learned patterns. We apply the method of stochastic stability to derive the thermodynamics of the model, by considering a formal extension of this technique to the case of multiple sets of stored patterns, which may act as a benchmark for the study of correlated sets. Our results imply that simulating the dynamics of a Hopfield network, requiring the update of N neurons and the storage of N(N-1)/2 synapses, can be accomplished by a hybrid Boltzmann Machine, requiring the update of N+P neurons but the storage of only NP synapses. In addition, the well known glass transition of the Hopfield network has a counterpart in the Boltzmann Machine: It corresponds to an optimum criterion for selecting the relative sizes of the hidden and visible layers, resolving the trade-off between flexibility and generality of the model. The low storage phase of the Hopfield model corresponds to few hidden units and hence a overly constrained RBM, while the spin-glass phase (too many hidden units) corresponds to unconstrained RBM prone to overfitting of the observed data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑