Hopfield模型中的平均场消息传递方程及其推广
Mean-field message-passing equations in the Hopfield model and its generalizations
摘要：
受使用限制玻尔兹曼机作为深度神经网络预处理算法的最新进展的启发，我们重新审视了这种机器中最好理解的平均场方程（置信传播和TAP方程），即神经网络的Hopfield模型，并明确了它们如何用作迭代消息传递算法，提供了一种快速计算神经元局部极化的方法。在“检索阶段”，神经元在一个记忆的模式的方向上，我们指出的信念传播和TAP方程之间的一个主要区别：信念传播方程的集合取决于检索的模式，而可以使用一个独特的一组TAP方程。这使得后一种方法更适合应用于受限玻尔兹曼机的学习过程。在Hopfield模型中记忆的模式不是独立的，而是通过组合结构相关的情况下，我们表明TAP方程必须修改。这种修改可以被看作是TAP方程中反应项的改变，或者更有趣的是，作为在具有几个隐藏层的图形模型上传递消息的结果，其中隐藏层的数量取决于记忆模式中相关性的深度。这种分层结构实际上是必要的，当人们处理更一般的限制玻尔兹曼机。
英文摘要：
Motivated by recent progress in using restricted Boltzmann machines as preprocessing algorithms for deep neural network, we revisit the mean-field equations (belief-propagation and TAP equations) in the best understood such machine, namely the Hopfield model of neural networks, and we explicit how they can be used as iterative message-passing algorithms, providing a fast method to compute the local polarizations of neurons. In the "retrieval phase" where neurons polarize in the direction of one memorized pattern, we point out a major difference between the belief propagation and TAP equations : the set of belief propagation equations depends on the pattern which is retrieved, while one can use a unique set of TAP equations. This makes the latter method much better suited for applications in the learning process of restricted Boltzmann machines. In the case where the patterns memorized in the Hopfield model are not independent, but are correlated through a combinatorial structure, we show that the TAP equations have to be modified. This modification can be seen either as an alteration of the reaction term in TAP equations, or, more interestingly, as the consequence of message passing on a graphical model with several hidden layers, where the number of hidden layers depends on the depth of the correlations in the memorized patterns. This layered structure is actually necessary when one deals with more general restricted Boltzmann machines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
