arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.01743cond-mat.dis-nncs.NEstat.ML
收藏

基于力学类比的Hopfield神经网络的相对论扩展

A relativistic extension of Hopfield neural networks via the mechanical analogy

Adriano Barra, Matteo Beccaria, Alberto Fachechi

摘要:

我们提出了Hopfield模型的成本函数的修改,其显着特征在其泰勒展开中闪耀,并导致与交替符号的多对交互,这表明了一个统一的框架来处理深度学习和网络修剪。在我们的分析中,我们在很大程度上依赖于哈密顿-雅可比对应的统计模型与力学系统。在这幅图中,我们的模型只不过是原始Hopfield模型的相对论扩展(其成本函数是Mattis磁化中的二次形式,它模仿了自由粒子的非相对论哈密顿量)。我们专注于低存储制度和解决模型解析利用机械类比,从而获得一个完整的表征的自由能和相关的自洽方程的热力学极限。在数值方面,我们用MC模拟测试了我们的建议的性能,结果表明,由于在这个扩展框架中存在遗忘贡献,虚假状态的稳定性(限制了标准Hebbian构造的能力)明显降低。

英文摘要:

We propose a modification of the cost function of the Hopfield model whose salient features shine in its Taylor expansion and result in more than pairwise interactions with alternate signs, suggesting a unified framework for handling both with deep learning and network pruning. In our analysis, we heavily rely on the Hamilton-Jacobi correspondence relating the statistical model with a mechanical system. In this picture, our model is nothing but the relativistic extension of the original Hopfield model (whose cost function is a quadratic form in the Mattis magnetization which mimics the non-relativistic Hamiltonian for a free particle). We focus on the low-storage regime and solve the model analytically by taking advantage of the mechanical analogy, thus obtaining a complete characterization of the free energy and the associated self-consistency equations in the thermodynamic limit. On the numerical side, we test the performances of our proposal with MC simulations, showing that the stability of spurious states (limiting the capabilities of the standard Hebbian construction) is sensibly reduced due to presence of unlearning contributions in this extended framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑