Dense Associative Memory with biased patterns: a Replica Symmetric analysis
Università del Salento(萨伦托大学);Istituto Nazionale di Fisica Nucleare, Sezione di Lecce(意大利国家核物理研究所,莱切分部);Fondazione CMCC - Centro Euro-Mediterraneo sui Cambiamenti Climatici(CMCC基金会-欧洲地中海气候变化中心);Università di Bologna(博洛尼亚大学)
Linda Albanese, Andrea Alessandrelli, Federico Carella
我们研究了在高存储 regime 下,存储模式偏置(即模式条目不对称分布于零周围)的密集高阶联想记忆。在此设定下,标准 Hebbian 预制必须通过重新中心和重新缩放模式条目进行修改,并在哈密顿量中引入额外项以确保网络平均活动与存储模式平均活动的一致性。作为第一步,我们在零温极限下进行信号与噪声分析,证明偏置通过乘法修正因子(1-b^2)^P降低有效存储容量,同时保持与系统规模的超线性缩放。随后,我们通过 Guerra 的插值方法在 Replica 对称框架内推导出淬火统计压力,并获得相关序参数的自洽方程。解析处理证实了信号与噪声论证的启发式预测,显示相同的偏置依赖重正化自然出现在交叉噪声方差中。最后,我们讨论了该模型的结果相变行为及其对模型检索性能的影响。
英文摘要:
We investigate dense higher-order associative memories in the high storage regime when the stored patterns are biased, namely when the entries of the patterns are not symmetrically distributed around zero. In this setting, the standard Hebbian prescription must be modified by recentering and rescaling the pattern entries, and an additional term must be introduced in the Hamiltonian to enforce consistency between the average activity of the network and that of the stored patterns. As a first step, we perform a signal-to-noise analysis in the zero-temperature limit and show that the bias reduces the effective storage capacity through a multiplicative correction factor (1-b^2)^P, while preserving the superlinear scaling with the system size. We then derive the quenched statistical pressure within the Replica Symmetric framework by means of Guerra's interpolation method and obtain the corresponding self consistency equations for the relevant order parameters. The analytical treatment confirms the heuristic prediction of the signal-to-noise argument, showing that the same bias dependent renormalization naturally emerges in the variance of the cross-talk noise. Finally, we discuss the resulting phase behavior of the model and its implications for retrieval performance in the model.
Partial annealing and pattern decorrelation in associative neural networks
Università del Salento(萨伦托大学);Istituto Nazionale di Fisica Nucleare, Sezione di Lecce(意大利国家核物理研究所莱切分部);Sapienza Università di Roma(罗马第一大学);Istituto Nazionale di Fisica Nucleare, Sezione di Roma 1(意大利国家核物理研究所罗马一分部);CNR-Nanotec, unità di Lecce(国家研究委员会纳米技术研究所莱切单元);Université Paris-Saclay(巴黎萨克雷大学)
Tohoku University(东北大学);Institute of Science Tokyo(东京科学大学);Kumamoto University(熊本大学);Sigma-i Co., Ltd.(Sigma-i有限公司);Sapienza University of Rome(罗马第一大学);CNR–Nanotec, Rome unit(国家研究委员会纳米技术研究所罗马分部);INFN–Sezione di Roma 1(意大利国家核物理研究所罗马第一分部)
Do Hopfield Networks Dream of Stored Patterns? A Statistical-Mechanical Theory of Dreaming in Multidirectional Associative Memories
Sapienza Università di Roma(罗马大学);Istituto Nazionale di Fisica Nucleare(意大利国家核物理研究所);Istituto Nazionale d’Alta Matematica, GNFM(意大利高等数学研究所);Università del Salento(莱切萨伦托大学)
Explosive neural networks via higher-order interactions in curved statistical manifolds
BCAM -- Basque Center for Applied Mathematics(巴斯克应用数学中心);IKERBASQUE, Basque Foundation for Science(伊克尔巴斯科科学基金会);Araya Inc.(Araya公司);Imperial College London(伦敦帝国理工学院);University of Sussex(萨塞克斯大学);University of Oxford(牛津大学);Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS)(生物与社会系统中智能行为原理研究所);Kyoto University(京都大学);Hokkaido University(北海道大学)