发表机构
Max-Planck-Institute for the Physics of Complex Systems; Ludwig-Maximilians-Universität München; Gurdon Institute, University of Cambridge（马克斯·普朗克复杂系统物理研究所; 慕尼黑路德维希-马克西米利安大学; 剑桥大学格登研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文为现代Hopfield网络建立了统一的分岔理论，推导出一般凸对偶网络不动点的稳定性与分岔判据，并证明其能定量预测MNIST检索分岔及重现造血细胞层次组织。
AI 中文摘要
现代Hopfield网络为联想记忆、Transformer注意力机制、基于扩散的生成模型以及生物吸引子动力学提供了一个统一框架，通过一种共同的基于能量的动力学将这些系统联系起来，在该动力学中，状态向存储模式的加权组合更新。在这些设置中，网络动力学由能量景观的组织及其不动点的分岔决定。尽管这些分岔起着核心作用，但除了特定架构和理想化模式集合之外，关于这些分岔的一般理论仍然不可用。在这里，我们针对一般凸对偶现代Hopfield网络的不动点，在存储模式的一般统计下推导出稳定性和分岔判据。将该框架应用于随机、块相关和无限层次模式集合，我们展示了记忆相关性如何通过连续分岔系统地组织层次吸引子的出现。我们进一步证明，这些预测定量地描述了存储从MNIST采样的模式的MHN中的检索分岔，并重现了造血细胞身份的层次组织。我们的结果建立了现代Hopfield网络的一般分岔理论，并将不动点的组织确定为其计算和生物学行为背后的统一原则。
英文摘要
Modern Hopfield networks provide a unifying framework for associative memory, transformer attention, diffusion-based generative models, and biological attractor dynamics, linking these systems through a common energy-based dynamics in which states are updated toward weighted combinations of stored patterns. Across these settings, network dynamics is determined by the organization of the energy landscape and the bifurcations of its fixed points. Despite their central role, a general theory of these bifurcations has remained unavailable beyond specific architectures and idealized pattern ensembles. Here we derive stability and bifurcation criteria for the fixed points of general convex-dual Modern Hopfield networks for a general statistics of the stored patterns. Applying this framework to random, block-correlated, and infinitely hierarchical pattern ensembles, we show how memory correlations systematically organize the emergence of hierarchical attractors through successive bifurcations. We further demonstrate that these predictions quantitatively describe retrieval bifurcations in MHNs storing patterns sampled from MNIST, and recapitulate the hierarchical organization of hematopoietic cell identities. Our results establish a general bifurcation theory for Modern Hopfield networks and identify the organization of fixed points as a unifying principle underlying their computational and biological behavior.
Comments21 pages, 5 figures
