发表机构
Stanford University（斯坦福大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究解析了密集Hopfield网络中的层次记忆稳定性条件，发现仅需准多项式信息即可实现原型级泛化，并在Fashion-MNIST上验证了相图相似性。
AI 中文摘要
层次相关性是任何现实数据模型的普遍特征，而联想记忆模型如何学习这些相关性并超越它们以构建新的合理图像，是理解扩散模型等更复杂现代架构的重要一步。我们考虑一个层次化记忆模型，其中记忆被采样并存储在一个具有多项式激活的密集Hopfield网络中。我们解析地推导了该层次每一级局部稳定（即它们是局部能量最小值）的条件。我们使用原型重建作为泛化的最小模型，并发现仅需准多项式量的信息即可超越特定记忆甚至层次中的特定组进行泛化。我们在Fashion-MNIST数据上观察到，在记忆数量、激活函数尖锐度（多项式次数）方面存在定性类似的相图。
英文摘要
Hierarchical correlations are a universal feature of any realistic model of data, and the question of how associative memory models may learn these correlations and generalize beyond them to construct new sensible images is an important step towards understanding more complex modern architectures such as diffusion models. We consider a hierarchical model for memories which are sampled and stored in a dense Hopfield network with polynomial activation. We analytically derive conditions for each level of this hierarchy to be locally stable - that is they are local energy minima. We use prototype reconstruction as a minimal model of generalization and we find that it takes only a quasi-polynomial amount of information to generalize beyond particular memories and even particular groups in the hierarchy. We observe a qualitatively analogous phase diagram in the number of memories, sharpness of the activation function (polynomial degree) for data from Fashion-MNIST.
Comments5 pages, 19 page supplementary information. Additional citation requests welcome
