We present a Hopfield-like autoassociative network for memories representing examples of concepts. Each memory is encoded by two activity patterns with complementary properties. The first is dense and correlated across examples within concepts, and the second is sparse and exhibits no correlation among examples. The network stores each memory as a linear combination of its encodings. During retrieval, the network recovers sparse or dense patterns with a high or low activity threshold, respectively. As more memories are stored, the dense representation at low threshold shifts from examples to concepts, which are learned from accumulating common example features. Meanwhile, the sparse representation at high threshold maintains distinctions between examples due to the high capacity of sparse, decorrelated patterns. Thus, a single network can retrieve memories at both example and concept scales and perform heteroassociation between them. We obtain our results by deriving macroscopic mean-field equations that yield capacity formulas for sparse examples, dense examples, and dense concepts. We also perform network simulations that verify our theoretical results and explicitly demonstrate the capabilities of the network.
McGovern Institute for Brain Research, Massachusetts Institute of Technology（麻省理工学院麦戈文脑科学研究所）；The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Massachusetts Institute of Technology（麻省理工学院美国国家科学基金会人工智能基础相互作用研究所）
John A. Paulson School of Engineering and Applied Sciences（约翰·A·保尔森工程与应用科学学院）；Center for Brain Science（脑科学中心）；Department of Physics（物理系）；MIT-IBM Watson AI Lab, IBM Research（MIT-IBM沃森人工智能实验室，IBM研究院）；Kempner Institute for the Study of Natural and Artificial Intelligence（肯普纳自然与人工智能研究所）
Input-Driven Dynamics for Robust Memory Retrieval in Hopfield Networks
Department of Information Engineering（信息工程系）；Università degli Studi di Padova（帕多瓦大学）；Center for Control, Dynamical Systems and Computation（控制、动态系统与计算中心）；University of California at Santa Barbara（加州大学圣巴巴拉分校）
Large Associative Memory Problem in Neurobiology and Machine Learning
MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；IBM Research（IBM研究院）；Princeton Neuroscience Institute（普林斯顿神经科学研究所）；Princeton University（普林斯顿大学）
密集关联记忆或现代Hopfield网络允许存储和可靠检索指数级大（在特征空间的维度上）数量的记忆。与此同时，它们的朴素实现是非生物的，因为它似乎需要神经元之间存在多体突触连接。我们表明，这些模型是一个更微观的（书面的生物自由度）理论，有额外的（隐藏的）神经元，只需要两个身体之间的相互作用的有效描述。由于这个原因，我们提出的微观理论是一个有效的模型，大联想记忆与一定程度的生物相容性。我们的网络的动力学和它的降维等效都最小化能量（李雅普诺夫）函数。当某些动态变量（隐藏神经元）从我们的微观理论中整合出来时，可以恢复许多以前在文献中讨论过的模型，例如“Hopfield Networks is All You Need”论文中提出的模型。我们还提供了另一种推导的能量函数和更新规则中提出的上述文件，并澄清这一类的各种模型之间的关系。
