arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.16880cond-mat.dis-nncs.LG
收藏

随机特征Hopfield模型中的存储与学习相变

Storage and Learning phase transitions in the Random-Features Hopfield Model

University of Rome ‘La Sapienza’(罗马第一大学);CNR-NANOTEC, Institute of Nanotechnology(意大利国家研究委员会纳米技术研究所);Politecnico di Torino(都灵理工大学);Bocconi University(博科尼大学)

Matteo Negri, Clarissa Lauditi, Gabriele Perugini, Carlo Lucibello, Enrico M. Malatesta

AI总结:

该研究受流形假设启发提出随机特征Hopfield模型,用统计物理复制方法推导其相图,发现了可无监督推断投影特征的学习相变新相态。

摘要:

Hopfield模型是神经网络的范式模型,数十年来在统计物理、神经科学和机器学习领域被广泛研究。受机器学习中流形假设的启发,我们提出并研究了标准设定的一种推广形式,将其命名为Random-Features Hopfield Model(随机特征Hopfield模型)。该模型中,长度为$N$的$P$个二值模式,是通过对从维度为$D$的隐空间中采样的高斯向量施加随机投影再经过非线性变换生成的。我们利用统计物理中的复制方法(replica method),推导了在$P,N,D\to\infty$且比例$α=P/N$、$α_D=D/N$固定的极限下模型的相图。除了常见的可从初始受损状态动态恢复模式的检索相之外,我们还发现了一种新的相,在该相中可以恢复表征投影的特征。我们将这一现象称为学习相变,因为这些特征并未被显式提供给模型,而是以无监督的方式从模式中推断得到的。

英文摘要:

The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. Inspired by the manifold hypothesis in machine learning, we propose and investigate a generalization of the standard setting that we name Random-Features Hopfield Model. Here $P$ binary patterns of length $N$ are generated by applying to Gaussian vectors sampled in a latent space of dimension $D$ a random projection followed by a non-linearity. Using the replica method from statistical physics, we derive the phase diagram of the model in the limit $P,N,D\to\infty$ with fixed ratios $α=P/N$ and $α_D=D/N$. Besides the usual retrieval phase, where the patterns can be dynamically recovered from some initial corruption, we uncover a new phase where the features characterizing the projection can be recovered instead. We call this phenomena the learning phase transition, as the features are not explicitly given to the model but rather are inferred from the patterns in an unsupervised fashion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑