arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.01304cond-mat.dis-nncs.LGcs.NE
收藏

监督赫布学习

Supervised Hebbian Learning

Università del Salento(萨兰托大学);Istituto Nazionale di Fisica Nucleare(国家核物理研究所);Sapienza Università di Roma(罗马第一大学);Istituto Nazionale d’Alta Matematica(国家高等数学研究所);Bar-Ilan University(巴伊兰大学)

Francesco Alemanno, Miriam Aquaro, Ido Kanter, Adriano Barra, Elena Agliari

AI总结:

本文提出一种监督赫布学习协议,使霍普菲尔德网络能从模糊示例中推断原型,并通过相图、受限玻尔兹曼机等价性及副本隐藏层将 MNIST 分类准确率从 75% 提升至 95%。

摘要:

在神经网络文献中,赫布学习传统上指霍普菲尔德模型及其推广存储原型(即仅经历一次便形成突触矩阵的确定模式)的过程。然而,机器学习中的“学习”一词指的是机器从所提供的数据集(例如由这些原型的模糊示例组成)中提取特征的能力,以便对不可用的原型形成自身的表示。在此,给定一组示例,我们定义了一种监督学习协议,通过该协议霍普菲尔德网络能够推断原型,并且我们检测出正确的控制参数(包括数据集的规模和质量),以描绘系统性能的相图。我们还证明,对于无结构数据集,配备该监督学习规则的霍普菲尔德模型等价于受限玻尔兹曼机,这提示了一种最优且可解释的训练流程。最后,该方法被推广到结构化数据集:我们强调在所分析的数据集中存在一种准超度量组织(令人联想到副本对称破缺),并因此引入一个额外的“副本隐藏层”用于其(部分)解纠缠,结果表明该做法将 MNIST 分类准确率从 75% 提高到 95%,并为深度架构提供了新的视角。

英文摘要:

In neural network's Literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term "Learning" in Machine Learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here, given a sample of examples, we define a supervised learning protocol by which the Hopfield network can infer the archetypes, and we detect the correct control parameters (including size and quality of the dataset) to depict a phase diagram for the system performance. We also prove that, for structureless datasets, the Hopfield model equipped with this supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal and interpretable training routine. Finally, this approach is generalized to structured datasets: we highlight a quasi-ultrametric organization (reminiscent of replica-symmetry-breaking) in the analyzed datasets and, consequently, we introduce an additional "replica hidden layer" for its (partial) disentanglement, which is shown to improve MNIST classification from 75% to 95%, and to offer a new perspective on deep architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑