arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1812.09077cond-mat.dis-nncs.AIstat.ML
收藏

做梦的神经网络:严格的结果

Dreaming neural networks: rigorous results

Sapienza Università di Roma(罗马第一大学);GNFM-INdAM Sezione di Roma(意大利国家数学物理研究所罗马分会);Università del Salento(萨兰托大学);GNFM-INdAM Sezione di Lecce(意大利国家数学物理研究所莱切分会);INFN, Istituto Nazionale di Fisica Nucleare(意大利国家核物理研究所)

Elena Agliari, Francesco Alemanno, Adriano Barra, Alberto Fachechi

摘要:

最近提出了一种联想神经网络的日常程序:网络在清醒状态下进行赫布学习(因此表现为标准的Hopfield模型),然后在睡眠状态下优化信息存储,巩固纯模式并删除虚假模式:这迫使突触矩阵崩溃为投影矩阵(最终接近Kanter-Sompolinksy模型)。这个过程保持了基于赫布的学习(生物学上必须的),但是,通过利用(适当程式化的)睡眠阶段,仍然达到了最大的临界容量(对于对称交互)。到目前为止,这一新兴的图片(以及大量的论文对unlearning技术)的支持完全是由具有挑战性的路线,例如,主要是复制技巧分析和数值模拟:在这里,我们广泛依赖于格拉的插值技术开发的神经网络,特别是,我们扩展了广义随机稳定性的方法的情况下。将我们的描述限制在复制对称近似(先前的描述所在)内,关于这种概括(以及先前存在的主题变化)的描述在这里得到了完全证实。此外,仍然依赖于格拉的计划,我们开发了一个系统的波动分析,检查遍历性被打破(分析完全没有在以前的调查)。我们发现,只要网络是清醒的,遍历性是有界的Amit-Gutfreund-Sompolinsky临界线(这是应该的),但是,当网络休眠时,休眠通过扩展检索区域和遍历区域来破坏自旋玻璃状态:在整个睡眠会话之后,唯一幸存的区域是检索区域和遍历区域,这使得网络能够实现完美的检索机制(可存储模式的数量等于网络中神经元的数量)。

英文摘要:

Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it consolidates pure patterns and removes spurious ones: this forces the synaptic matrix to collapse to the projector one (ultimately approaching the Kanter-Sompolinksy model). This procedure keeps the learning Hebbian-based (a biological must) but, by taking advantage of a (properly stylized) sleep phase, still reaches the maximal critical capacity (for symmetric interactions). So far this emerging picture (as well as the bulk of papers on unlearning techniques) was supported solely by mathematically-challenging routes, e.g. mainly replica-trick analysis and numerical simulations: here we rely extensively on Guerra's interpolation techniques developed for neural networks and, in particular, we extend the generalized stochastic stability approach to the case. Confining our description within the replica symmetric approximation (where the previous ones lie), the picture painted regarding this generalization (and the previously existing variations on theme) is here entirely confirmed. Further, still relying on Guerra's schemes, we develop a systematic fluctuation analysis to check where ergodicity is broken (an analysis entirely absent in previous investigations). We find that, as long as the network is awake, ergodicity is bounded by the Amit-Gutfreund-Sompolinsky critical line (as it should), but, as the network sleeps, sleeping destroys spin glass states by extending both the retrieval as well as the ergodic region: after an entire sleeping session the solely surviving regions are retrieval and ergodic ones and this allows the network to achieve the perfect retrieval regime (the number of storable patterns equals the number of neurons in the network).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑