arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1709.05340cs.NEcs.LG
收藏

Hopfield网络的动态容量估计

Dynamic Capacity Estimation in Hopfield Networks

Columbia University(哥伦比亚大学)

Saarthak Sarup, Mingoo Seok

摘要:

理解神经网络的记忆容量仍然是实现人工智能系统的一个具有挑战性的问题。在本文中,我们解决了Hopfield网络的容量的概念,并提出了一个动态的方法来监控网络的容量。我们定义我们的理解能力的最大数量的存储模式,可以检索时,探测存储的模式。这方面的先前工作已经提出了依赖于神经元计数$N$的静态表达式,迫使网络设计者在设置网络容量时假设最坏情况下的偏置和相关性输入特性。相反,我们的模型与学习Hopfield网络同时运行,并根据存储的模式得出容量估计。通过不断更新与存储模式相关的串扰,我们的模型可以保护网络免受内存痕迹的影响,并超过其容量。我们使用人工生成的随机模式来模拟我们的模型,这些模式可以设置为所需的偏差和相关性,并观察到容量估计值在93%到97%之间的准确性。因此,与静态和最坏情况下的容量估计相比,我们的模型使Hopfield网络的记忆效率提高了一倍,同时最大限度地降低了丢失模式的风险。

英文摘要:

Understanding the memory capacity of neural networks remains a challenging problem in implementing artificial intelligence systems. In this paper, we address the notion of capacity with respect to Hopfield networks and propose a dynamic approach to monitoring a network's capacity. We define our understanding of capacity as the maximum number of stored patterns which can be retrieved when probed by the stored patterns. Prior work in this area has presented static expressions dependent on neuron count $N$, forcing network designers to assume worst-case input characteristics for bias and correlation when setting the capacity of the network. Instead, our model operates simultaneously with the learning Hopfield network and concludes on a capacity estimate based on the patterns which were stored. By continuously updating the crosstalk associated with the stored patterns, our model guards the network from overwriting its memory traces and exceeding its capacity. We simulate our model using artificially generated random patterns, which can be set to a desired bias and correlation, and observe capacity estimates between 93% and 97% accurate. As a result, our model doubles the memory efficiency of Hopfield networks in comparison to the static and worst-case capacity estimate while minimizing the risk of lost patterns.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑