arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1407.6513cs.NEcs.AI
收藏

卷积神经联想记忆:具有噪声容限的大容量

Convolutional Neural Associative Memories: Massive Capacity with Noise Tolerance

Swiss Federal Institute of Technology Zurich(苏黎世联邦理工学院);Ecole Polytechnique Federale de Lausanne(洛桑联邦理工学院)

Amin Karbasi, Amir Hesam Salavati, Amin Shokrollahi

摘要:

神经联想记忆的任务是使用神经元网络从它们的噪声版本中检索一组先前记忆的模式。一个理想的网络应该有能力:1)在模式到达时学习一组模式,2)从嘈杂的查询中检索正确的模式,3)最大化模式检索能力,同时保持响应查询的可靠性。大部分关于神经联想记忆的工作都集中在设计能够以限制检索能力为代价来记忆任何随机选择的模式集的网络。在本文中,我们表明,如果我们的目标是只记住那些具有固有冗余的模式(即,属于一个子空间),我们可以获得所有上述性质。这与以前的工作形成了鲜明的对比,以前的工作只能改善一个或两个方面,而牺牲了第三个方面。更具体地说,我们提出了一个卷积神经网络的基础上的框架,随着迭代算法,学习模式之间的冗余。由此产生的网络具有检索能力,在网络的大小上是指数的。此外,我们的网络的渐近误差校正性能是线性的模式的大小。然后,我们将我们的方法推广到处理近似位于子空间中的模式.这种扩展允许我们记住包含自然模式的数据集(例如,图像)。最后,我们报告了在合成和真实数据集上的实验结果,以支持我们的主张。

英文摘要:

The task of a neural associative memory is to retrieve a set of previously memorized patterns from their noisy versions using a network of neurons. An ideal network should have the ability to 1) learn a set of patterns as they arrive, 2) retrieve the correct patterns from noisy queries, and 3) maximize the pattern retrieval capacity while maintaining the reliability in responding to queries. The majority of work on neural associative memories has focused on designing networks capable of memorizing any set of randomly chosen patterns at the expense of limiting the retrieval capacity. In this paper, we show that if we target memorizing only those patterns that have inherent redundancy (i.e., belong to a subspace), we can obtain all the aforementioned properties. This is in sharp contrast with the previous work that could only improve one or two aspects at the expense of the third. More specifically, we propose framework based on a convolutional neural network along with an iterative algorithm that learns the redundancy among the patterns. The resulting network has a retrieval capacity that is exponential in the size of the network. Moreover, the asymptotic error correction performance of our network is linear in the size of the patterns. We then ex- tend our approach to deal with patterns lie approximately in a subspace. This extension allows us to memorize datasets containing natural patterns (e.g., images). Finally, we report experimental results on both synthetic and real datasets to support our claims.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑