arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1302.1156cs.NE
收藏

具有指数模式检索能力和迭代学习的非二元联想记忆:扩展结果

A Non-Binary Associative Memory with Exponential Pattern Retrieval Capacity and Iterative Learning: Extended Results

Amir Hesam Salavati, K. Raj Kumar, Amin Shokrollahi

摘要:

我们考虑非二元神经元网络的神经关联问题。在这里,任务是首先使用神经元网络记住一组模式,这些神经元的状态假设来自有限数量的整数级别的值。之后,同一个网络应该能够从嘈杂的版本中回忆起以前记忆的模式。这方面的先前工作考虑存储有限数量的纯随机模式,并已表明模式检索能力(可记忆的模式的最大数量)仅与网络中的神经元数量呈线性关系。在我们制定的问题,我们专注于利用冗余和内部结构的模式,以提高模式检索能力。我们的第一个结果表明,如果给定的模式有一个合适的线性代数结构,即包括一个子空间的所有可能的模式的集合,那么模式检索能力实际上是指数的神经元的数量。第二个结果扩展了以前的发现的情况下,模式具有弱的次要成分,即相关矩阵的最小特征值趋于零。我们将使用这些次要成分(或模式零空间的基向量)来增加模式检索能力和纠错能力。在学习阶段提出了一种迭代算法,在回忆阶段提出了两种简单的神经元更新算法。使用分析结果和模拟,我们表明,所提出的方法可以容忍相当数量的错误输入,同时能够记住指数大量的模式。

英文摘要:

We consider the problem of neural association for a network of non-binary neurons. Here, the task is to first memorize a set of patterns using a network of neurons whose states assume values from a finite number of integer levels. Later, the same network should be able to recall previously memorized patterns from their noisy versions. Prior work in this area consider storing a finite number of purely random patterns, and have shown that the pattern retrieval capacities (maximum number of patterns that can be memorized) scale only linearly with the number of neurons in the network. In our formulation of the problem, we concentrate on exploiting redundancy and internal structure of the patterns in order to improve the pattern retrieval capacity. Our first result shows that if the given patterns have a suitable linear-algebraic structure, i.e. comprise a sub-space of the set of all possible patterns, then the pattern retrieval capacity is in fact exponential in terms of the number of neurons. The second result extends the previous finding to cases where the patterns have weak minor components, i.e. the smallest eigenvalues of the correlation matrix tend toward zero. We will use these minor components (or the basis vectors of the pattern null space) to both increase the pattern retrieval capacity and error correction capabilities. An iterative algorithm is proposed for the learning phase, and two simple neural update algorithms are presented for the recall phase. Using analytical results and simulations, we show that the proposed methods can tolerate a fair amount of errors in the input while being able to memorize an exponentially large number of patterns.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑