稀疏稀疏关联记忆中的反腐败信息
Combating Corrupt Messages in Sparse Clustered Associative Memories
摘要:
本文对Gripon和Berrou提出的基于神经网络的联想记忆进行了分析和扩展。这种联想记忆类似于著名的Willshaw模型,增加了分簇结构。在文献中,已经提出了两种检索方案的网络动力学,即总和和最大值和。他们都提供了更好的性能比Willshaw和Hopfield网络,当比较检索的情况下被认为是。以前的讨论和实验集中在擦除场景,其中部分消息被用作对网络的探测,希望检索完整的消息。在这方面,最大和在检索率方面大大优于和和。然而,我们观察到,当存在噪声和错误并且网络被损坏的探针查询时,最大值和面临严重的限制,因为其严格的激活规则阻止神经元在停用后重新恢复。在本文中,我们对不同的错误场景进行分类和分析,以便一致地处理擦除和损坏场景。我们对网络结构进行了修正,以提高检索率,代价是每个神经元增加一个额外的标量。之后,提出了五种不同的方法来处理腐败的探针。因此,我们扩展了网络的能力,也增加了检索过程的鲁棒性。然后,我们通过实验比较所有这些建议,并讨论在不同类型的错误下,每种方法的利弊。仿真结果表明,如果精心设计,网络是能够同时保持高检索率和低运行时间,即使被腐败的探针查询。
英文摘要:
In this paper we analyze and extend the neural network based associative memory proposed by Gripon and Berrou. This associative memory resembles the celebrated Willshaw model with an added partite cluster structure. In the literature, two retrieving schemes have been proposed for the network dynamics, namely sum-of-sum and sum-of-max. They both offer considerably better performance than Willshaw and Hopfield networks, when comparable retrieval scenarios are considered. Former discussions and experiments concentrate on the erasure scenario, where a partial message is used as a probe to the network, in the hope of retrieving the full message. In this regard, sum-of-max outperforms sum-of-sum in terms of retrieval rate by a large margin. However, we observe that when noise and errors are present and the network is queried by a corrupt probe, sum-of-max faces a severe limitation as its stringent activation rule prevents a neuron from reviving back into play once deactivated. In this manuscript, we categorize and analyze different error scenarios so that both the erasure and the corrupt scenarios can be treated consistently. We make an amendment to the network structure to improve the retrieval rate, at the cost of an extra scalar per neuron. Afterwards, five different approaches are proposed to deal with corrupt probes. As a result, we extend the network capability, and also increase the robustness of the retrieving procedure. We then experimentally compare all these proposals and discuss pros and cons of each approach under different types of errors. Simulation results show that if carefully designed, the network is able to preserve both a high retrieval rate and a low running time simultaneously, even when queried by a corrupt probe.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。