具有相关模式的Hopfield-Hebbian动力学的容量
Capacity of the Hopfield-Hebbian dynamics with correlated patterns
AI总结:
该研究针对相关模式场景,开发基于RDT的框架分析Hopfield-Hebbian动力学容量,明确了相关参数M、r对相变的影响,填补了高相关 regime 中谱方法的近似空白。
摘要:
在允许检索存储的二元模式存在小比例误差的情况下,Hopfield-Hebbian神经网络(NN)的联想记忆动力学[14]可实现线性缩放的容量。在大维度场景中,当均匀随机选择的二元模式彼此独立时,可靠存储的模式数量m与模式长度n的比值会得到著名的比例常数α_c≈0.14。尽管这种理想化场景已被充分理解,但对于经典统计力学的复制方法而言,具有相关/噪声数据集的更实际场景仍然具有挑战性。我们正是考虑了这种实际设置,其中每个原型(archetype)模式被替换为M个相关的示例,参数r控制着相关性的质量。我们开发了一种强大的基于随机对偶理论(Random Duality Theory, RDT)的框架来分析相关数据,仅在RDT提升的第一级就获得了容量表征,该表征与模拟动力学所展现的行为非常吻合。此外,这些表征明确包含相关参数M和r,这使得能够精确确定它们对基础动力学相变行为的影响。我们的结果也是现有文献的良好补充:为了规避具有挑战性的统计力学分析,文献[4]考虑了谱方法与复制方法的替代组合,我们证实文献[4]的优雅谱表征在低相关 regime 中是极好的近似;同时,我们也填补了高相关 regime 中谱方法与实际动力学性能偏离的空白。
英文摘要:
Allowing for small fraction of errors in retrieved stored binary patterns, Hopfield-Hebbian neural network (NN) associative memory dynamics [14] achieves linearly scaled capacity. The famous proportionality constant $α_c\approx 0.14$ is obtained in a large dimensional scenario as the ratio of the number of reliably stored patterns $m$ and the patterns' length $n$ under the assumption that uniformly randomly chosen binary patterns are \emph{independent} among themselves. While such an idealized scenario is now well understood, more practical scenarios with correlated/noisy datasets remaining challenging even for classical statistical mechanics replica approaches. We consider precisely such practical settings where each \emph{archetype} pattern is replaced with $M$ correlated \emph{examples} with parameter $r$ governing the quality of correlation. We develop a powerful Random Duality Theory (RDT) based framework to analyze correlated data and already on the first level of RDT lifting obtain capacity characterizations that remarkably well match behavior exhibited through simulated dynamics. Moreover, characterizations explicitly feature correlation parameters $M$ and $r$, which allows to precisely determine their effects on phase-transitioning behavior of the underlying dynamics. Our results are also a well-fitting complement to the existing literature. Namely, to circumvent challenging statistical mechanics analyses, an alternative combination of spectral and replica methods is considered in [4]. We confirm that [4]'s elegant spectral characterizations are excellent approximations in the low correlated regime. At the same time, we also close the gap in the regimes with higher correlation levels where spectral methods deviate from the practical dynamics performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。