arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.04520cs.LGcs.AIstat.ML
收藏

关于现代Hopfield模型的计算极限:一项细粒度复杂度分析

On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis

Northwestern University(西北大学);National Taiwan University(台湾大学);Adobe Research(奥多比研究院)

Jerry Yao-Chieh Hu, Thomas Lin, Zhao Song, Han Liu

AI总结:

本文从细粒度复杂度分析研究现代Hopfield模型记忆检索的计算极限,基于模式范数刻画效率相变行为,确立上界准则,证明在SETH假设下次二次变体仅存于此准则下,并给出低秩近似高效构造示例及误差与容量证明。

摘要:

我们从细粒度复杂度分析出发,研究了现代Hopfield模型记忆检索动力学的计算极限。我们的核心贡献是刻画了所有可能的现代Hopfield模型在效率上的相变行为,该行为基于模式的范数。具体而言,我们为输入查询模式和记忆模式的范数建立了一个上界准则。假设强指数时间假设(SETH)成立,仅在此准则之下,现代Hopfield模型的次二次(高效)变体才存在。为展示我们的理论,当高效准则成立时,我们提供了一个使用低秩近似构建现代Hopfield模型高效构造的正式示例。这包括对计算时间下界的推导,该时间随max{存储记忆模式数量, 输入查询序列长度}呈线性扩展。此外,我们证明了其记忆检索误差界限和指数级记忆容量。

英文摘要:

We investigate the computational limits of the memory retrieval dynamics of modern Hopfield models from the fine-grained complexity analysis. Our key contribution is the characterization of a phase transition behavior in the efficiency of all possible modern Hopfield models based on the norm of patterns. Specifically, we establish an upper bound criterion for the norm of input query patterns and memory patterns. Only below this criterion, sub-quadratic (efficient) variants of the modern Hopfield model exist, assuming the Strong Exponential Time Hypothesis (SETH). To showcase our theory, we provide a formal example of efficient constructions of modern Hopfield models using low-rank approximation when the efficient criterion holds. This includes a derivation of a lower bound on the computational time, scaling linearly with $\max\{$# of stored memory patterns, length of input query sequence$\}$. In addition, we prove its memory retrieval error bound and exponential memory capacity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑