Transformers与海马结构模型和神经表征的关系
Relating transformers to models and neural representations of the hippocampal formation
摘要：
许多松散地基于大脑网络的深度神经网络架构最近被证明可以复制在大脑中观察到的神经放电模式。Transformer神经网络是最令人兴奋和最有前途的新架构之一，它的开发没有考虑大脑。在这项工作中，我们表明，当配备重复位置编码时，Transformers可以复制海马结构精确调整的空间表示;最明显的是位置和网格细胞。此外，我们表明，这一结果并不令人惊讶，因为它与神经科学目前的海马模型密切相关。我们还表明，Transformer版本提供了显着的性能增益超过神经科学版本。这项工作继续绑定人工和大脑网络的计算，提供了对大脑皮层相互作用的新理解，并表明更广泛的皮层区域如何执行超出当前神经科学模型的复杂任务，如语言理解。
英文摘要：
Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the neuroscience version. This work continues to bind computations of artificial and brain networks, offers a novel understanding of the hippocampal-cortical interaction, and suggests how wider cortical areas may perform complex tasks beyond current neuroscience models such as language comprehension.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
