多模态神经网络更好地解释海马体中的多体元模式
Multimodal neural networks better explain multivoxel patterns in the hippocampus
摘要：
人类海马体拥有“概念细胞”，当受到属于特定概念的刺激时，神经元会放电，而不管其形式如何。最近，在称为CLIP的多模态网络中发现了类似的概念细胞（Radford et al.，2021年）。在这里，我们问CLIP是否能比纯粹的视觉（或语言）模型更好地解释人类海马体的fMRI活动。我们将我们的分析扩展到一系列公开的单模态和多模态模型。我们证明，“多模态”突出作为一个关键组成部分时，评估网络的能力来解释海马体中的多体素活动。
英文摘要：
The human hippocampus possesses "concept cells", neurons that fire when presented with stimuli belonging to a specific concept, regardless of the modality. Recently, similar concept cells were discovered in a multimodal network called CLIP (Radford et at., 2021). Here, we ask whether CLIP can explain the fMRI activity of the human hippocampus better than a purely visual (or linguistic) model. We extend our analysis to a range of publicly available uni- and multi-modal models. We demonstrate that "multimodality" stands out as a key component when assessing the ability of a network to explain the multivoxel activity in the hippocampus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
