向CORnet传授人类fMRI表征以增强模型-大脑对齐
Teaching CORnet Human fMRI Representations for Enhanced Model-Brain Alignment
AI总结：
本研究提出基于CORnet的ReAlnet-fMRI模型，通过多层编码对齐框架利用人类fMRI数据优化，提升模型与大脑表征的相似性，为增强视觉模型类脑性、弥合计算机视觉与视觉神经科学差距提供可能。
摘要：
深度卷积神经网络（DCNNs）在目标识别任务中表现出优异性能，且已被发现与大脑视觉处理存在一定相似性。然而，DCNNs与人类视觉感知之间仍存在巨大差距。功能磁共振成像（fMRI）作为认知神经科学中广泛使用的技术，可记录人类视觉皮层在视觉感知过程中的神经激活情况。我们能否向DCNNs传授人类fMRI信号，以构建更类脑的模型？为回答这一问题，本研究提出了ReAlnet-fMRI，该模型基于当前最优（SOTA）视觉模型CORnet，通过基于多层编码的对齐框架利用人类fMRI数据进行优化。研究表明，该框架可有效使模型学习人类大脑表征。经过fMRI优化的ReAlnet-fMRI在被试内、被试间以及模态内、跨模态的模型-大脑（fMRI和EEG）对齐评估中，均表现出比CORnet和对照模型更高的与人类大脑的相似性。此外，我们开展了深入分析，探究ReAlnet-fMRI与CORnet在编码各类目标维度时内部表征的差异。这些发现为通过整合人类神经数据增强视觉模型的类脑性提供了可能，有助于弥合计算机视觉与视觉神经科学之间的差距。
英文摘要：
Deep convolutional neural networks (DCNNs) have demonstrated excellent performance in object recognition and have been found to share some similarities with brain visual processing. However, the substantial gap between DCNNs and human visual perception still exists. Functional magnetic resonance imaging (fMRI) as a widely used technique in cognitive neuroscience can record neural activation in the human visual cortex during the process of visual perception. Can we teach DCNNs human fMRI signals to achieve a more brain-like model? To answer this question, this study proposed ReAlnet-fMRI, a model based on the SOTA vision model CORnet but optimized using human fMRI data through a multi-layer encoding-based alignment framework. This framework has been shown to effectively enable the model to learn human brain representations. The fMRI-optimized ReAlnet-fMRI exhibited higher similarity to the human brain than both CORnet and the control model in within-and across-subject as well as within- and across-modality model-brain (fMRI and EEG) alignment evaluations. Additionally, we conducted an in-depth analyses to investigate how the internal representations of ReAlnet-fMRI differ from CORnet in encoding various object dimensions. These findings provide the possibility of enhancing the brain-likeness of visual models by integrating human neural data, helping to bridge the gap between computer vision and visual neuroscience.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
