视觉皮层中视觉刺激与神经反应的跨模态对齐
Cross-Modal Alignment between Visual Stimuli and Neural Responses in the Visual Cortex
AI总结:
本文提出视觉-神经对齐方法VNA,通过判别性编码和解码任务在视觉皮层中实现更精确的视觉刺激与神经反应映射。
摘要:
研究视觉刺激与视觉皮层神经反应之间的映射有助于更深入地理解生物视觉处理机制。大多数现有研究通过训练模型直接将视觉刺激编码为神经反应或将神经反应解码为视觉刺激来表征这种映射。然而,由于神经反应的变异性以及有限的神经记录技术,这些研究面临过拟合和缺乏泛化能力的问题。受这一挑战的启发,本文将任务从传统的直接编码和解码转向更具判别性的编码和解码,这更为合理。在此基础上,我们提出了一种跨模态对齐方法,称为视觉-神经对齐(VNA)。为了彻底测试三种方法(直接编码、直接解码和我们提出的方法VNA)在判别编码和解码任务上的性能,我们在三个侵入性视觉皮层数据集中进行了广泛的实验,涉及两种类型的实验动物(小鼠和猕猴)。结果表明,我们的VNA在三种方法中普遍优于直接编码和直接解码,表明我们的VNA可以最精确地表征这三种方法中的视觉-神经映射。
英文摘要:
Investigating the mapping between visual stimuli and neural responses in the visual cortex contributes to a deeper understanding of biological visual processing mechanisms. Most existing studies characterize this mapping by training models to directly encode visual stimuli into neural responses or decode neural responses into visual stimuli. However, due to neural response variability and limited neural recording techniques, these studies suffer from overfitting and lack generalizability. Motivated by this challenge, in this paper we shift the tasks from conventional direct encoding and decoding to discriminative encoding and decoding, which are more reasonable. And on top of this we propose a cross-modal alignment approach, named Visual-Neural Alignment (VNA). To thoroughly test the performance of the three methods (direct encoding, direct decoding, and our proposed VNA) on discriminative encoding and decoding tasks, we conduct extensive experiments on three invasive visual cortex datasets, involving two types of subject mammals (mice and macaques). The results demonstrate that our VNA generally outperforms direct encoding and direct decoding, indicating our VNA can most precisely characterize the above visual-neural mapping among the three methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。