基于DNN的腹侧视觉皮层编码模型的分布外泛化能力基准测试
Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex
AI总结:
本研究构建了大规模猕猴IT皮层神经响应数据集MacaqueITBench,系统评估了基于DNN的视觉皮层编码模型在多种分布偏移下的泛化能力,发现OOD性能显著下降,并指出预训练模型提取的图像表示余弦距离是泛化性能的强预测因子。
摘要:
我们表征了基于DNN的编码模型在预测视觉皮层神经元响应时的泛化能力。我们收集了MacaqueITBench,这是一个大规模神经群体响应数据集,记录了猕猴下颞叶(IT)皮层对超过300,000张图像的响应,包含在109个实验环节中向七只猕猴展示的8,233张独特自然图像。利用MacaqueITBench,我们通过将图像划分为分布外(OOD)训练集和测试集,研究了分布偏移对预测神经活动模型的影响。OOD划分包括多种图像可计算类型,如图像对比度、色调、强度、温度和饱和度。与在分布内测试图像(这些模型传统的评估方式)上的表现相比,模型在预测对分布外图像的神经元响应时表现更差,仅保留了在分布内测试图像上20%的性能。OOD偏移下的泛化性能可以通过一个简单的图像相似性度量很好地解释——从预训练物体识别模型中提取的图像表示之间的余弦距离,是不同分布偏移下神经预测性的强预测因子。图像数据集、神经元发放率记录和计算基准公开托管于:https://bit.ly/3zeutVd。
英文摘要:
We characterized the generalization capabilities of DNN-based encoding models when predicting neuronal responses from the visual cortex. We collected \textit{MacaqueITBench}, a large-scale dataset of neural population responses from the macaque inferior temporal (IT) cortex to over $300,000$ images, comprising $8,233$ unique natural images presented to seven monkeys over $109$ sessions. Using \textit{MacaqueITBench}, we investigated the impact of distribution shifts on models predicting neural activity by dividing the images into Out-Of-Distribution (OOD) train and test splits. The OOD splits included several different image-computable types including image contrast, hue, intensity, temperature, and saturation. Compared to the performance on in-distribution test images -- the conventional way these models have been evaluated -- models performed worse at predicting neuronal responses to out-of-distribution images, retaining as little as $20\%$ of the performance on in-distribution test images. The generalization performance under OOD shifts can be well accounted by a simple image similarity metric -- the cosine distance between image representations extracted from a pre-trained object recognition model is a strong predictor of neural predictivity under different distribution shifts. The dataset of images, neuronal firing rate recordings, and computational benchmarks are hosted publicly at: https://bit.ly/3zeutVd.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。