BOLDreams:基于视觉皮层剪枝的计算机模拟fMRI编码模型的梦境生成
BOLDreams: Dreaming with pruned in-silico fMRI Encoding Models of the Visual Cortex
AI总结:
本文基于NSD训练特征加权感受野神经编码模型,通过可解释AI技术(如“梦境生成”和积分梯度)分析驱动大脑预测信号的特征,并发现了一个在性能与简单性间取得平衡的极简模型。
摘要:
本文使用 Natural Scenes Dataset (NSD) 训练了一系列特征加权感受野神经编码模型。这些模型采用预训练的视觉或文本骨干网络,并通过感受野读出将提取的特征映射到体素空间。我们全面评估了此类模型,量化了在使用文本或图像等不同模态、切换微调、使用不同预训练骨干网络以及改变读出宽度时的性能变化。我们还利用可解释AI (XAI) 技术剖析每个模型,例如通过输入优化进行特征可视化(在AI文献中被称为“梦境生成”),以及使用积分梯度方法计算隐式注意力图,以说明哪些特征驱动了不同大脑区域的预测信号。这些XAI工具展示了驱动预测信号的具有生物学合理性的特征。遍历模型超参数空间揭示了一个极大极小模型的存在,它在保持性能的同时平衡了简单性。
英文摘要:
In this article we use the Natural Scenes Dataset (NSD) to train a family of feature-weighted receptive field neural encoding models. These models use a pre-trained vision or text backbone and map extracted features to the voxel space via receptive field readouts. We comprehensively assess such models, quantifying performance changes based on using different modalities like text or images, toggling finetuning, using different pre-trained backbones, and changing the width of the readout. We also dissect each model using explainable AI (XAI) techniques, such as feature visualization via input optimization, also referred to as ``dreaming'' in the AI literature, and the integrated gradients approach to calculate implicit attention maps to illustrate which features drive the predicted signal in different brain areas. These XAI tools illustrate biologically plausible features that drive the predicted signal. Traversing the model hyperparameter space reveals the existence of a maximally minimal model, balancing simplicity while maintaining performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。