arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.09983q-bio.NCcs.AIcs.CVcs.HCcs.LGeess.SP
收藏

用于预测真实图片大脑反应的深度学习架构探索与比较

Exploration and Comparison of Deep Learning Architectures to Predict Brain Response to Realistic Pictures

Università degli Studi di Milano Bicocca(米兰比可卡大学);University of Essex(埃塞克斯大学)

Riccardo Chimisso, Sathya Buršić, Paolo Marocco, Giuseppe Vizzari, Dimitri Ognibene

AI总结:

本文面向 Algonauts Challenge 2023,探索多种深度学习架构预测真实图片引起的大脑反应,最终发现以 CLIP 图像嵌入为输入、按受试者半球 ROI 分别构建的简单全连接线性模型效果最佳,虽超过基线但关联仍不够稳健。

摘要:

我们针对 Algonauts Challenge 2023,开展了用于预测真实图像大脑反应的机器学习架构探索。研究过程中,我们对多种预训练模型进行了大量实验。最初,我们采用较简单的模型来预测大脑活动,随后逐步引入更复杂的架构,利用现有数据以及由大规模预训练模型生成的嵌入。我们遇到了机器学习问题中常见的困难,例如正则化与过拟合,也遇到了该挑战特有的问题,例如难以组合多种输入编码,以及输出具有高维度、结构不清晰和噪声较大的特点。为了克服这些问题,我们测试了单边 3D 位置模型、多感兴趣区域(ROI)模型和半球预测模型,但最终发现,采用多个简单模型,每个模型分别针对每个受试者大脑每个半球中的一个 ROI,能够取得最佳结果——即一个以 CLIP 生成的图像嵌入作为输入的全连接线性层。尽管我们超过了挑战赛基线,但结果仍未能与数据建立起稳健的关联。

英文摘要:

We present an exploration of machine learning architectures for predicting brain responses to realistic images on occasion of the Algonauts Challenge 2023. Our research involved extensive experimentation with various pretrained models. Initially, we employed simpler models to predict brain activity but gradually introduced more complex architectures utilizing available data and embeddings generated by large-scale pre-trained models. We encountered typical difficulties related to machine learning problems, e.g. regularization and overfitting, as well as issues specific to the challenge, such as difficulty in combining multiple input encodings, as well as the high dimensionality, unclear structure, and noisy nature of the output. To overcome these issues we tested single edge 3D position-based, multi-region of interest (ROI) and hemisphere predictor models, but we found that employing multiple simple models, each dedicated to a ROI in each hemisphere of the brain of each subject, yielded the best results - a single fully connected linear layer with image embeddings generated by CLIP as input. While we surpassed the challenge baseline, our results fell short of establishing a robust association with the data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑