Exploration and Comparison of Deep Learning Architectures to Predict Brain Response to Realistic Pictures
Università degli Studi di Milano Bicocca(米兰比可卡大学);University of Essex(埃塞克斯大学)
Riccardo Chimisso, Sathya Buršić, Paolo Marocco, Giuseppe Vizzari, Dimitri Ognibene
备注:
Submitted to The Algonauts Project 2023 - Exploration and Comparison of Deep Learning Architectures to Predict Brain Response to Realistic Pictures - http://algonauts.csail.mit.edu/
本文面向 Algonauts Challenge 2023,探索多种深度学习架构预测真实图片引起的大脑反应,最终发现以 CLIP 图像嵌入为输入、按受试者半球 ROI 分别构建的简单全连接线性模型效果最佳,虽超过基线但关联仍不够稳健。
摘要:
我们针对 Algonauts Challenge 2023,开展了用于预测真实图像大脑反应的机器学习架构探索。研究过程中,我们对多种预训练模型进行了大量实验。最初,我们采用较简单的模型来预测大脑活动,随后逐步引入更复杂的架构,利用现有数据以及由大规模预训练模型生成的嵌入。我们遇到了机器学习问题中常见的困难,例如正则化与过拟合,也遇到了该挑战特有的问题,例如难以组合多种输入编码,以及输出具有高维度、结构不清晰和噪声较大的特点。为了克服这些问题,我们测试了单边 3D 位置模型、多感兴趣区域(ROI)模型和半球预测模型,但最终发现,采用多个简单模型,每个模型分别针对每个受试者大脑每个半球中的一个 ROI,能够取得最佳结果——即一个以 CLIP 生成的图像嵌入作为输入的全连接线性层。尽管我们超过了挑战赛基线,但结果仍未能与数据建立起稳健的关联。
英文摘要:
We present an exploration of machine learning architectures for predicting brain responses to realistic images on occasion of the Algonauts Challenge 2023. Our research involved extensive experimentation with various pretrained models. Initially, we employed simpler models to predict brain activity but gradually introduced more complex architectures utilizing available data and embeddings generated by large-scale pre-trained models. We encountered typical difficulties related to machine learning problems, e.g. regularization and overfitting, as well as issues specific to the challenge, such as difficulty in combining multiple input encodings, as well as the high dimensionality, unclear structure, and noisy nature of the output. To overcome these issues we tested single edge 3D position-based, multi-region of interest (ROI) and hemisphere predictor models, but we found that employing multiple simple models, each dedicated to a ROI in each hemisphere of the brain of each subject, yielded the best results - a single fully connected linear layer with image embeddings generated by CLIP as input. While we surpassed the challenge baseline, our results fell short of establishing a robust association with the data.
Brain Tumor Segmentation with Deep Neural Networks
Université de Sherbrooke(舍布鲁克大学);École Normale supérieure(巴黎高等师范学院);Université de Montréal(蒙特利尔大学);École polytechnique(巴黎综合理工学院);École Polytechnique de Montréal(蒙特利尔工程学院);Twitter(推特)
Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics
Neuroscience Institute, Carnegie Mellon University(卡内基梅隆大学神经科学研究所);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);Machine Learning Department, Carnegie Mellon University(卡内基梅隆大学机器学习系);IBM Thomas J. Watson Research Center, IBM Research(IBM沃森研究中心)
Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models
School of Automation, Northwestern Polytechnical University, Xi’an, China(自动化学院,西北工业大学,西安,中国);School of Computing, University of Georgia, Athens, USA(计算机学院,佐治亚大学,亚特兰大,美国);School of Physics and Information Technology, Shaanxi Normal University, Xi’an, China(物理与信息技术学院,陕西师范大学,西安,中国)
Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease
Department of Electrical and Computer Engineering, Clemson University(电子与计算机工程系,克莱姆森大学);Department of Biomedical Engineering, Michigan Technological University(生物医学工程系,密歇根技术大学)
Introducing and Applying Newtonian Blurring: An Augmented Dataset of 126,000 Human Connectomes at braingraph.org
Eötvös University(厄特沃什大学);Uratim Ltd.(乌拉蒂姆有限公司)
高斯模糊是一种成熟的图像数据增强方法:它可以从一小组图片中生成一大组图像,用于人工智能(AI)应用的训练和测试目的。当我们将AI应用于非图像化的生物学数据时,几乎没有任何相关的方法存在。本文介绍了人脑中的“牛顿模糊”现象(或连接体)扩增:从1053名受试者的数据集开始,我们首先重复10次概率加权脑电构建算法来描述不同大脑区域的连接,然后以各种可能的方式重复7次,删除下限和上限,并对每个受试者的数据的剩余7-2=5个边权重求平均。这样,我们将1053个图集增加到120 x 1053 = 126,360个图。在增强技术中,一个重要的要求是不应将人工添加引入数据集。高斯模糊和牛顿模糊满足这个目标。结果数据集包含126,360个图形,每个图形具有5种分辨率(即,总共631,800张图),可在网站https://braingraph.org/cms/download-pit-group-connectomes/上免费获得。利用牛顿模糊的增强也可以应用于其他非图像相关领域,其中实现了概率处理和数据平均。
Transcranial Alternating Current Stimulation (tACS) for patients with Post-Stroke Anomia: Preliminary Data on Picture Naming Performance
Aristotle University of Thessaloniki(萨洛尼卡亚里士多德大学);University of Ioannina(约阿尼纳大学);EuroMov Digital Health in Motion, University Montpellier, IMT Mines Ales(欧洲运动数字健康研究所,蒙彼利埃大学,阿尔勒国立高等矿业学院);Corstim, EuroMov Digital Health in Motion, Montpellier(Corstim,欧洲运动数字健康研究所,蒙彼利埃);University of Peloponnese(伯罗奔尼撒大学)