arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.01034cs.CVeess.IVq-bio.NC
收藏

学习聚合要素制图表达

Learning to aggregate feature representations

The Weizmann Institute of Science(魏茨曼科学研究所)

Guy Gaziv

摘要:

Algonauts挑战需要构建一个多学科编码器的图像大脑活动。众所周知,为图像分类训练的ResNet-50和AlexNet等深度网络会沿着其中间阶段产生特征表示,这些特征表示密切模仿视觉层次结构。然而,在Algonauts项目中引入的挑战,包括组合来自多个主题的数据,依赖于非常少的相似性数据点,解决各种ROI和多模态,需要设计一个灵活的框架,可以有效地适应它们。在这里,我们建立在最近的最先进的分类网络(SE-ResNeXt-50),并构建其中间表示的自适应组合。虽然预训练的网络作为我们模型的骨干,但我们学习如何沿着网络的五个阶段聚合特征表示。在学习过程中,我们的方法能够根据优化目标调节和筛选网络中每个阶段的输出。我们将我们的方法应用于Algonauts2019 fMRI和MEG挑战。结合使用功能磁共振成像和脑磁图数据,我们的方法被评为这两个挑战的前五名。令人惊讶的是,我们发现,对于较低和较高的顺序区域(EVC和IT)的自适应聚合有利于功能在网络的后期阶段。

英文摘要:

The Algonauts challenge requires to construct a multi-subject encoder of images to brain activity. Deep networks such as ResNet-50 and AlexNet trained for image classification are known to produce feature representations along their intermediate stages which closely mimic the visual hierarchy. However the challenges introduced in the Algonauts project, including combining data from multiple subjects, relying on very few similarity data points, solving for various ROIs, and multi-modality, require devising a flexible framework which can efficiently accommodate them. Here we build upon a recent state-of-the-art classification network (SE-ResNeXt-50) and construct an adaptive combination of its intermediate representations. While the pretrained network serves as a backbone of our model, we learn how to aggregate feature representations along five stages of the network. During learning, our method enables to modulate and screen outputs from each stage along the network as governed by the optimized objective. We applied our method to the Algonauts2019 fMRI and MEG challenges. Using the combined fMRI and MEG data, our approach was rated among the leading five for both challenges. Surprisingly we find that for both the lower and higher order areas (EVC and IT) the adaptive aggregation favors features stemming at later stages of the network.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑