arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.09310cs.CVcs.LG
收藏

贫数据域中的深度多模态迁移学习回归

Deep Multimodal Transfer-Learned Regression in Data-Poor Domains

Texas A&M University(德克萨斯农工大学);Army Research Lab(陆军研究实验室)

Levi McClenny, Mulugeta Haile, Vahid Attari, Brian Sadler, Ulisses Braga-Neto, Raymundo Arroyave

摘要:

在深度学习的许多现实应用中,目标的估计可能依赖于各种类型的输入数据模式,例如音频-视频、图像-文本等。在这里,我们提出了一种深度多模态转移学习回归器(DMTL-R),用于在深度回归架构中对图像和特征数据进行多模态学习,有效地预测数据贫乏领域的目标参数。我们的模型能够在少量训练图像数据上微调给定的一组预训练CNN权重,同时在网络训练期间对来自互补数据模式的特征信息进行调节,从而产生比单独使用图像或特征更准确的单目标或多目标回归。我们使用相场模拟微观结构图像与一组附带的物理特征,使用来自各种知名的CNN架构的预训练权重,展示了所提出的多模态方法的有效性。

英文摘要:

In many real-world applications of deep learning, estimation of a target may rely on various types of input data modes, such as audio-video, image-text, etc. This task can be further complicated by a lack of sufficient data. Here we propose a Deep Multimodal Transfer-Learned Regressor (DMTL-R) for multimodal learning of image and feature data in a deep regression architecture effective at predicting target parameters in data-poor domains. Our model is capable of fine-tuning a given set of pre-trained CNN weights on a small amount of training image data, while simultaneously conditioning on feature information from a complimentary data mode during network training, yielding more accurate single-target or multi-target regression than can be achieved using the images or the features alone. We present results using phase-field simulation microstructure images with an accompanying set of physical features, using pre-trained weights from various well-known CNN architectures, which demonstrate the efficacy of the proposed multimodal approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑