arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.09061cs.CVcs.CL
收藏

VLP:视觉-语言预训练综述

VLP: A Survey on Vision-Language Pre-training

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)

Feilong Chen, Duzhen Zhang, Minglun Han, Xiuyi Chen, Jing Shi, Shuang Xu, Bo Xu

AI总结:

本文是首篇聚焦视觉-语言预训练(VLP)的综述,从五大维度梳理VLP进展,总结相关模型并探讨前沿方向,为该领域未来研究提供参考。

摘要:

过去几年,预训练模型的出现将计算机视觉(CV)、自然语言处理(NLP)等单模态领域带入了新纪元。大量研究表明,这类模型对下游单模态任务有益,可避免从零开始训练新模型。那么这类预训练模型能否应用于多模态任务?研究人员已就此问题展开探索并取得显著进展。本文综述了视觉-语言预训练(VLP)的最新进展与前沿方向,涵盖图文预训练和视频-文本预训练。为帮助读者更好地整体把握VLP,我们首先从特征提取、模型架构、预训练目标、预训练数据集、下游任务五个方面梳理其最新进展,随后详细总结了具体的VLP模型,最后探讨了VLP的新前沿方向。据我们所知,这是首篇聚焦VLP的综述,我们希望本综述能为VLP领域的未来研究提供启发。

英文摘要:

In the past few years, the emergence of pre-training models has brought uni-modal fields such as computer vision (CV) and natural language processing (NLP) to a new era. Substantial works have shown they are beneficial for downstream uni-modal tasks and avoid training a new model from scratch. So can such pre-trained models be applied to multi-modal tasks? Researchers have explored this problem and made significant progress. This paper surveys recent advances and new frontiers in vision-language pre-training (VLP), including image-text and video-text pre-training. To give readers a better overall grasp of VLP, we first review its recent advances from five aspects: feature extraction, model architecture, pre-training objectives, pre-training datasets, and downstream tasks. Then, we summarize the specific VLP models in detail. Finally, we discuss the new frontiers in VLP. To the best of our knowledge, this is the first survey focused on VLP. We hope that this survey can shed light on future research in the VLP field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑