arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1705.09142cs.CV
收藏

使用字幕生成器的深度图像表示

Deep image representations using caption generators

Indian Institute of Science(印度科学学院)

Konda Reddy Mopuri, Vishal B. Athreya, R. Venkatesh Babu

摘要:

深度学习利用大量标记数据来学习强大的模型。当目标数据集很小时,通常的做法是使用预训练的模型来执行迁移学习,以学习新的任务特定表示。然而,用于图像识别的预先训练的CNN在训练期间提供了关于图像的有限信息,这是单独的标签。场景检索等任务会受到从这种弱监督中学习到的特征的影响,需要更强的监督才能更好地理解图像的内容。在本文中,我们利用从字幕生成模型中学习到的特征来学习新的特定于任务的图像表示。特别地,我们考虑了最先进的字幕系统Show and Tell~ \cite{SnT-pami-2016}和密集区域描述模型DenseCap~ \cite{densecap-cvpr-2016}。我们证明,由于在训练过程中提供了更丰富的监督,字幕系统学习到的特征比CNN的表现更好。此外,我们训练一个siamese网络与一个修改的成对损失融合的功能学习的~ \cite{SnT-pami-2016}和~ \cite{densecap-cvpr-2016}和学习图像表示适合检索。实验表明,该融合利用了个体特征的互补性,并在基准数据集上产生了最先进的检索结果。

英文摘要:

Deep learning exploits large volumes of labeled data to learn powerful models. When the target dataset is small, it is a common practice to perform transfer learning using pre-trained models to learn new task specific representations. However, pre-trained CNNs for image recognition are provided with limited information about the image during training, which is label alone. Tasks such as scene retrieval suffer from features learned from this weak supervision and require stronger supervision to better understand the contents of the image. In this paper, we exploit the features learned from caption generating models to learn novel task specific image representations. In particular, we consider the state-of-the art captioning system Show and Tell~\cite{SnT-pami-2016} and the dense region description model DenseCap~\cite{densecap-cvpr-2016}. We demonstrate that, owing to richer supervision provided during the process of training, the features learned by the captioning system perform better than those of CNNs. Further, we train a siamese network with a modified pair-wise loss to fuse the features learned by~\cite{SnT-pami-2016} and~\cite{densecap-cvpr-2016} and learn image representations suitable for retrieval. Experiments show that the proposed fusion exploits the complementary nature of the individual features and yields state-of-the art retrieval results on benchmark datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑