用于视频文本检索的层叠卷积深度编码网络
Stacked Convolutional Deep Encoding Network for Video-Text Retrieval
摘要：
现有的跨模态视频文本检索任务的主要方法是学习一个联合嵌入空间来衡量跨模态的相似性。然而，这些方法很少探索视频帧或文本单词内部的长距离依赖性，导致文本和视觉细节不足。在本文中，我们提出了一种用于视频-文本检索任务的堆栈卷积深度编码网络，该网络考虑同时对视频和文本中的长程和短程依赖性进行编码。具体来说，我们的方法中的多尺度扩张卷积（MSDC）块能够通过采用不同尺度的卷积层的内核大小和扩张大小来编码视频帧或文本单词之间的短程时间线索。一个堆叠的结构被设计成通过重复采用MSDC块来扩展感受野，这进一步捕获了这些线索之间的长程关系。此外，为了获得更鲁棒的文本表示，我们充分利用了强大的语言模型命名为Transformer在两个阶段：预训练短语和微调短语。在两个不同的基准数据集（MSR-VTT，MSVD）上的实验表明，我们提出的方法优于其他最先进的方法。
英文摘要：
Existing dominant approaches for cross-modal video-text retrieval task are to learn a joint embedding space to measure the cross-modal similarity. However, these methods rarely explore long-range dependency inside video frames or textual words leading to insufficient textual and visual details. In this paper, we propose a stacked convolutional deep encoding network for video-text retrieval task, which considers to simultaneously encode long-range and short-range dependency in the videos and texts. Specifically, a multi-scale dilated convolutional (MSDC) block within our approach is able to encode short-range temporal cues between video frames or text words by adopting different scales of kernel size and dilation size of convolutional layer. A stacked structure is designed to expand the receptive fields by repeatedly adopting the MSDC block, which further captures the long-range relations between these cues. Moreover, to obtain more robust textual representations, we fully utilize the powerful language model named Transformer in two stages: pretraining phrase and fine-tuning phrase. Extensive experiments on two different benchmark datasets (MSR-VTT, MSVD) show that our proposed method outperforms other state-of-the-art approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
