arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1612.04061cs.CVcs.CL
收藏

学习使用Tag2Vec为视频添加哈希标签

Learning to Hash-tag Videos with Tag2Vec

CVIT, KCIS, IIIT Hyderabad(印度海得拉巴国际信息技术研究所视觉信息技术中心与科利智能系统中心)

Aditya Singh, Saurabh Saini, Rajvi Shah, PJ Narayanan

摘要:

用户给定的标签或标签是用于视觉媒体(例如图像和视频)的语义理解的有价值的资源。最近,一种被称为哈希标签的新型标签机制在社交媒体网站上越来越受欢迎。在本文中,我们研究的问题,生成相关的和有用的哈希标签的短视频剪辑。传统的数据驱动的标签丰富和推荐方法使用直接的视觉相似性进行标签传输和传播。我们尝试使用两步训练过程来学习从视频到哈希标签的直接低成本映射。我们首先采用自然语言处理(NLP)技术,跳过语法模型与神经网络训练学习低维向量表示的哈希标签(Tag2Vec)使用语料库的1000万哈希标签。然后,我们训练一个嵌入函数来将视频特征映射到低维的Tag2vec空间。我们学习了29类带有哈希标签的短视频片段的嵌入。然后,可以使用学习的嵌入将没有任何标签信息的查询视频直接映射到标签的向量空间,并且可以通过在Tag2Vec空间中执行简单的最近邻检索来找到相关标签。我们验证了我们的系统所建议的标签的相关性,定性和定量的用户研究。

英文摘要:

User-given tags or labels are valuable resources for semantic understanding of visual media such as images and videos. Recently, a new type of labeling mechanism known as hash-tags have become increasingly popular on social media sites. In this paper, we study the problem of generating relevant and useful hash-tags for short video clips. Traditional data-driven approaches for tag enrichment and recommendation use direct visual similarity for label transfer and propagation. We attempt to learn a direct low-cost mapping from video to hash-tags using a two step training process. We first employ a natural language processing (NLP) technique, skip-gram models with neural network training to learn a low-dimensional vector representation of hash-tags (Tag2Vec) using a corpus of 10 million hash-tags. We then train an embedding function to map video features to the low-dimensional Tag2vec space. We learn this embedding for 29 categories of short video clips with hash-tags. A query video without any tag-information can then be directly mapped to the vector space of tags using the learned embedding and relevant tags can be found by performing a simple nearest-neighbor retrieval in the Tag2Vec space. We validate the relevance of the tags suggested by our system qualitatively and quantitatively with a user study.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑