本文提出了直接学习的图像分类从用户提供的标签,没有过滤。每个标签都是由在线共享图像的用户提供的。大量的这些标签可以在网上免费获得,它们提供了对用户和图像分类重要的图像类别的见解。我们的方法是对传统的手工注释方法的补充,这种方法成本极高。我们分析了Flickr 100 Million Image数据集,对这些标签的统计数据进行了一些有用的观察。我们引入了一个大规模的强大的分类算法,以处理这些标签中的固有噪声,以及一个校准过程,以更好地预测客观的注释。我们表明,免费提供的,用户提供的标签可以获得类似或优越的结果,昂贵的手动注释的大型数据库。
英文摘要:
This paper proposes direct learning of image classification from user-supplied tags, without filtering. Each tag is supplied by the user who shared the image online. Enormous numbers of these tags are freely available online, and they give insight about the image categories important to users and to image classification. Our approach is complementary to the conventional approach of manual annotation, which is extremely costly. We analyze of the Flickr 100 Million Image dataset, making several useful observations about the statistics of these tags. We introduce a large-scale robust classification algorithm, in order to handle the inherent noise in these tags, and a calibration procedure to better predict objective annotations. We show that freely available, user-supplied tags can obtain similar or superior results to large databases of costly manual annotations.
Dual-level Semantic Transfer Deep Hashing for Efficient Social Image Retrieval
Shandong Normal University(山东师范大学);Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东省计算中心(国家超级计算济南中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));University of Electronic Science and Technology of China(电子科技大学);Harbin Institute of Technology(哈尔滨工业大学)
A novel image tag completion method based on convolutional neural network
Soochow University(苏州大学);Huawei Technologies Co., Ltd.(华为技术有限公司);Suning Commerce R&D Center USA, Inc.(苏宁美国研发中心);Travelers(Travelers公司);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Jiangsu University of Technology(江苏理工学院);Intermediate People’s Court of Linyi City(临沂市中级人民法院);Hebei University of Economics and Business(河北经贸大学);Savitribai Phule Pune University(萨维特里巴伊·普勒浦那大学)
Weakly Supervised Deep Hyperspherical Quantization for Image Retrieval
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);University College London(伦敦大学学院);University of Cambridge(剑桥大学)
Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality
The University of Texas at Austin(德克萨斯大学奥斯汀分校)
最近的视觉语言学预训练模型在各种最终任务上显示出有希望的进展,例如图像检索和视频描述。然而,他们在最近提出的Winoground数据集上失败得很惨,该数据集挑战模型匹配配对图像和英文字幕,其中项目被构造为在词汇上重叠但含义不同(例如,“There is a mug in some grass”与“There is some grass in a mug”的对比)。通过使用新的细粒度标签注释数据集,我们表明解决Winoground任务不仅需要组合语言理解,而且还需要许多其他能力,如常识推理或在低分辨率图像中定位小的,焦点外的对象。在本文中,我们通过一系列相关任务(探测任务,图像检索任务),数据增强和手动检查数据集的实验来确定数据集的主要挑战。我们的分析表明,视觉语言学模型的一个主要挑战可能在于融合视觉和文本表征,而不是成分语言理解。我们在https://github.com/ajd12342/why-winoground-hard上发布我们的注释和代码。
A Multi-View Embedding Space for Modeling Internet Images, Tags, and their Semantics
University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Microsoft Research Silicon Valley(微软研究院硅谷分院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags
Ahsanullah University of Science & Technology(阿萨努拉科技大学);University of Alberta(阿尔伯塔大学);Université de Sherbrooke(谢布鲁克大学);Salesforce Research(Salesforce研究院);Nanyang Technological University(南洋理工大学)
UrbanCross: Enhancing Satellite Image-Text Retrieval with Cross-Domain Adaptation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Northwestern Polytechnical University(西北工业大学);The Hong Kong University of Science and Technology(香港科技大学)
Image2song: Song Retrieval via Bridging Image Content and Lyric Words
School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University(西北工业大学计算机学院暨光学影像分析与学习中心)