arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2004.00436cs.CVcs.LGstat.ML
收藏

基于大词汇量的长尾视觉关系识别研究

Exploring Long Tail Visual Relationship Recognition with Large Vocabulary

King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);IIT Roorkee(印度理工学院鲁尔基分校);Stanford University(斯坦福大学);Baidu(百度);NTU Singapore(新加坡南洋理工大学)

Sherif Abdelkarim, Aniket Agarwal, Panos Achlioptas, Jun Chen, Jiaji Huang, Boyang Li, Kenneth Church, Mohamed Elhoseiny

摘要:

在最近的文献中已经提出了几种方法来缓解长尾问题,主要是在对象分类任务中。本文首次对长尾视觉关系识别(LTVRR)任务进行了大规模的研究。LTVRR旨在改善来自长尾的结构化视觉关系的学习(例如,“兔子吃草”)。在这个设置中,subject、relation和object类都遵循长尾分布。为了开始我们的研究并为社区制定未来的基准,我们引入了两个与LTVRR相关的基准,称为VG 8 K-LT和GQA-LT,它们基于广泛使用的Visual Genome和GQA数据集。我们使用这些基准来研究几个国家的最先进的长尾模型的LTVRR设置的性能。最后,我们提出了一种基于语言的无中心(VilHub)损失和一种适用于LTVRR设置的Mixup增强技术,称为RelMix。VilHub和RelMix都可以很容易地集成在现有模型之上,尽管很简单,但我们的结果表明它们可以显着提高性能,特别是在尾部类上。基准、代码和模型已在https://github.com/Vision-CAIR/LTVRR上提供。

英文摘要:

Several approaches have been proposed in recent literature to alleviate the long-tail problem, mainly in object classification tasks. In this paper, we make the first large-scale study concerning the task of Long-Tail Visual Relationship Recognition (LTVRR). LTVRR aims at improving the learning of structured visual relationships that come from the long-tail (e.g., "rabbit grazing on grass"). In this setup, the subject, relation, and object classes each follow a long-tail distribution. To begin our study and make a future benchmark for the community, we introduce two LTVRR-related benchmarks, dubbed VG8K-LT and GQA-LT, built upon the widely used Visual Genome and GQA datasets. We use these benchmarks to study the performance of several state-of-the-art long-tail models on the LTVRR setup. Lastly, we propose a visiolinguistic hubless (VilHub) loss and a Mixup augmentation technique adapted to LTVRR setup, dubbed as RelMix. Both VilHub and RelMix can be easily integrated on top of existing models and despite being simple, our results show that they can remarkably improve the performance, especially on tail classes. Benchmarks, code, and models have been made available at: https://github.com/Vision-CAIR/LTVRR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑