arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.11749cs.CV
收藏

在开放世界中检测一切:迈向通用目标检测

Detecting Everything in the Open World: Towards Universal Object Detection

Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology (BNRist)(北京信息科学与技术国家研究中心(BNRist));The University of Hong Kong(香港大学);Meta AI;Massachusetts Institute of Technology(麻省理工学院)

Zhenyu Wang, Yali Li, Xi Chen, Ser-Nam Lim, Antonio Torralba, Hengshuang Zhao, Shengjin Wang

摘要:

在本文中,我们正式解决通用的目标检测,其目的是检测每一个场景和预测每一个类别。传统检测器对人类注释的依赖、有限的视觉信息以及开放世界中的新类别严重限制了其通用性。我们提出了UniDetector,一个通用的对象检测器,有能力在开放的世界中识别巨大的类别。UniDetector通用性的关键点是:1)它利用多源图像和异构标签空间进行训练,通过对齐图像和文本空间,这保证了通用表示的足够信息。2)由于视觉和语言模态提供了丰富的信息,它很容易推广到开放世界,同时保持了可见和不可见类之间的平衡。3)通过我们提出的解耦训练方式和概率校准,进一步提高了对新类别的泛化能力。这些贡献使UniDetector能够检测超过7 k个类别,这是迄今为止最大的可测量类别大小,只有大约500个类参与训练。我们的UniDetector在大词汇量数据集(如LVIS,ImageNetBoxes和VisualGenome)上表现出强大的zero-shot泛化能力-它平均超过传统的监督基线4%以上,而看不到任何对应的图像。在13个不同场景的公共检测数据集上,UniDetector仅用3%的训练数据就实现了最先进的性能。

英文摘要:

In this paper, we formally address universal object detection, which aims to detect every scene and predict every category. The dependence on human annotations, the limited visual information, and the novel categories in the open world severely restrict the universality of traditional detectors. We propose UniDetector, a universal object detector that has the ability to recognize enormous categories in the open world. The critical points for the universality of UniDetector are: 1) it leverages images of multiple sources and heterogeneous label spaces for training through the alignment of image and text spaces, which guarantees sufficient information for universal representations. 2) it generalizes to the open world easily while keeping the balance between seen and unseen classes, thanks to abundant information from both vision and language modalities. 3) it further promotes the generalization ability to novel categories through our proposed decoupling training manner and probability calibration. These contributions allow UniDetector to detect over 7k categories, the largest measurable category size so far, with only about 500 classes participating in training. Our UniDetector behaves the strong zero-shot generalization ability on large-vocabulary datasets like LVIS, ImageNetBoxes, and VisualGenome - it surpasses the traditional supervised baselines by more than 4\% on average without seeing any corresponding images. On 13 public detection datasets with various scenes, UniDetector also achieves state-of-the-art performance with only a 3\% amount of training data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑