DetCLIP：面向开放世界检测的词典增强视觉概念训练预训练
DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection
摘要：
开放世界目标检测是一个更普遍、更具有挑战性的目标，其目标是识别和定位由任意类别名称描述的对象。最近的工作GLIP制定了这个问题作为接地问题，通过连接检测数据集的所有类别名称到句子，这导致类别名称之间的交互效率低下。本文提出了DetCLIP，一种基于视觉概念的开放世界检测预训练方法，通过从设计的概念词典中提取知识丰富。为了实现更好的学习效率，我们提出了一种新的可扩展概念公式，该公式分别提取概念以更好地利用异构数据集（即，检测、接地和图像-文本对）用于训练。我们进一步设计了一个概念字典~（与描述）从各种在线来源和检测数据集，为每个概念提供先验知识。通过丰富的概念与他们的描述，我们明确地建立各种概念之间的关系，以促进开放域学习。所提出的概念词典进一步用于为词-区域对齐损失的构造提供足够的否定概念，并为图像-文本对数据的标题中缺少描述的对象完成标签。所提出的框架展示了强的zero-shot检测性能，例如，在LVIS数据集上，我们的DetCLIP-T优于GLIP-T 9.9%mAP，并且与具有与我们相同主干的完全监督模型相比，在罕见类别上获得了13.5%的改进。
英文摘要：
Open-world object detection, as a more general and challenging goal, aims to recognize and localize objects described by arbitrary category names. The recent work GLIP formulates this problem as a grounding problem by concatenating all category names of detection datasets into sentences, which leads to inefficient interaction between category names. This paper presents DetCLIP, a paralleled visual-concept pre-training method for open-world detection by resorting to knowledge enrichment from a designed concept dictionary. To achieve better learning efficiency, we propose a novel paralleled concept formulation that extracts concepts separately to better utilize heterogeneous datasets (i.e., detection, grounding, and image-text pairs) for training. We further design a concept dictionary~(with descriptions) from various online sources and detection datasets to provide prior knowledge for each concept. By enriching the concepts with their descriptions, we explicitly build the relationships among various concepts to facilitate the open-domain learning. The proposed concept dictionary is further used to provide sufficient negative concepts for the construction of the word-region alignment loss\, and to complete labels for objects with missing descriptions in captions of image-text pair data. The proposed framework demonstrates strong zero-shot detection performances, e.g., on the LVIS dataset, our DetCLIP-T outperforms GLIP-T by 9.9% mAP and obtains a 13.5% improvement on rare categories compared to the fully-supervised model with the same backbone as ours.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
