arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.09799cs.CVcs.AI
收藏

CP-DETR:概念提示引导的DETR面向更强的通用目标检测

CP-DETR: Concept Prompt Guide DETR Toward Stronger Universal Object Detection

China Mobile(Zhejiang) Research & Innovation Institute(中国移动(浙江)研究院)

Qibo Chen, Weizhong Jin, Jianyue Ge, Mengdi Liu, Yuchao Yan, Jian Jiang, Li Yu, Xuanjiang Guo, Shuchang Li, Jianzhong Chen

AI总结:

针对通用目标检测中提示先验利用低效、下游对齐偏差大的问题,提出带提示-视觉混合编码器与多类概念提示生成方法的CP-DETR模型,仅单套预训练权重即在多数据集多场景下取得领先的通用检测性能。

摘要:

当前通用目标检测研究旨在将语言引入现有最优(SoTA)闭集检测器,再通过构建大规模(文本-区域)数据集进行训练,以实现对开集概念的泛化。然而这类方法面临两大核心挑战:一是如何高效利用提示中的先验信息实现目标泛化,二是如何降低下游任务中的对齐偏差,这两个问题都会导致预训练外的部分场景下性能未达最优。\n为解决上述挑战,我们提出了一种高性能通用检测基础模型CP-DETR,仅需一套预训练权重即可在几乎所有场景中具备竞争力。具体而言,我们设计了高效的提示-视觉混合编码器,通过逐尺度与多尺度融合模块增强提示与视觉信息的交互;同时结合提示多标签损失与辅助检测头,助力混合编码器充分利用提示信息。除文本提示外,我们还设计了视觉提示、优化提示两种实用的概念提示生成方法,前者通过具体视觉示例提取抽象概念,后者可稳定降低下游任务的对齐偏差。\n凭借这些有效设计,CP-DETR在广泛场景中展现出优异的通用检测性能:例如,采用Swin-T骨干网络的模型在LVIS数据集上取得47.6的零样本AP,Swin-L骨干模型在ODinW35数据集上取得32.2的零样本AP;此外,我们的视觉提示生成方法通过交互式检测在COCO验证集上达到68.4 AP,优化提示在ODinW13数据集上取得73.1的全样本AP。

英文摘要:

Recent research on universal object detection aims to introduce language in a SoTA closed-set detector and then generalize the open-set concepts by constructing large-scale (text-region) datasets for training. However, these methods face two main challenges: (i) how to efficiently use the prior information in the prompts to genericise objects and (ii) how to reduce alignment bias in the downstream tasks, both leading to sub-optimal performance in some scenarios beyond pre-training. To address these challenges, we propose a strong universal detection foundation model called CP-DETR, which is competitive in almost all scenarios, with only one pre-training weight. Specifically, we design an efficient prompt visual hybrid encoder that enhances the information interaction between prompt and visual through scale-by-scale and multi-scale fusion modules. Then, the hybrid encoder is facilitated to fully utilize the prompted information by prompt multi-label loss and auxiliary detection head. In addition to text prompts, we have designed two practical concept prompt generation methods, visual prompt and optimized prompt, to extract abstract concepts through concrete visual examples and stably reduce alignment bias in downstream tasks. With these effective designs, CP-DETR demonstrates superior universal detection performance in a broad spectrum of scenarios. For example, our Swin-T backbone model achieves 47.6 zero-shot AP on LVIS, and the Swin-L backbone model achieves 32.2 zero-shot AP on ODinW35. Furthermore, our visual prompt generation method achieves 68.4 AP on COCO val by interactive detection, and the optimized prompt achieves 73.1 fully-shot AP on ODinW13.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑