arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.11005cs.CV
收藏

用于开放词汇目标检测的循环对比知识转移

Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection

The Hong Kong University of Science and Technology(香港科技大学);AI Chip Center for Emerging Smart Systems (ACCESS)(新兴智能系统AI芯片中心)

Chuhan Zhang, Chaoyang Zhu, Pingcheng Dong, Long Chen, Dong Zhang

AI总结:

本文提出 CCKT-Det,通过循环动态知识转移和区域对比损失,在无需额外监督下实现开放词汇目标检测,在 COCO 基准上显著提升了性能。

摘要:

为了检测超出预定义类别的无限目标,开放词汇目标检测(OVD)的现有技术通常借助预训练的视觉语言模型(VLMs)来实现从基础类别到新类别的泛化。然而,为了缓解上游图像-文本预训练与下游区域级感知之间的错位,额外的监督是不可或缺的,例如图像-文本对或通过自训练策略生成的伪标注。在本研究中,我们提出了在无需任何额外监督下训练的 CCKT-Det。所提出的框架构建了从语言查询和从 VLMs 提取的视觉区域特征之间的循环动态知识转移,迫使检测器与 VLMs 的视觉语义空间紧密对齐。具体而言,1)我们预过滤并注入语义先验以指导查询的学习,2)引入区域对比损失以提高查询对新目标的感知能力。随着 VLMs 规模的增加,CCKT-Det 能持续提升性能,同时仅需检测器维持适度的计算开销。综合实验结果表明,在具有挑战性的 COCO 基准上,无论有无更强的教师模型,我们的方法相比之前的最先进方法分别实现了 +2.9% 和 +10.2% AP50 的性能提升。

英文摘要:

In pursuit of detecting unstinted objects that extend beyond predefined categories, prior arts of open-vocabulary object detection (OVD) typically resort to pretrained vision-language models (VLMs) for base-to-novel category generalization. However, to mitigate the misalignment between upstream image-text pretraining and downstream region-level perception, additional supervisions are indispensable, eg, image-text pairs or pseudo annotations generated via self-training strategies. In this work, we propose CCKT-Det trained without any extra supervision. The proposed framework constructs a cyclic and dynamic knowledge transfer from language queries and visual region features extracted from VLMs, which forces the detector to closely align with the visual-semantic space of VLMs. Specifically, 1) we prefilter and inject semantic priors to guide the learning of queries, and 2) introduce a regional contrastive loss to improve the awareness of queries on novel objects. CCKT-Det can consistently improve performance as the scale of VLMs increases, all while requiring the detector at a moderate level of computation overhead. Comprehensive experimental results demonstrate that our method achieves performance gain of +2.9% and +10.2% AP50 over previous state-of-the-arts on the challenging COCO benchmark, both without and with a stronger teacher model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑