VL-DINO: 利用CLIP视觉-语言知识进行开放词汇目标检测
VL-DINO: Leveraging CLIP Vision-Language Knowledge for Open-Vocabulary Object Detectio
AI总结：
提出VL-DINO，通过QPSC模块构建高质量正样本增强视觉-语言对齐，VSE模块蒸馏CLIP视觉知识，ORSA模块对齐区域特征与文本嵌入，在LVIS零样本检测上达到36.3/38.1 AP。
摘要：
像CLIP这样的视觉-语言模型可以为开放词汇目标检测提供丰富的语义先验。然而，将文本和视觉知识联合集成到检测架构中仍然具有挑战性。在本文中，我们提出了VL-DINO，一种通过更有效地利用CLIP的视觉-语言知识来增强DINO的开放词汇检测器。具体来说，首先开发了一个查询引导的正样本构建（QPSC）模块，以构建额外的高质量正样本，使原始DINO框架能够更好地适应跨异构数据源的混合训练，同时提供更多的视觉-语言对齐信号，从而在训练过程中融入更丰富的文本知识。然后引入了一个视觉语义编码器（VSE）模块，将CLIP视觉知识蒸馏到骨干网络提取的特征中，生成用于后续编码器精炼的融合特征。基于融合特征，一个目标-区域语义对齐（ORSA）模块提取以目标为中心的区域特征，并将其与相应的文本嵌入对齐，进一步融入文本线索。在零样本设置下，VL-DINO-T和VL-DINO-L在LVIS基准上分别达到了36.3和38.1 AP，持续优于先前的高级方法。大量实验证明了所提出设计的有效性和竞争性能。
英文摘要：
Vision-language models like CLIP can provide rich semantic priors for open-vocabulary object detection. However, jointly integrating both textual and visual knowledge into detection architectures remains challenging. In this paper, we propose VL-DINO, an open-vocabulary detector that enhances DINO through more effective exploitation of CLIP's vision-language knowledge. Specifically, a Query-guided Positive Sample Construction (QPSC) module is first developed to construct additional high-quality positive samples, enabling the vanilla DINO framework to better accommodate mixed training across heterogeneous data sources while providing more vision-language alignment signals, thereby incorporating richer textual knowledge during training. A Visual Semantic Encoder (VSE) module is then introduced to distill CLIP visual knowledge into backbone-extracted features, producing fused features for subsequent encoder refinement. Based on the fused features, an Object-Region Semantic Alignment (ORSA) module extracts object-centric region features and aligns them with the corresponding textual embeddings, further incorporating textual cues. In the zero-shot setting, VL-DINO-T and VL-DINO-L achieve 36.3 and 38.1 AP on the LVIS benchmark, respectively, consistently outperforming prior advanced approaches. Extensive experiments demonstrate the effectiveness and competitive performance of the proposed design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
