面向开放词汇目标检测的视图袋采样
Sampling Bag of Views for Open-Vocabulary Object Detection
AI总结：
提出基于概念对齐的视图袋采样方法，通过分组上下文相关概念并调整尺度优化嵌入对齐，在COCO和LVIS上提升新类别检测性能并减少80.3%的CLIP计算量。
摘要：
现有的开放词汇目标检测（OVD）通过将目标区域嵌入与相应的VLM特征对齐，来开发测试未见类别的方法。最近的一项研究利用了VLM在图像内隐式学习语义概念组合结构的思路。它不使用单个区域嵌入，而是利用一组区域嵌入作为新表示，将组合结构纳入OVD任务。然而，这种方法往往无法捕获每个区域的上下文概念，导致组合结构存在噪声。这导致性能提升微乎其微且效率降低。为了解决这个问题，我们提出了一种新颖的基于概念的对齐方法，该方法采样更强大且高效的组合结构。我们的方法将上下文相关的“概念”分组到一个袋中，并调整袋内概念的尺度以实现更有效的嵌入对齐。结合Faster R-CNN，我们的方法在开放词汇COCO和LVIS基准的新类别上，比先前工作分别提升了2.6 box AP50和0.5 mask AP。此外，与先前研究相比，我们的方法将FLOPs中的CLIP计算量减少了80.3%，显著提升了效率。实验结果表明，所提出的方法在OVD数据集上优于先前的最先进模型。
英文摘要：
Existing open-vocabulary object detection (OVD) develops methods for testing unseen categories by aligning object region embeddings with corresponding VLM features. A recent study leverages the idea that VLMs implicitly learn compositional structures of semantic concepts within the image. Instead of using an individual region embedding, it utilizes a bag of region embeddings as a new representation to incorporate compositional structures into the OVD task. However, this approach often fails to capture the contextual concepts of each region, leading to noisy compositional structures. This results in only marginal performance improvements and reduced efficiency. To address this, we propose a novel concept-based alignment method that samples a more powerful and efficient compositional structure. Our approach groups contextually related ``concepts'' into a bag and adjusts the scale of concepts within the bag for more effective embedding alignment. Combined with Faster R-CNN, our method achieves improvements of 2.6 box AP50 and 0.5 mask AP over prior work on novel categories in the open-vocabulary COCO and LVIS benchmarks. Furthermore, our method reduces CLIP computation in FLOPs by 80.3% compared to previous research, significantly enhancing efficiency. Experimental results demonstrate that the proposed method outperforms previous state-of-the-art models on the OVD datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
