arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.16073cs.CVcs.RO
收藏

OW-Rep: 开放世界目标检测中的实例表示学习

OW-Rep: Open World Object Detection with Instance Representation Learning

KAIST(韩国科学技术院);Carnegie Mellon University(卡内基梅隆大学);Agency for Defense Development(国防发展局)

Sunoh Lee, Minsik Jeon, Jihong Min, Junwon Seo

AI总结:

OW-Rep通过引入实例表示学习,提升开放世界目标检测中未知目标的检测精度和语义嵌入质量,增强下游任务表现。

摘要:

开放世界目标检测(OWOD)旨在解决现实场景中出现未知目标类别的问题,使训练于已知类别的检测器能够检测未知目标并逐步整合其提供的知识。尽管现有OWOD方法主要聚焦于检测未知目标,但往往忽略了检测目标之间的丰富语义关系,这对于场景理解和开放世界环境中的应用(如开放世界跟踪和新类发现)至关重要。在本文中,我们扩展OWOD框架,联合检测未知目标并学习语义丰富的实例嵌入,使检测器能够捕捉实例之间的细粒度语义关系。为此,我们提出了两个模块,利用Vision Foundation Models(VFMs)的丰富且可泛化知识,并可集成到开放世界目标检测器中。首先,未知框细化模块利用Segment Anything Model的实例掩码来准确定位未知目标。然后,嵌入转移模块通过放松对比损失将实例级语义相似性从VFM特征蒸馏到检测器的嵌入中,使检测器能够学习具有语义意义且可泛化的实例特征。大量实验表明,我们的方法显著提高了未知目标检测和实例嵌入质量,同时在下游任务如开放世界跟踪中也提升了性能。

英文摘要:

Open World Object Detection(OWOD) addresses realistic scenarios where unseen object classes emerge, enabling detectors trained on known classes to detect unknown objects and incrementally incorporate the knowledge they provide. While existing OWOD methods primarily focus on detecting unknown objects, they often overlook the rich semantic relationships between detected objects, which are essential for scene understanding and applications in open-world environments (e.g., open-world tracking and novel class discovery). In this paper, we extend the OWOD framework to jointly detect unknown objects and learn semantically rich instance embeddings, enabling the detector to capture fine-grained semantic relationships between instances. To this end, we propose two modules that leverage the rich and generalizable knowledge of Vision Foundation Models(VFMs) and can be integrated into open-world object detectors. First, the Unknown Box Refine Module uses instance masks from the Segment Anything Model to accurately localize unknown objects. The Embedding Transfer Module then distills instance-wise semantic similarities from VFM features to the detector's embeddings via a relaxed contrastive loss, enabling the detector to learn a semantically meaningful and generalizable instance feature. Extensive experiments show that our method significantly improves both unknown object detection and instance embedding quality, while also enhancing performance in downstream tasks such as open-world tracking.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑