arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.12246cs.CV
收藏

RT-OVAD:基于图文协作的实时开放词汇航空目标检测

RT-OVAD: Real-Time Open-Vocabulary Aerial Object Detection via Image-Text Collaboration

Nanjing University of Science and Technology(南京理工大学);Northwestern Polytechnical University(西北工业大学);Zhejiang Lab(浙江实验室);Intellifusion Inc.(Intellifusion公司)

Guoting Wei, Xia Yuan, Yu Liu, Zhenhao Shang, Xizhe Xue, Peng Wang, Kelu Yao, Chunxia Zhao, Haokui Zhang, Rong Xiao

AI总结:

本文提出首个航空场景实时开放词汇检测器RT-OVAD,通过图文协作策略与图像到文本对齐损失消除类别约束,在多项检测任务中超越现有方法并实现34 FPS实时推理。

摘要:

航空目标检测在众多应用中发挥着至关重要的作用。然而,现有大多数方法专注于检测预定义的目标类别,限制了其在现实开放场景中的适用性。本文通过图文协作将航空目标检测扩展至开放场景,并提出了RT-OVAD,这是首个面向航空场景的实时开放词汇检测器。具体而言,我们首先引入图像到文本的对齐损失来替代传统的类别回归损失,从而消除类别约束。接着,我们提出了一种轻量级的图文协作策略,包含一个图文协作编码器和一个文本引导的解码器。编码器同时增强视觉特征并细化文本嵌入,而解码器则引导目标查询关注与类别相关的图像特征。该设计在不产生显著计算开销的情况下进一步提升了检测精度。大量实验表明,RT-OVAD在开放词汇、零样本和传统闭集检测任务中均持续优于现有的最先进方法。例如,在开放词汇航空检测基准DIOR、DOTA-v2.0和LAE-80C上,RT-OVAD分别取得了87.7 AP$_{50}$、53.8 mAP和23.7 mAP,超越了此前最先进方法(LAE-DINO)2.2、7.0和3.5个百分点。此外,RT-OVAD在RTX 4090 GPU上的推理速度达到34 FPS,比LAE-DINO(10 FPS)快约三倍,满足了多样化应用的实时检测需求。代码将发布于https://github.com/GT-Wei/RT-OVAD。

英文摘要:

Aerial object detection plays a crucial role in numerous applications. However, most existing methods focus on detecting predefined object categories, limiting their applicability in real-world open scenarios. In this paper, we extend aerial object detection to open scenarios through image-text collaboration and propose RT-OVAD, the first real-time open-vocabulary detector for aerial scenes. Specifically, we first introduce an image-to-text alignment loss to replace the conventional category regression loss, thereby eliminating category constraints. Next, we propose a lightweight image-text collaboration strategy comprising an image-text collaboration encoder and a text-guided decoder. The encoder simultaneously enhances visual features and refines textual embeddings, while the decoder guides object queries to focus on class-relevant image features. This design further improves detection accuracy without incurring significant computational overhead. Extensive experiments demonstrate that RT-OVAD consistently outperforms existing state-of-the-art methods across open-vocabulary, zero-shot, and traditional closed-set detection tasks. For instance, on the open-vocabulary aerial detection benchmarks DIOR, DOTA-v2.0, and LAE-80C, RT-OVAD achieves 87.7 AP$_{50}$, 53.8 mAP, and 23.7 mAP, respectively, surpassing the previous state-of-the-art (LAE-DINO) by 2.2, 7.0, and 3.5 points. In addition, RT-OVAD achieves an inference speed of 34 FPS on an RTX 4090 GPU, approximately three times faster than LAE-DINO (10 FPS), meeting the real-time detection requirements of diverse applications. The code will be released at https://github.com/GT-Wei/RT-OVAD.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑