arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.09110cs.CV

定位地球上的任何事物:推进遥感社区的开放词汇目标检测

Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community

Tsinghua University(清华大学)

Jiancheng Pan, Yanxing Liu, Yuqian Fu, Muyuan Ma, Jiahao Li, Danda Pani Paudel, Luc Van Gool, Xiaomeng Huang

AI总结:

针对现有开放词汇检测器难以泛化到遥感图像的问题,提出LAE任务,构建大规模遥感数据集LAE-1M,并训练首个开放词汇基础目标检测器LAE-DINO,实验证明其有效。

摘要:

目标检测,特别是开放词汇目标检测,在环境监测、自然灾害评估和土地利用规划等地球科学中发挥着至关重要的作用。然而,现有的开放词汇检测器主要在自然世界图像上训练,由于显著的数据域差距,难以泛化到遥感图像。因此,本文旨在推进遥感社区开放词汇目标检测的发展。为此,我们首先将该任务重新定义为“定位地球上的任何事物”(LAE),目标是检测地球上的任何新概念。然后我们开发了 LAE-Label Engine,它收集、自动标注并统一了多达10个遥感数据集,创建了 LAE-1M——首个具有广泛类别覆盖的大规模遥感目标检测数据集。利用 LAE-1M,我们进一步提出并训练了新颖的 LAE-DINO 模型,这是用于 LAE 任务的首个开放词汇基础目标检测器,具备动态词汇构建(DVC)和视觉引导文本提示学习(VisGT)模块。DVC 为每个训练批次动态构建词汇,而 VisGT 将视觉特征映射到语义空间,增强文本特征。我们在既定的遥感基准 DIOR、DOTAv2.0 以及我们新引入的80类 LAE-80C 基准上进行了全面实验。结果证明了 LAE-1M 数据集的优势和 LAE-DINO 方法的有效性。

英文摘要:

Object detection, particularly open-vocabulary object detection, plays a crucial role in Earth sciences, such as environmental monitoring, natural disaster assessment, and land-use planning. However, existing open-vocabulary detectors, primarily trained on natural-world images, struggle to generalize to remote sensing images due to a significant data domain gap. Thus, this paper aims to advance the development of open-vocabulary object detection in remote sensing community. To achieve this, we first reformulate the task as Locate Anything on Earth (LAE) with the goal of detecting any novel concepts on Earth. We then developed the LAE-Label Engine which collects, auto-annotates, and unifies up to 10 remote sensing datasets creating the LAE-1M - the first large-scale remote sensing object detection dataset with broad category coverage. Using the LAE-1M, we further propose and train the novel LAE-DINO Model, the first open-vocabulary foundation object detector for the LAE task, featuring Dynamic Vocabulary Construction (DVC) and Visual-Guided Text Prompt Learning (VisGT) modules. DVC dynamically constructs vocabulary for each training batch, while VisGT maps visual features to semantic space, enhancing text features. We comprehensively conduct experiments on established remote sensing benchmark DIOR, DOTAv2.0, as well as our newly introduced 80-class LAE-80C benchmark. Results demonstrate the advantages of the LAE-1M dataset and the effectiveness of the LAE-DINO method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑