arXiv 2411.119042025-05-10cs.CV

GeoGround：用于遥感视觉定位的统一大型视觉-语言模型

GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding

Nanyang Technological University（南洋理工大学）；University of Reading（阅读大学）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）；SenseTime Research（商汤科技研究院）

Yue Zhou, Mengcheng Lan, Xiang Li, Litong Feng, Yiping Ke, Xue Jiang, Qingyun Li, Xue Yang, Wayne Zhang

备注：: 9 pages, 5 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出GeoGround框架，通过Text-Mask技术及提示辅助、几何引导学习，使大型视觉-语言模型统一支持HBB、OBB和掩码三类遥感视觉定位任务，并在多个基准上取得与专用方法相当的性能。

摘要：

遥感（RS）视觉定位旨在利用自然语言表达在RS图像中定位特定目标（以边界框或分割掩码的形式），从而增强人类与智能RS解译系统的交互。该领域的早期研究主要基于水平边界框（HBB），但随着更多样化的RS数据集出现，涉及旋转边界框（OBB）和分割掩码的任务也随之产生。在实际应用中，不同目标需要不同的定位类型：HBB可以定位目标位置，OBB提供其朝向，而掩码描绘其形状。然而，现有专用方法通常是为单一类型的RS视觉定位任务定制的，难以跨任务泛化。相比之下，大型视觉-语言模型（VLM）展现出强大的多任务学习能力，但难以处理分割这类密集预测任务。本文提出GeoGround，这是一个统一支持HBB、OBB和掩码RS视觉定位任务的新颖框架，允许灵活选择输出。我们的工作并非定制VLM架构，而是旨在通过Text-Mask技术优雅地支持像素级视觉定位输出。我们定义了提示辅助和几何引导学习，以增强不同信号之间的一致性。实验结果表明，GeoGround在四项RS视觉定位任务上均表现出强劲性能，并在多个基准上达到与专用方法相当的性能。代码可在https://github.com/zytx121/GeoGround获取。

英文摘要：

Remote sensing (RS) visual grounding aims to use natural language expression to locate specific objects (in the form of the bounding box or segmentation mask) in RS images, enhancing human interaction with intelligent RS interpretation systems. Early research in this area was primarily based on horizontal bounding boxes (HBBs), but as more diverse RS datasets have become available, tasks involving oriented bounding boxes (OBBs) and segmentation masks have emerged. In practical applications, different targets require different grounding types: HBB can localize an object's position, OBB provides its orientation, and mask depicts its shape. However, existing specialized methods are typically tailored to a single type of RS visual grounding task and are hard to generalize across tasks. In contrast, large vision-language models (VLMs) exhibit powerful multi-task learning capabilities but struggle to handle dense prediction tasks like segmentation. This paper proposes GeoGround, a novel framework that unifies support for HBB, OBB, and mask RS visual grounding tasks, allowing flexible output selection. Rather than customizing the architecture of VLM, our work aims to elegantly support pixel-level visual grounding output through the Text-Mask technique. We define prompt-assisted and geometry-guided learning to enhance consistency across different signals. Experimental results show that GeoGround demonstrates strong performance across four RS visual grounding tasks, matching the performance of specialized methods on multiple benchmarks. Code available at https://github.com/zytx121/GeoGround

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。