arXiv 2403.202132024-12-19cs.CV

VHM：用于遥感图像分析的通用且诚实的视觉语言模型

VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis

Wuhan University（武汉大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Sun Yat-Sen University（中山大学）；Sensetime Research（商汤科技研究院）

Chao Pang, Xingxing Weng, Jiang Wu, Jiayu Li, Yi Liu, Jiaxing Sun, Weijia Li, Shuai Wang, Litong Feng, Gui-Song Xia, Conghui He

备注：: Equal contribution: Chao Pang, Xingxing Weng, Jiang Wu; Corresponding author: Gui-Song Xia, Conghui He

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出用于遥感图像分析的通用且诚实的视觉语言模型VHM，通过构建丰富描述数据集VersaD和包含欺骗性问题的指令数据集HnstD，提升模型对遥感图像的理解与诚实性，在多项任务上表现优异。

摘要：

本文开发了一种用于遥感图像分析的通用且诚实的视觉语言模型（VHM）。VHM建立在带有丰富内容描述的大规模遥感图文数据集（VersaD）以及包含事实性和欺骗性问题的诚实指令数据集（HnstD）之上。与现有的遥感图文数据集不同，其图像描述仅关注少数显著物体及其关系，VersaD的描述提供了关于图像属性、物体属性和整体场景的详细信息。这种全面的描述使VHM能够深入理解遥感图像并执行多种遥感任务。此外，与仅包含事实性问题的现有遥感指令数据集不同，HnstD包含因物体不存在而产生的欺骗性问题。这一特性防止了VHM对无意义查询给出肯定回答，从而确保了其诚实性。在我们的实验中，VHM在场景分类、视觉问答和视觉定位等常见任务上显著优于各种视觉语言模型。此外，VHM在建筑矢量化、多标签分类和诚实问答等几个未探索任务上也取得了胜任的性能。我们将在https://github.com/opendatalab/VHM发布代码、数据和模型权重。

英文摘要：

This paper develops a Versatile and Honest vision language Model (VHM) for remote sensing image analysis. VHM is built on a large-scale remote sensing image-text dataset with rich-content captions (VersaD), and an honest instruction dataset comprising both factual and deceptive questions (HnstD). Unlike prevailing remote sensing image-text datasets, in which image captions focus on a few prominent objects and their relationships, VersaD captions provide detailed information about image properties, object attributes, and the overall scene. This comprehensive captioning enables VHM to thoroughly understand remote sensing images and perform diverse remote sensing tasks. Moreover, different from existing remote sensing instruction datasets that only include factual questions, HnstD contains additional deceptive questions stemming from the non-existence of objects. This feature prevents VHM from producing affirmative answers to nonsense queries, thereby ensuring its honesty. In our experiments, VHM significantly outperforms various vision language models on common tasks of scene classification, visual question answering, and visual grounding. Additionally, VHM achieves competent performance on several unexplored tasks, such as building vectorizing, multi-label classification and honest question answering. We will release the code, data and model weights at https://github.com/opendatalab/VHM .

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。