LHRS-Bot：利用VGI增强的大型多模态语言模型赋能遥感
LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model
AI总结：
本文提出LHRS-Bot，一种基于VGI增强数据集和课程学习的多模态大模型，用于遥感图像理解与推理，并通过LHRS-Bench基准验证其有效性。
摘要：
大型语言模型（LLM）的革命性能力为多模态大型语言模型（MLLM）铺平了道路，并促进了各专业领域中的多样化应用。然而，在遥感（RS）领域，近期MLLM的努力并未充分考虑RS图像中多样的地理景观和变化的目标。为弥补这一差距，我们利用广泛的志愿地理信息（VGI）和全球可用的RS图像，构建了一个大规模的RS图像-文本数据集LHRS-Align，以及一个信息丰富的RS专用指令数据集LHRS-Instruct。在此基础上，我们引入了LHRS-Bot，这是一个通过新颖的多层级视觉-语言对齐策略和课程学习方法专门为RS图像理解定制的MLLM。此外，我们引入了LHRS-Bench，一个用于全面评估MLLM在RS图像理解能力的基准。综合实验表明，LHRS-Bot对RS图像展现出深刻的理解，并具备在RS领域内进行细致推理的能力。
英文摘要：
The revolutionary capabilities of large language models (LLMs) have paved the way for multimodal large language models (MLLMs) and fostered diverse applications across various specialized domains. In the remote sensing (RS) field, however, the diverse geographical landscapes and varied objects in RS imagery are not adequately considered in recent MLLM endeavors. To bridge this gap, we construct a large-scale RS image-text dataset, LHRS-Align, and an informative RS-specific instruction dataset, LHRS-Instruct, leveraging the extensive volunteered geographic information (VGI) and globally available RS images. Building on this foundation, we introduce LHRS-Bot, an MLLM tailored for RS image understanding through a novel multi-level vision-language alignment strategy and a curriculum learning method. Additionally, we introduce LHRS-Bench, a benchmark for thoroughly evaluating MLLMs' abilities in RS image understanding. Comprehensive experiments demonstrate that LHRS-Bot exhibits a profound understanding of RS images and the ability to perform nuanced reasoning within the RS domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
