GeoRSMLLM：一种用于地球科学与遥感中视觉-语言任务的多模态大语言模型
GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing
AI总结：
本文提出 GeoRSMLLM，通过分层遥感视觉-语言任务集、点集统一表示、条件解析器与循环指代自增强，提升模型处理复杂指令及像素级遥感任务的通用能力。
摘要：
视觉-语言模型（VLM）在遥感（RS）中的应用，已在场景分类、目标检测和图像描述等传统任务中展现出巨大潜力。然而，当前在指代表达理解（REC）方面表现优异的模型，在处理涉及复杂指令（例如存在多个条件）或分割、变化检测等像素级操作的任务时仍面临困难。在本白皮书中，我们按照所需认知能力的不同层级，对遥感中的视觉-语言任务进行了全面的分层总结。我们介绍了遥感视觉-语言任务集（RSVLTS），其中包括难度递增的开放词汇任务（OVT）、指代表达任务（RET）和描述目标任务（DOT），以及视觉问答（VQA）。此外，我们针对 RSVLTS 提出了一种采用点集方法的新型统一数据表示，并配套提出条件解析器和基于循环指代的自增强策略。这些特性被集成到 GeoRSMLLM 模型中；该增强模型旨在处理 RSVLTS 的广泛任务，为形成更通用的地球科学与遥感视觉-语言任务解决方案铺平道路。
英文摘要：
The application of Vision-Language Models (VLMs) in remote sensing (RS) has demonstrated significant potential in traditional tasks such as scene classification, object detection, and image captioning. However, current models, which excel in Referring Expression Comprehension (REC), struggle with tasks involving complex instructions (e.g., exists multiple conditions) or pixel-level operations like segmentation and change detection. In this white paper, we provide a comprehensive hierarchical summary of vision-language tasks in RS, categorized by the varying levels of cognitive capability required. We introduce the Remote Sensing Vision-Language Task Set (RSVLTS), which includes Open-Vocabulary Tasks (OVT), Referring Expression Tasks (RET), and Described Object Tasks (DOT) with increased difficulty, and Visual Question Answering (VQA) aloneside. Moreover, we propose a novel unified data representation using a set-of-points approach for RSVLTS, along with a condition parser and a self-augmentation strategy based on cyclic referring. These features are integrated into the GeoRSMLLM model, and this enhanced model is designed to handle a broad range of tasks of RSVLTS, paving the way for a more generalized solution for vision-language tasks in geoscience and remote sensing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
