arXiv 2509.256542025-09-30cs.CV

DescribeEarth：描述遥感图像中的一切

DescribeEarth: Describe Anything for Remote Sensing Images

School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；College of Artificial Intelligence, Xi’an Jiaotong University（西安交通大学人工智能学院）；School of Computer Science and Technology and Ministry of Education Key Lab For Intelligent Networks and Network Security, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院和教育部智能网络与网络安全重点实验室）；School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University（西安交通大学数学与统计学院和教育部智能网络与网络安全重点实验室）；Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China（琶洲实验室（黄埔），广州，广东，中国）

Kaiyu Li, Zixuan Jiang, Xiangyong Cao, Jiayu Wang, Yuchen Xiao, Deyu Meng, Zhi Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对遥感图像缺乏对象级细粒度描述的问题，本文提出 Geo-DLC 任务、DE-Dataset、DE-Benchmark 及 DescribeEarth 模型，并验证其优于通用 MLLM。

摘要：

对遥感图像进行自动化文本描述，对于释放其在从环境监测到城市规划和灾害管理等多种应用中的全部潜力至关重要。然而，现有遥感图像描述研究主要集中在图像层面，缺乏对象级细粒度解读，这阻碍了遥感图像所包含的丰富语义和结构信息的充分利用与转化。为解决这一局限，我们提出 Geo-DLC，这是一项面向遥感的对象级细粒度图像描述新任务。为支持该任务，我们构建了 DE-Dataset，这一大规模数据集包含25个类别和261,806个带标注实例，并具有关于对象属性、关系和上下文的详细描述。此外，我们提出 DE-Benchmark，这是一个由 LLM 辅助、基于问答的评估套件，旨在系统衡量模型在 Geo-DLC 任务上的能力。我们还提出 DescribeEarth，这是一种明确为 Geo-DLC 设计的多模态大语言模型（MLLM）架构；该架构集成尺度自适应聚焦策略和领域引导融合模块，利用遥感视觉-语言模型特征来编码高分辨率细节和遥感类别先验，同时保持全局上下文。我们的 DescribeEarth 模型在 DE-Benchmark 上持续优于最先进的通用 MLLM，展现出更优的事实准确性、描述丰富性和语法合理性，尤其是在简单、复杂乃至分布外遥感场景中捕捉对象内在特征和周围环境属性方面。所有数据、代码和权重均发布于 https://github.com/earth-insights/DescribeEarth。

英文摘要：

Automated textual description of remote sensing images is crucial for unlocking their full potential in diverse applications, from environmental monitoring to urban planning and disaster management. However, existing studies in remote sensing image captioning primarily focus on the image level, lacking object-level fine-grained interpretation, which prevents the full utilization and transformation of the rich semantic and structural information contained in remote sensing images. To address this limitation, we propose Geo-DLC, a novel task of object-level fine-grained image captioning for remote sensing. To support this task, we construct DE-Dataset, a large-scale dataset contains 25 categories and 261,806 annotated instances with detailed descriptions of object attributes, relationships, and contexts. Furthermore, we introduce DE-Benchmark, a LLM-assisted question-answering based evaluation suite designed to systematically measure model capabilities on the Geo-DLC task. We also present DescribeEarth, a Multi-modal Large Language Model (MLLM) architecture explicitly designed for Geo-DLC, which integrates a scale-adaptive focal strategy and a domain-guided fusion module leveraging remote sensing vision-language model features to encode high-resolution details and remote sensing category priors while maintaining global context. Our DescribeEarth model consistently outperforms state-of-the-art general MLLMs on DE-Benchmark, demonstrating superior factual accuracy, descriptive richness, and grammatical soundness, particularly in capturing intrinsic object features and surrounding environmental attributes across simple, complex, and even out-of-distribution remote sensing scenarios. All data, code and weights are released at https://github.com/earth-insights/DescribeEarth.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。