arXiv 2502.095982026-01-21cs.CV

GAIA: 一个全球性的多模态、多尺度遥感图像分析数据集

GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis

Orion Lab, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens（奥里昂实验室，农村测绘与地理信息工程学院，希腊雅典国家技术大学）；Institute of Astronomy, Astrophysics, Space Applications and Remote Sensing, National Observatory of Athens（天文、天体物理、空间应用与遥感研究所，希腊雅典国家天文台）；Department of Informatics and Telematics, Harokopio University of Athens（信息与电信技术系，雅典霍罗科皮欧大学）；Chair of Data Science in Earth Observation, Technical University of Munich（地球观测数据科学教授职位，慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）

Angelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis

备注：: 26 pages, 14 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

GAIA是一个全球性的多模态遥感图像分析数据集，通过精心构建的图像-文本对提升遥感任务的性能。

摘要：

现有的视觉-语言模型（VLMs）主要是在网络爬取的噪声图像-文本数据上进行训练，表现出在遥感（RS）领域专业知识方面的局限性。这种不足导致在RS特定任务上的性能较差，因为常用的数据库通常缺乏详细的、科学准确的文本描述，而只强调日期和位置等属性。为填补这一关键差距，我们引入了GAIA，一个专为多尺度、多传感器和多模态RS图像分析设计的新数据集。GAIA包含201,005个精心挑选的RS图像-文本对，代表了不同空间分辨率的RS模态。与现有的遥感视觉-语言数据集不同，GAIA特别关注捕捉广泛的RS应用，提供了关于环境变化、自然灾害以及各种动态现象的独特信息。该数据集提供了空间和时间平衡的分布，覆盖全球，时间跨度达25年，具有平衡的时间分布观测。GAIA的构建涉及两个阶段的过程：（1）有针对性地从可靠的RS相关来源爬取图像及其配套文本，（2）为每张图像生成五个高质量的、科学基础的合成描述，使用精心设计的提示来利用GPT-4o先进的视觉-语言能力。我们的广泛实验，包括CLIP和BLIP2模型的微调，证明GAIA在RS图像分类、跨模态检索和图像描述任务上显著提高了性能。我们公开了数据集、自动化处理框架和微调模型权重，可在我们的项目GitHub仓库上找到：https://github.com/Orion-AI-Lab/GAIA。

英文摘要：

Existing Vision-Language Models (VLMs) are predominantly trained on web-scraped, noisy image-text data, exhibiting limited exposure to the specialized domain of RS. This deficiency results in poor performance on RS-specific tasks, as commonly used datasets often lack detailed, scientifically accurate textual descriptions and instead emphasize solely on attributes like date and location. To bridge this critical gap, we introduce GAIA, a novel dataset designed for multi-scale, multi-sensor, and multi-modal RS image analysis. GAIA comprises of 201,005 meticulously curated RS image-text pairs, representing a diverse range of RS modalities associated to different spatial resolutions. Unlike existing vision-language datasets in RS, GAIA specifically focuses on capturing a diverse range of RS applications, providing unique information about environmental changes, natural disasters, and various other dynamic phenomena. The dataset provides a spatially and temporally balanced distribution, spanning across the globe, covering the last 25 years with a balanced temporal distribution of observations. GAIA's construction involved a two-stage process: (1) targeted web-scraping of images and accompanying text from reputable RS-related sources, and (2) generation of five high-quality, scientifically grounded synthetic captions for each image using carefully crafted prompts that leverage the advanced vision-language capabilities of GPT-4o. Our extensive experiments, including fine-tuning of CLIP and BLIP2 models, demonstrate that GAIA significantly improves performance on RS image classification, cross-modal retrieval and image captioning tasks. We make our dataset, automated processing framework and fine-tuned model weights publicly available on our project's GitHub repository: https://github.com/Orion-AI-Lab/GAIA.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。