arXiv 2603.296302026-04-01cs.CV

BigEarthNet.txt: 一个大规模多传感器图像-文本数据集和地球观测基准

BigEarthNet.txt: A Large-Scale Multi-Sensor Image-Text Dataset and Benchmark for Earth Observation

BIFOLD（柏林智能数据与机器学习研究所）；Technische Universität Berlin（柏林工业大学）；University of Trento（特伦托大学）；National Technical University（雅典国家技术大学）；National Observatory of Athens（雅典国家天文台）

Johann-Ludwig Herzog, Mathis Jürgen Adler, Leonard Hackel, Yan Shu, Angelos Zavras, Ioannis Papoutsis, Paolo Rota, Begüm Demir

备注：: For details, see https://txt.bigearth.net

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出BigEarthNet.txt，一个大规模多传感器图像-文本数据集，用于提升地球观测中的指令驱动图像-文本学习，包含9.6M文本注释，涵盖土地利用/覆盖类、空间关系及环境背景描述，通过对比分析证明其在文本丰富性和注释类型多样性上的优势。

摘要：

视觉-语言模型（VLMs）在计算机视觉（CV）中表现出色，但在遥感（RS）数据上的性能受限于缺乏大规模、多传感器的RS图像-文本数据集，这些数据集通常只包含航空红绿蓝图像，具有简短或弱关联的描述，注释类型多样性有限。为解决这一问题，我们引入BigEarthNet.txt，一个大规模、多传感器图像-文本数据集，旨在推动地球观测中的指令驱动图像-文本学习。BigEarthNet.txt包含464044幅共注册的Sentinel-1合成孔径雷达和Sentinel-2多谱段图像，配有9.6M文本注释，包括：i) 地理定位描述，涵盖土地利用/覆盖（LULC）类别、空间关系及环境背景；ii) 与不同任务相关的视觉问答对；iii) 用于边界框预测的指称表达检测指令。通过比较统计分析，我们证明BigEarthNet.txt在文本丰富性和注释类型多样性上优于现有RS图像-文本数据集。我们进一步建立了手动验证的基准分割来评估VLMs在RS和CV中的表现。结果表明，这些模型在涉及复杂LULC类别的任务中存在局限性，而使用BigEarthNet.txt微调可使所有考虑的任务性能获得一致提升。

英文摘要：

Vision-langugage models (VLMs) have shown strong performance in computer vision (CV), yet their performance on remote sensing (RS) data remains limited due to the lack of large-scale, multi-sensor RS image-text datasets with diverse textual annotations. Existing datasets predominantly include aerial Red-Green-Blue imagery, with short or weakly grounded captions, and provide limited diversity in annotation types. To address this limitation, we introduce BigEarthNet$.$txt, a large-scale, multi-sensor image-text dataset designed to advance instruction-driven image-text learning in Earth observation across multiple tasks. BigEarthNet$.$txt contains 464044 co-registered Sentinel-1 synthetic aperture radar and Sentinel-2 multispectral images with 9.6M text annotations, including: i) geographically anchored captions describing land-use/land-cover (LULC) classes, their spatial relations, and environmental context; ii) visual question answering pairs relevant for different tasks; and iii) referring expression detection instructions for bounding box prediction. Through a comparative statistical analysis, we demonstrate that BigEarthNet$.$txt surpasses existing RS image-text datasets in textual richness and annotation type variety. We further establish a manually-verified benchmark split to evaluate VLMs in RS and CV. The results show the limitations of these models on tasks that involve complex LULC classes, whereas fine-tuning using BigEarthNet$.$txt results in consistent performance gains across all considered tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。