地球观测(EO)卫星捕获了大量未标记数据,其中 Sentinel-2 星座每天生成 1.6 TB 的数据。这使得遥感成为一个数据丰富的领域,非常适合机器学习(ML)解决方案。然而,将 ML 模型应用于 EO 的一个瓶颈是缺乏标注数据,因为标注是一个劳动密集且成本高昂的过程。因此,该领域的研究一直聚焦于自监督学习和基础模型方法。本文通过引入 PhilEO Bench(一个用于 EO 基础模型的新型评估框架),解决了在不同基础模型之间进行公平且统一基准评估的需求。该框架包含一个测试平台和一个新颖的 400 GB Sentinel-2 数据集,该数据集包含三个下游任务的标签:建筑物密度估计、道路分割和土地覆盖分类。我们展示了使用该框架在不同 n-shot 和收敛速率下评估不同基础模型(包括 Prithvi 和 SatMAE)的实验。
英文摘要:
Massive amounts of unlabelled data are captured by Earth Observation (EO) satellites, with the Sentinel-2 constellation generating 1.6 TB of data daily. This makes Remote Sensing a data-rich domain well suited to Machine Learning (ML) solutions. However, a bottleneck in applying ML models to EO is the lack of annotated data as annotation is a labour-intensive and costly process. As a result, research in this domain has focused on Self-Supervised Learning and Foundation Model approaches. This paper addresses the need to evaluate different Foundation Models on a fair and uniform benchmark by introducing the PhilEO Bench, a novel evaluation framework for EO Foundation Models. The framework comprises of a testbed and a novel 400 GB Sentinel-2 dataset containing labels for three downstream tasks, building density estimation, road segmentation, and land cover classification. We present experiments using our framework evaluating different Foundation Models, including Prithvi and SatMAE, at multiple n-shots and convergence rates.
GEO-Bench: Toward Foundation Models for Earth Monitoring
ServiceNow Research(ServiceNow研究院);Technical University of Munich(慕尼黑工业大学);Stanford University(斯坦福大学);Arizona State University(亚利桑那州立大学);MIT(麻省理工学院);ETH Zurich(苏黎世联邦理工学院);Clark University(克拉克大学);Mila-Quebec(魁北克人工智能研究所);University of Montreal(蒙特利尔大学)
GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models
Tsinghua University(清华大学);SenseTime Research(商汤科技研究院);Peking University(北京大学);Technical University of Munich, Heilbronn Data Science Center, Munich Data Science Institute(慕尼黑技术大学,海德堡数据科学中心,慕尼黑数据科学研究所)
ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning
School of Intelligent Science and Technology, Nanjing University(南京大学智能科学与技术学院);School of Computer Science, Peking University(北京大学计算机科学学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models
University of Texas at Austin(德克萨斯大学奥斯汀分校);Brown University(布朗大学);University of Central Florida(中佛罗里达大学);NVIDIA Research(英伟达研究院);Stanford University(斯坦福大学)
GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI
IBM Research Europe(IBM欧洲研究院);Technical University Munich(慕尼黑技术大学);Clark University(克拉克大学);MBZUAI;NASA Impact(NASA影响计划);ESA Φ \Phi -lab(欧洲航天局Φ实验室);Arizona State University(亚利桑那州立大学);ServiceNow AI Research(ServiceNow人工智能研究)
HyBiomass: Global Hyperspectral Imagery Benchmark Dataset for Evaluating Geospatial Foundation Models in Forest Aboveground Biomass Estimation
Remote Sensing Technology Institute, German Aerospace Center (DLR)(远程传感技术研究所,德国航天中心);Global Land Monitoring Group, Helmholtz German Research Centre for Geosciences (GFZ)(全球土地监测组,海德堡德国地理科学研究中心);Department of Aerospace Engineering, University of the Bundeswehr Munich(航空航天工程系,联邦国防军 Munich 大学);Chair of Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM)(地球观测数据科学教授职位(SiPEO),慕尼黑技术大学)
PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models
ESA Φ Φ \Phi roman_Φ -lab(欧洲航天局(ESA)Φ实验室);Geomatics Section, Department of Civil Engineering, Faculty of Engineering Technology(工程科技学院土木工程系);KTH - Royal Institute of Technology(皇家理工学院);Conservatoire national des arts et métiers, CEDRIC(国家艺术与工艺学院,CEDRIC);University of Glasgow(格拉斯哥大学);Univ. Gustave Eiffel, ENSG, IGN, LASTIG(根特大学,IGN,LASTIG)
Cross-view Geo-localization via Learning Disentangled Geometric Layout Correspondence
University of Vermont(佛蒙特大学);Vermont Complex Systems Center, University of Vermont(佛蒙特大学佛蒙特复杂系统中心);Shanghai Center for Brain Science and Brain-Inspired Technology(上海脑科学与类脑研究中心);Information Technology University(信息技术大学);University of Science and Technology of China(中国科学技术大学)
ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models
Beijing University of Posts and Telecommunications(北京邮电大学);State Key Laboratory of Networking and Switching Technology(网络与交换技术国家重点实验室);School of Materials Science and Engineering(材料科学与工程学院);China Mobile Research Institute(中国移动研究院);College of Computing and Data Science(计算与数据科学学院)
TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning
University of Georgia(佐治亚大学);UC Santa Barbara(加利福尼亚大学圣巴巴拉分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);UC Davis(加利福尼亚大学戴维斯分校);Basis Independent Fremont(贝西斯独立弗里蒙特分校);Google LLC(谷歌公司);Stanford University(斯坦福大学);Microsoft Research(微软研究院)