地理空间基础模型的扩展规律：以PhilEO基准测试为例
Scaling Laws for Geospatial Foundation Models: A case study on PhilEO Bench
AI总结：
本文研究地理空间基础模型的扩展规律，通过不同数据集和架构的实验，发现CNN在低样本下表现优异，ViT-UPerNet在大规模数据上表现最佳，Mamba模型展现出效率优势但需进一步训练。
摘要：
基础模型（FMs）通过利用大规模预训练在多个领域实现了最先进的性能。在地球观测（EO）中，近年来大规模卫星档案的可用性使地理空间基础模型（GFMs）的开发成为可能。然而，关于数据集大小、模型架构和规模如何共同决定下游性能的基本问题仍然存在。在本工作中，我们通过在三个数据集规模上预训练和微调模型系统地探索了这一设计空间：PhilEO Globe（0.5TB）、FastTOM（2TB，此处引入）和MajorTOM（23TB）。我们评估了三种架构家族：Geo-Aware U-Net（CNN）、ViT-UPerNet（Transformer）和Mamba（状态空间模型）；在参数规模从4400万到3亿的范围内。所有模型都在PhilEO基准测试上进行评估，涵盖道路密度和建筑物密度回归、土地覆盖分割，并与现有的GFMs如TerraMind和Prithvi-EO-2.0进行比较。我们的结果表明，在低样本设置中，基于CNN的模型仍然具有高度竞争力，一个2亿参数的Geo-Aware U-Net在回归任务上优于更大的架构。然而，当扩展到多TB数据集时，ViT-UPerNet在MajorTOM（23TB）上的语义分割任务中表现最佳。最后，我们提供了在EO中Mamba模型的首次全面评估，突显了其潜在的效率优势，尽管需要进一步的大规模预训练以完全匹配CNN和ViT。所有代码、预训练模型和FastTOM数据集均已公开发布，以实现可重复性和进一步探索GFMs的扩展规律。
英文摘要：
Foundation Models (FMs) have achieved state-of-the-art performance across domains by leveraging large-scale pretraining. In Earth Observation (EO), the availability of petabyte-scale satellite archives has recently enabled the development of GeoSpatial Foundation Models (GFMs). Yet, fundamental questions remain regarding how dataset size, model architecture, and size interact to determine downstream performance. In this work, we systematically explore this design space by pretraining and fine-tuning models on three dataset scales: PhilEO Globe (0.5TB), FastTOM (2TB, introduced here), and MajorTOM (23TB). We evaluate three architectural families: Geo-Aware U-Net (CNN), ViT-UPerNet (Transformer), and Mamba (State-Space Model); across model sizes ranging from 44M to 300M parameters. All models are benchmarked on the PhilEO Bench, covering: road density and building density regression, and land cover segmentation, and are compared against existing GFMs such as TerraMind and Prithvi-EO-2.0. Our results show that CNN-based models remain highly competitive in low-shot settings, with a 200M-parameter Geo-Aware U-Net outperforming larger architectures on regression tasks. However, when scaling to multi-terabyte datasets, ViT-UPerNet achieves the best performance, particularly for semantic segmentation on MajorTOM (23TB). Finally, we provide the first extensive evaluation of Mamba models in EO, highlighting their potential efficiency advantages, though further large-scale pretraining is required to fully match CNNs and ViTs. All code, pretrained models, and the FastTOM dataset are released publicly, enabling reproducibility and further exploration of scaling laws for GFMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
