Cryo-Bench:面向冰冻圈应用的基础模型基准测试
Cryo-Bench: Benchmarking Foundation Models for Cryosphere Mapping
AI总结:
提出Cryo-Bench基准,评估14个地理基础模型在冰冻圈关键组件(如冰川、冰湖、海冰等)上的性能,发现UNet在冻结编码器下平均mIoU最高(66.38),而全微调结合学习率调整可提升性能12.77%。
摘要:
地理基础模型(GFMs)已在涵盖多个领域的地球观测任务中得到评估,并展现出即使在标签稀疏的情况下也能生成可靠地图的强大潜力。然而,针对冰冻圈应用的GFMs基准测试仍然有限,主要原因是缺乏合适的评估数据集。为填补这一空白,我们引入了 extbf{Cryo-Bench},这是一个用于评估GFMs在关键冰冻圈组件上性能的基准。Cryo-Bench包括覆盖冰川、冰湖、海冰和崩解前沿,涉及多种传感器和广泛的地理区域。我们评估了14个GFMs以及UNet和ViT基线,以分析它们的优势、局限性和最佳使用策略。在冻结编码器的情况下,UNet在Cryo-Bench包含的五个评估数据集上取得了最高的平均mIoU extbf{66.38},其次是TerraMind的 extbf{64.02}。在少样本设置(10%输入数据)下,DOFA和TerraMind等GFMs优于UNet,分别达到 extbf{59.53}、 extbf{56.62}和 extbf{56.60}的mIoU分数,而U-Net为56.60。当完全微调GFMs时,我们观察到不同数据集和模型之间的性能不一致。然而,调整学习率并配合微调显著提升了GFM性能。例如,在两个代表性数据集(GLID和CaFFe)上的评估显示平均相对提升为 extbf{12.77\%}。尽管预训练数据中冰冻圈表示极少,GFMs仍展现出显著的领域适应能力,并在各项任务中产生有意义的结果。基于我们的发现,我们建议通过超参数优化进行编码器微调以获得最佳性能,而在用户需要快速结果且无需大量实验时使用冻结编码器。(\href{https://github.com/Sk-2103/Cryo-Bench}{GitHub})
英文摘要:
Geo-Foundation Models (GFMs) have been evaluated across diverse Earth observation tasks and domains, showing strong potential to produce reliable maps even with sparse labels. However, systematic benchmarking of GFMs for Cryosphere applications remains limited, primarily because suitable evaluation datasets are scarce. We address this gap by introducing Cryo-Bench, a benchmark comprising six semantic segmentation datasets covering five cryospheric components: supraglacial debris, glacial lakes under two sensing configurations, sea ice, calving fronts and Antarctic ice-shelf extent. The benchmark includes multispectral, RGB, and synthetic aperture radar observations from regions underrepresented in existing pretraining archives. We evaluate thirteen GFMs alongside U-Net and Vision Transformer baselines trained from scratch under a unified evaluation protocol. With frozen encoders, the U-Net achieves the highest six-dataset average mean intersection over union (mIoU) of 69.31\%, exceeding TerraMind (67.86\%) by 1.45 points. The paired difference has a 95\% confidence interval of [+0.94, +1.98], indicating that the U-Net's lead is statistically significant. In contrast, learning-rate optimization substantially improves fine-tuning performance: DOFA reaches 93.97\% mIoU on the RGB glacial lake task, ranking the U-Net fourth, while Scale-MAE and GFM-Swin surpass U-Net on calving fronts. Averaged across all six datasets, four GFMs, TerraMind, GFM-Swin, DOFA, and Scale-MAE, exceed the U-Net baseline (69.31\%). In the few-shot setting, five GFMs, DOFA, RemoteCLIP, TerraMind, GFM-Swin, and Scale-MAE, likewise outperform U-Net; averaged across all thirteen GFMs, retention is 92.5\% of full-label accuracy compared with 86.1\% for U-Net.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。