远离人群：基于地理隔离的可扩展自监督学习
Far from the Crowd: Scalable Self-Supervised Learning via Geographic Isolation
- AIKO
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究提出基于地理隔离的无标签代理指标，将其集成到MoCoV2和MAE预训练中，在CopernicusBench的三个下游任务上验证，可大幅降低预计算成本并提升下游性能。
AI中文摘要：
针对遥感影像的自监督预训练通常将所有样本视为同等信息，而忽略了地理与视觉结构的巨大差异。我们提出一种面向地球观测的自监督课程学习策略，该策略通过地理隔离对样本进行排序，地理隔离是一种完全源自地理空间数据集已有地理定位元数据的无标签代理指标，无需图像解码、模型反馈或人工标注。与视觉复杂度代理不同，该指标随数据集规模D的扩展复杂度为O(D log D)，且对对比式和重建式目标均适用。我们将该指标集成到MoCoV2和MAE预训练中，并在CopernicusBench的三个下游任务（BigEarthNet、DFC-2020、LCZ）上进行评估。该课程策略在仅使用20%训练预算（MAE）、最多40%训练预算（MoCo）时即可达到基线最终epoch性能，在BigEarthNet上将最终下游性能提升了最多5个平均精度均值（mAP），在各基准上的提升幅度为1-5个百分点，与视觉复杂度课程的效果相当，同时将预计算成本降低了140倍以上（在SSL4EO上为4秒对568秒）。CKA与有效秩分析进一步表明，经该课程训练的编码器在整个训练过程中形成了更高维度、利用更均匀的嵌入空间。
英文摘要：
Self-supervised pretraining on remote sensing imagery typically treats all samples as equally informative, despite large variability in geographic and visual structure. We propose a curriculum learning strategy for self-supervised Earth observation that ranks samples by geographic isolation, a label-free proxy derived entirely from geolocation metadata already present in geospatial datasets, requiring no image decoding, no model feedback, and no manual annotation. Unlike visual complexity proxies, it scales as O(D log D) with dataset size D and is well-defined for both contrastive and reconstructive objectives. We integrate the proposed measure into MoCoV2 and MAE pretraining and evaluate across three downstream tasks from CopernicusBench (BigEarthNet, DFC-2020, LCZ). Our curriculum reaches baseline final-epoch performance using as few as 20% of the training budget (MAE) and at most 40% (MoCo) of the training budget, and improves final downstream performance by up to +5 mAP on BigEarthNet, with gains of 1-5 points across benchmarks, matching visual-complexity curricula while reducing pre-computation cost by more than 140x (4 s vs. 568 s on SSL4EO). A CKA and effective-rank analysis further reveals that curriculum-trained encoders develop higher-dimensional, more uniformly utilized embedding spaces throughout training.
