arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.01656cs.CVcs.LG
收藏

S4:跨频谱的自监督感知

S4: Self-Supervised Sensing Across the Spectrum

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Jayanth Shenoy, Xingjian Davis Zhang, Shlok Mehrotra, Bill Tao, Rem Yang, Han Zhao, Deepak Vasisht

AI总结:

本文提出自监督预训练方法S4,利用卫星图像跨频谱捕获和地理配准特性设计预训练任务,并构建大规模多模态数据集m2s2-SITS,在有限标注数据下显著提升SITS分割性能。

摘要:

卫星图像时间序列(SITS)分割对于环境监测、土地覆盖制图和农作物类型分类等许多应用至关重要。然而,由于缺乏丰富的训练数据且需要细粒度标注,训练SITS分割模型仍然是一项具有挑战性的任务。我们提出了S4,一种新的自监督预训练方法,通过利用两个新见解显著降低了对标注训练数据的需求:(a)卫星在不同频谱部分捕获图像,例如无线电频率和可见光频率;(b)卫星图像经过地理配准,允许进行细粒度的空间对齐。我们利用这些见解在S4中制定预训练任务。我们还构建了m2s2-SITS,一个大规模的无标注、空间对齐、多模态且具有地理特定性的SITS数据集,作为S4的代表性预训练数据。最后,我们在多个SITS分割数据集上评估S4,并证明其在仅使用有限标注数据的情况下相对于竞争基线的有效性。

英文摘要:

Satellite image time series (SITS) segmentation is crucial for many applications like environmental monitoring, land cover mapping and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine grained annotation. We propose S4 a new self-supervised pre-training approach that significantly reduces the requirement for labeled training data by utilizing two new insights: (a) Satellites capture images in different parts of the spectrum such as radio frequencies, and visible frequencies. (b) Satellite imagery is geo-registered allowing for fine-grained spatial alignment. We use these insights to formulate pre-training tasks in S4. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially-aligned, multi-modal and geographic specific SITS that serves as representative pre-training data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑