arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.04944cs.CVcs.LG
收藏

ViTs for SITS:用于卫星图像时间序列的视觉Transformer

ViTs for SITS: Vision Transformers for Satellite Image Time Series

Imperial College London(帝国理工学院)

Michail Tarasiou, Erik Chavez, Stefanos Zafeiriou

AI总结:

本文提出TSViT模型,通过先时间后空间的分解编码和采集时间位置编码等机制,在卫星图像时间序列处理中取得最先进性能。

摘要:

本文提出了时-空视觉Transformer(TSViT),这是一种基于视觉Transformer(ViT)的、用于通用卫星图像时间序列(SITS)处理的全注意力模型。TSViT将SITS记录在空间和时间上分割成非重叠的图像块,这些图像块被标记化后由分解的时-空编码器处理。我们认为,与自然图像相比,先时间后空间的分解方式对SITS处理更直观,并给出了支持这一观点的实验证据。此外,我们通过引入两种新机制增强了模型的判别能力:一种是针对采集时间的时间位置编码,另一种是多个可学习的类别标记。所有新设计选择的效果通过广泛的消融研究进行了评估。我们提出的架构在三个公开可用的SITS语义分割和分类数据集上取得了最先进的性能,显著超越了先前的方法。所有模型、训练和评估代码均已公开,以促进进一步研究。

英文摘要:

In this paper we introduce the Temporo-Spatial Vision Transformer (TSViT), a fully-attentional model for general Satellite Image Time Series (SITS) processing based on the Vision Transformer (ViT). TSViT splits a SITS record into non-overlapping patches in space and time which are tokenized and subsequently processed by a factorized temporo-spatial encoder. We argue, that in contrast to natural images, a temporal-then-spatial factorization is more intuitive for SITS processing and present experimental evidence for this claim. Additionally, we enhance the model's discriminative power by introducing two novel mechanisms for acquisition-time-specific temporal positional encodings and multiple learnable class tokens. The effect of all novel design choices is evaluated through an extensive ablation study. Our proposed architecture achieves state-of-the-art performance, surpassing previous approaches by a significant margin in three publicly available SITS semantic segmentation and classification datasets. All model, training and evaluation codes are made publicly available to facilitate further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑