arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2308.00541cs.CV
收藏

使用基于RGB的CLIP视觉语言模型检测卫星图像中的云层存在

Detecting Cloud Presence in Satellite Images Using the RGB-based CLIP Vision-Language Model

University of Strathclyde(斯特拉斯克莱德大学)

Mikolaj Czerkawski, Robert Atkinson, Christos Tachtatzis

AI总结:

本研究探索预训练CLIP视觉语言模型在卫星图像云层检测中的能力,提出零样本及多种微调方法并验证跨数据集与传感器的迁移性,证明其表征可用于相关卫星图像处理任务。

摘要:

本研究探索了预训练CLIP视觉语言模型识别受云层影响的卫星图像的能力。论文提出并评估了多种利用该模型执行云层存在检测的方法,包括纯文本提示的零样本操作方案以及多种微调方法。此外,研究还测试了这些方法在不同数据集和传感器类型(Sentinel-2与Landsat-8)之间的可迁移性。结果表明,CLIP在云层存在检测任务上能取得不俗的性能,具备跨传感模态和传感波段的明显泛化能力;研究还发现,低成本的微调阶段可大幅提升真负率。这些结果证明CLIP模型学习到的表征可应用于涉及云层的卫星图像处理任务。

英文摘要:

This work explores capabilities of the pre-trained CLIP vision-language model to identify satellite images affected by clouds. Several approaches to using the model to perform cloud presence detection are proposed and evaluated, including a purely zero-shot operation with text prompts and several fine-tuning approaches. Furthermore, the transferability of the methods across different datasets and sensor types (Sentinel-2 and Landsat-8) is tested. The results that CLIP can achieve non-trivial performance on the cloud presence detection task with apparent capability to generalise across sensing modalities and sensing bands. It is also found that a low-cost fine-tuning stage leads to a strong increase in true negative rate. The results demonstrate that the representations learned by the CLIP model can be useful for satellite image processing tasks involving clouds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑