arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.19552cs.CV
收藏

GeoLLaVA:用于遥感时间变化检测的高效微调视觉-语言模型

GeoLLaVA: Efficient Fine-Tuned Vision-Language Models for Temporal Change Detection in Remote Sensing

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(莫扎德·本·扎耶德人工智能大学)

Hosam Elgendy, Ahmed Sharshar, Ahmed Aboeitta, Yasser Ashraf, Mohsen Guizani

AI总结:

GeoLLaVA通过引入视频帧对数据集和LoRA、QLoRA及剪枝等微调技术,显著提升视觉-语言模型在遥感时间变化检测中的性能,最佳BERT分数达0.864。

摘要:

检测地理景观中的时间变化对于环境监测和城市规划等应用至关重要。尽管遥感数据丰富,但现有的视觉-语言模型(VLMs)往往无法有效捕捉时间动态。本文通过引入一个带标注的视频帧对数据集来追踪随时间演变的地理模式,以解决这些局限性。利用低秩适配(LoRA)、量化LoRA(QLoRA)和模型剪枝等微调技术,应用于Video-LLaVA和LLaVA-NeXT-Video等模型,我们显著提升了视觉-语言模型在处理遥感时间变化方面的性能。结果表明,性能显著提升,最佳性能达到BERT分数0.864和ROUGE-1分数0.576,展示了在描述土地利用变化方面的卓越准确性。

英文摘要:

Detecting temporal changes in geographical landscapes is critical for applications like environmental monitoring and urban planning. While remote sensing data is abundant, existing vision-language models (VLMs) often fail to capture temporal dynamics effectively. This paper addresses these limitations by introducing an annotated dataset of video frame pairs to track evolving geographical patterns over time. Using fine-tuning techniques like Low-Rank Adaptation (LoRA), quantized LoRA (QLoRA), and model pruning on models such as Video-LLaVA and LLaVA-NeXT-Video, we significantly enhance VLM performance in processing remote sensing temporal changes. Results show significant improvements, with the best performance achieving a BERT score of 0.864 and ROUGE-1 score of 0.576, demonstrating superior accuracy in describing land-use transformations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑