一个以视觉为中心的遥感基准
A Vision Centric Remote Sensing Benchmark
AI总结：
本文针对基于 CLIP 的多模态大语言模型在遥感视觉定位与空间推理上的不足，提出 RSMMVP 基准并通过 VQA 评估识别 CLIP 盲对，揭示其遥感表征学习局限。
摘要：
多模态大语言模型（MLLMs）在视觉-语言任务中取得了显著成功，但其在遥感（RS）领域的对应应用仍相对缺乏探索。与自然图像不同，遥感图像带来了当前 MLLMs 难以应对的独特挑战，尤其是在视觉定位和空间推理方面。本研究考察了基于 CLIP 的 MLLMs 在遥感中的局限性，强调它们无法区分视觉上不同但语义上相似的遥感图像。为解决这一问题，我们提出了一个遥感多模态视觉模式（RSMMVP）基准。该基准旨在通过识别 CLIP-blind pairs（CLIP 盲对）来评估 MLLMs 在遥感任务中的表现；在这些样本对中，基于 CLIP 的模型会错误地给视觉上不同的遥感图像赋予较高相似度分数。通过视觉问答（VQA）评估，我们分析了最先进 MLLMs 的性能，揭示了遥感专用表征学习中存在的显著局限。结果为理解基于 CLIP 的视觉编码弱点提供了有价值的洞见，并为未来开发更有效、面向遥感应用定制的 MLLMs 奠定了基础。
英文摘要：
Multimodal Large Language Models (MLLMs) have achieved remarkable success in vision-language tasks but their remote sensing (RS) counterpart are relatively under explored. Unlike natural images, RS imagery presents unique challenges that current MLLMs struggle to handle, particularly in visual grounding and spatial reasoning. This study investigates the limitations of CLIP-based MLLMs in RS, highlighting their failure to differentiate visually distinct yet semantically similar RS images. To address this, we introduce a remote sensing multimodal visual patterns (RSMMVP) benchmark. It is designed to evaluate MLLMs in RS tasks by identifying the CLIP-blind pairs, where CLIP-based models incorrectly assign high similarity scores to visually distinct RS images. Through a visual question answering (VQA) evaluation, we analyze the performance of state-of-the-art MLLMs, revealing significant limitations in RS specific representation learning. The results provide valuable insights into the weaknesses of CLIP-based visual encoding and offer a foundation for future research to develop more effective MLLMs tailored for remote sensing applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
