VertiCue-Bench: 诊断多模态大语言模型是否利用高度线索解决遥感自然场景中的二维歧义
Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?
AI总结：
提出VertiCue-Bench基准，通过17个任务1534个实例诊断MLLMs是否真正利用冠层高度模型（CHM）的垂直线索解决遥感自然场景中的语义歧义，发现模型在感知高度线索与语义推理之间存在显著脱节。
摘要：
多模态大语言模型（MLLMs）最近在地理空间推理方面显示出有希望的进展。然而，现有的遥感基准仍然主要围绕二维中心，主要基于光学外观评估模型。在自然环境中，由于严重的光谱混淆，这种范式失效，其中生态上不同的区域共享相似的纹理但在垂直结构上根本不同。在这种情况下，明确的3D结构数据，如冠层高度模型（CHMs），成为语义消歧的基本几何证据。然而，目前尚不清楚当前的MLLMs是否能够真正利用垂直线索来解决外观级别的歧义。为了填补这一空白，我们引入了VertiCue-Bench，这是第一个基于CHM的地理空间推理诊断基准。VertiCue-Bench包含1534个精心策划的实例，涵盖17个任务，明确将低级高度感知与歧义感知的语义推理分离。对14个最先进的通用和遥感专用MLLMs的评估，结合反事实模态测试，揭示了惊人的感知-推理分离。虽然模型在读取原始CHM高度线索方面表现出新兴能力，但它们大多未能将几何感知转化为可靠的语义推理，在需要联合约束时通常表现不如仅使用RGB的基线。总体而言，VertiCue-Bench揭示了自然场景理解中关键的几何到语义的差距，为推进地理空间MLLMs提供了可行的见解。
英文摘要：
Multimodal large language models (MLLMs) have advanced rapidly in remote-sensing analysis, yet existing evaluations remain predominantly 2D-centric. Because spectrally confused regions can appear nearly identical yet differ substantially in vertical structure, appearance alone is often insufficient for reliable semantic interpretation in natural scenes. Vertical structure therefore provides decision-critical physical evidence, yet whether current MLLMs can effectively perceive, ground, and utilize such geometric evidence remains underexplored. To bridge this gap, we introduce VertiCue-Bench, the first diagnostic benchmark that uses controlled interventions to probe whether vertical height evidence is actually perceived, grounded, and utilized, and we establish a three-stage evidence-utilization framework of Perception--Grounding--Utilization. By constructing a Representation Intervention Spectrum spanning multiple presentation and interaction modalities, including Raw Visual, Tool-assisted, and Oracle Text conditions, together with controlled counterfactual tests, we conduct an in-depth disentangled diagnosis across 10 state-of-the-art models. Our experiments reveal and formally characterize the Vertical Structure Utilization Gap. Although current models exhibit emerging geometric perception capabilities, they still struggle to accurately ground vertical evidence to relevant spatial entities and integrate it into high-level semantic decisions. This finding identifies a critical bottleneck in developing physically grounded and geometry-aware remote-sensing MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
