CVSBench：跨视角空间推理与想象的全面基准
CVSBench: A Comprehensive Benchmark for Cross-view Spatial Reasoning and Dreaming
AI总结：
提出CVSBench基准，通过卫星-街景对评估视觉语言模型的跨视角空间推理能力，发现模型在视角剧变下难以保持物体与布局一致性，引入3D场景想象管道可显著提升性能。
摘要：
人类可以毫不费力地推理不同视角下的场景，但目前尚不清楚视觉语言模型（VLM）是否具备类似的跨视角空间能力。卫星-街景场景对，凭借其复杂的上下文和极端的视角变化，提供了一个理想的测试平台。受此启发，我们引入了CVSBench，一个通过卫星-街景对评估跨视角空间推理的大规模基准。该基准支持多项任务，包括跨视角VQA、跨视角定位和视角识别。CVSBench包含3,297个跨视角图像组，具有9,468个对象级标注和40,679个问答对，能够系统且受控地评估跨视角空间推理。大量评估表明，先进的VLM在视角剧烈变化下难以保持对象级和布局一致性。为了弥合这一差距，向人类空间认知靠拢，我们研究了两类方法：空间接地推理和认知地图输入的整合。我们的发现表明，仅语言推理带来微小的改进，而通过3D场景想象管道融入视觉空间想象则显著提升了跨视角推理。这些结果强调了显式视觉-空间表示对于VLM稳健空间认知的必要性。我们的数据和代码发布在https://this URL。
英文摘要：
Humans can effortlessly reason about scenes across different viewpoints, yet it remains unclear whether Vision-Language Models (VLMs) possess similar cross-view spatial abilities. Satellite-street scene pairs, with their complex contexts and extreme viewpoint variations, provide an ideal testbed. Motivated by this, we introduce CVSBench, a large-scale benchmark for evaluating cross-view spatial reasoning through satellite-street pairs. This benchmark supports multiple tasks, including cross-view VQA, cross-view grounding, and viewpoint identification. CVSBench comprises 3,297 cross-view image groups with 9,468 object-level annotations and 40,679 question-answer (QA) pairs, enabling systematic and controlled evaluation of cross-view spatial reasoning. Extensive evaluations reveal that advanced VLMs struggle to maintain object-level and layout consistency under drastic viewpoint changes. To bridge this gap towards human-like spatial cognition, we investigate two categories of approaches: spatially grounded reasoning and the incorporation of cognitive map inputs. Our findings demonstrate that language-only reasoning yields marginal improvements, while incorporating visual spatial imagination via a 3D scene imagination pipeline substantially improves cross-view reasoning. These results highlight the necessity of explicit visual-spatial representations for robust spatial cognition in VLMs. Our data and code are released at https://huggingface.co/datasets/zlyzlyzly/CVSBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
