CityCube：在城市环境中对视觉-语言模型的跨视角空间推理进行基准测试
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments
AI总结：
CityCube是一个用于评估视觉-语言模型在城市环境中跨视角空间推理能力的基准，通过多视角问答对揭示模型与人类表现的差距。
摘要：
跨视角空间推理对于具身AI至关重要，支撑着在复杂环境中空间理解、心理模拟和规划。现有基准主要强调室内或街道场景，忽视了开放性城市空间的独特挑战，这些空间具有丰富的语义、复杂的几何结构和视角变化。为此，我们引入了CityCube，一个系统性的基准，旨在探测当前VLMs在城市环境中的跨视角推理能力。CityCube集成了四种视角动态以模拟摄像机运动，并涵盖了从多种平台（如车辆、无人机和卫星）的广泛视角范围。为了全面评估，它包含5,022个精心标注的多视角问答对，分为五个认知维度和三种空间关系表达。对33个VLMs的全面评估揭示了显著的性能差异：即使大规模模型也难以超过54.1%的准确率，仍有34.2%低于人类表现。相比之下，小规模微调的VLMs准确率超过60.0%，突显了我们基准的必要性。进一步分析表明，VLMs与人类推理之间存在任务相关性和根本认知差异。
英文摘要：
Cross-view spatial reasoning is essential for embodied AI, underpinning spatial understanding, mental simulation and planning in complex environments. Existing benchmarks primarily emphasize indoor or street settings, overlooking the unique challenges of open-ended urban spaces characterized by rich semantics, complex geometries, and view variations. To address this, we introduce CityCube, a systematic benchmark designed to probe cross-view reasoning capabilities of current VLMs in urban settings. CityCube integrates four viewpoint dynamics to mimic camera movements and spans a wide spectrum of perspectives from multiple platforms, e.g., vehicles, drones and satellites. For a comprehensive assessment, it features 5,022 meticulously annotated multi-view QA pairs categorized into five cognitive dimensions and three spatial relation expressions. A comprehensive evaluation of 33 VLMs reveals a significant performance disparity with humans: even large-scale models struggle to exceed 54.1% accuracy, remaining 34.2% below human performance. By contrast, small-scale fine-tuned VLMs achieve over 60.0% accuracy, highlighting the necessity of our benchmark. Further analyses indicate the task correlations and fundamental cognitive disparity between VLMs and human-like reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
