Open3D-VQA：面向开放空间多模态大语言模型综合空间推理的基准
Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space
AI总结：
本研究提出用于评估多模态大语言模型空中视角空间推理能力的基准Open3D-VQA，含7.3万跨模态问答对，覆盖7类任务，实验揭示了现有模型的能力特点与微调效果，并开源相关工具。
摘要：
空间推理是多模态大语言模型（MLLMs）的一项核心能力，但其在开放空中环境中的表现仍未得到充分探究。本研究提出了Open3D-VQA，这是一个全新的基准，用于评估MLLMs从空中视角推理复杂空间关系的能力。该基准包含7.3万个问答对，覆盖7类通用空间推理任务，题型包括选择题、判断题和简答题，同时支持视觉和点云两种模态。所有问题均基于从真实和模拟空中场景中提取的空间关系自动生成。通过对13款主流MLLMs的评估发现：1）模型回答相对空间关系问题的表现普遍优于绝对距离问题；2）3D大语言模型并未展现出相较于2D大语言模型的显著优势；3）仅在模拟数据集上进行微调，就能显著提升模型在真实场景中的空间推理表现。我们已开源该基准、数据生成流水线和评估工具包，以支持后续研究，相关资源见：https://github.com/EmbodiedCity/Open3D-VQA.code。
英文摘要：
Spatial reasoning is a fundamental capability of multimodal large language models (MLLMs), yet their performance in open aerial environments remains underexplored. In this work, we present Open3D-VQA, a novel benchmark for evaluating MLLMs' ability to reason about complex spatial relationships from an aerial perspective. The benchmark comprises 73k QA pairs spanning 7 general spatial reasoning tasks, including multiple-choice, true/false, and short-answer formats, and supports both visual and point cloud modalities. The questions are automatically generated from spatial relations extracted from both real-world and simulated aerial scenes. Evaluation on 13 popular MLLMs reveals that: 1) Models are generally better at answering questions about relative spatial relations than absolute distances, 2) 3D LLMs fail to demonstrate significant advantages over 2D LLMs, and 3) Fine-tuning solely on the simulated dataset can significantly improve the model's spatial reasoning performance in real-world scenarios. We release our benchmark, data generation pipeline, and evaluation toolkit to support further research: https://github.com/EmbodiedCity/Open3D-VQA.code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
