MonoSR: 从单目图像进行开放词汇空间推理
MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images
AI总结：
本文提出MonoSR大规模单目空间推理数据集，涵盖多种场景并支持多种问题类型，评估先进视觉-语言模型并分析辅助信息对单目空间推理的重要性，为开放世界中的单目空间推理提供基础。
摘要：
空间推理（SR）是指从2D输入推断3D空间信息的能力，对现实世界应用如具身AI和自动驾驶至关重要。然而，现有研究主要聚焦于室内环境，通常依赖多视角观测，限制了其在户外场景中的泛化能力，并限制了其在单目图像上的应用，而单目图像是最常见的现实场景。在本文中，我们提出MonoSR，一个大规模单目空间推理数据集，涵盖室内、户外和以物体为中心的场景，并支持多种问题类型。MonoSR为开放世界的单目空间推理提供了路径。除了介绍数据集，我们评估了先进视觉-语言模型，揭示其在这一挑战性任务上的局限性。我们进一步分析辅助信息是否对单目空间推理至关重要，并为未来模型的设计提供实用指导。这些贡献共同为在现实世界和开放世界中推进单目空间推理奠定了基础。
英文摘要：
Spatial reasoning (SR), the ability to infer 3D spatial information from 2D inputs, is essential for real-world applications such as embodied AI and autonomous driving. However, existing research primarily focuses on indoor environments and typically relies on multi-view observations, which limits their generalizability to outdoor scenarios and constrains their applicability to monocular images, the most common real-world setting. In this work, we propose MonoSR, a large-scale monocular spatial reasoning dataset that spans diverse scenarios including indoor, outdoor, and object-centric settings, and supports multiple question types. MonoSR provides a path toward open-world monocular spatial reasoning. Beyond introducing the dataset, we evaluate advanced vision-language models to reveal their limitations on this challenging task. We further analyze whether auxiliary information is crucial for monocular spatial reasoning and offer practical guidance for designing future models. These contributions collectively establish a foundation for advancing monocular spatial reasoning in real-world, open-world environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
