arXiv 2506.048972025-10-28cs.CV

From Objects to Anywhere：3D场景中多层次视觉定位的综合基准

From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes

State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；Tsinghua University（清华大学）；Peking University（北京大学）；Beijing Institute of Technology（北京理工大学）

Tianxu Wang, Zhuofan Zhang, Ziyu Zhu, Yue Fan, Jing Xiong, Pengxiang Li, Xiaojian Ma, Qing Li

备注：: Update v3 of the NeurIPS 2025 Datasets and Benchmarks paper (v2), including additional evaluations of state-of-the-art multimodal large language models. Project page: https://anywhere-3d.github.io/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对3D视觉定位仅聚焦物体、缺乏多层次定位研究的问题，本文提出包含四个定位层级的综合基准Anywhere3D-Bench，评估发现现有模型在空间和部件层级任务上表现不佳，凸显了相关能力缺口。

摘要：

3D视觉定位在复杂3D场景中的物体定位方面已取得显著进展。然而，在3D场景中对超出物体范畴的指代表达进行定位仍有待探索。本文提出了Anywhere3D-Bench，这是一个综合性的3D视觉定位基准，包含2886个指代表达-3D边界框对，覆盖四个不同的定位层级：人类活动区域、物体外的未占用空间、场景中的单个物体以及细粒度的物体部件。我们在Anywhere3D-Bench上评估了一系列最先进的3D视觉定位方法，以及大语言模型（LLM）和多模态大语言模型（MLLM）。实验结果表明，空间层级和部件层级的视觉定位带来了最大的挑战：空间层级任务需要更全面的空间推理能力，例如对3D空间内的距离和空间关系进行建模；而部件层级任务则需要对物体组成结构的细粒度感知。即便是表现最佳的模型Google Gemini-2.5-Pro和OpenAI o3，在空间层级任务上的准确率也仅约30%，在部件层级任务上约为40%，远低于其在区域层级和物体层级任务上的表现。这些发现凸显了当前模型在理解和推理超出物体层级语义的3D场景方面存在的关键不足。

英文摘要：

3D visual grounding has made notable progress in localizing objects within complex 3D scenes. However, grounding referring expressions beyond objects in 3D scenes remains unexplored. In this paper, we introduce Anywhere3D-Bench, a holistic 3D visual grounding benchmark consisting of 2,886 referring expression-3D bounding box pairs spanning four different grounding levels: human-activity areas, unoccupied space beyond objects, individual objects in the scene, and fine-grained object parts. We assess a range of state-of-the-art 3D visual grounding methods alongside large language models (LLMs) and multimodal LLMs (MLLMs) on Anywhere3D-Bench. Experimental results reveal that space-level and part-level visual grounding pose the greatest challenges: space-level tasks require a more comprehensive spatial reasoning ability, for example, modeling distances and spatial relations within 3D space, while part-level tasks demand fine-grained perception of object composition. Even the best-performing models, Google Gemini-2.5-Pro and OpenAI o3, achieve just around 30% accuracy on space-level tasks and around 40% on part-level tasks, significantly lower than its performance on area-level and object-level tasks. These findings underscore a critical gap in current models' capacity to understand and reason about 3D scenes beyond object-level semantics.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。