视图-图：通过场景图上的视觉-语言推理实现零样本3D视觉定位
View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs
AI总结：
本文提出视图-图方法，通过场景图上的视觉-语言推理实现零样本3D视觉定位，有效解决空间语义关系处理难题。
摘要：
3D视觉定位（3DVG）从语言描述中识别3D场景中的物体。现有的零样本方法利用2D视觉-语言模型（VLMs）通过将3D空间信息（SI）转换为可被VLM处理的形式，通常作为复合输入，如指定视角的渲染或带有叠加物体标记的视频序列。然而，这种VLM + SI范式会产生纠缠的视觉表示，迫使VLM处理整个杂乱的提示，使其难以有效利用空间语义关系。在本工作中，我们提出了一种新的VLM x SI范式，将3D SI外部化为一种形式，使VLM在推理过程中能够逐步检索所需的内容。我们通过一种新的视图-图（VoG）方法实现了这一范式，该方法将场景组织成多模态、多层的场景图，并允许VLM作为主动代理，在遍历场景时选择性地访问必要的提示。这种设计提供了两个内在优势：（i）通过将3D上下文结构化为空间和语义上连贯的场景图，而不是将VLM与密集纠缠的视觉输入混淆，从而降低了VLM的推理难度；（ii）通过主动探索和推理场景图，自然地产生透明的、逐步的跟踪，以实现可解释的3DVG。大量实验表明，VoG在零样本性能上达到了最先进的水平，确立了结构化场景探索作为推进零样本3DVG的有前景策略。
英文摘要：
3D visual grounding (3DVG) identifies objects in 3D scenes from language descriptions. Existing zero-shot approaches leverage 2D vision-language models (VLMs) by converting 3D spatial information (SI) into forms amenable to VLM processing, typically as composite inputs such as specified view renderings or video sequences with overlaid object markers. However, this VLM + SI paradigm yields entangled visual representations that compel the VLM to process entire cluttered cues, making it hard to exploit spatial semantic relationships effectively. In this work, we propose a new VLM x SI paradigm that externalizes the 3D SI into a form enabling the VLM to incrementally retrieve only what it needs during reasoning. We instantiate this paradigm with a novel View-on-Graph (VoG) method, which organizes the scene into a multi-modal, multi-layer scene graph and allows the VLM to operate as an active agent that selectively accesses necessary cues as it traverses the scene. This design offers two intrinsic advantages: (i) by structuring 3D context into a spatially and semantically coherent scene graph rather than confounding the VLM with densely entangled visual inputs, it lowers the VLM's reasoning difficulty; and (ii) by actively exploring and reasoning over the scene graph, it naturally produces transparent, step-by-step traces for interpretable 3DVG. Extensive experiments show that VoG achieves state-of-the-art zero-shot performance, establishing structured scene exploration as a promising strategy for advancing zero-shot 3DVG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
