SceneGraphGrounder: 通过结构化场景图匹配实现零样本3D视觉定位
SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching
AI总结：
本文提出SceneGraphGrounder框架，通过结构化场景图匹配将3D定位问题转化为结构化图匹配问题，利用视觉标记提示策略从2D视图推断物体间关系，并在3D场景图中建立持久编码，从而在ScanRefer基准测试中实现了零样本条件下与现有方法相当的性能，并在真实机器人部署中验证了其在长周期物理环境中的鲁棒空间推理能力。
摘要：
零样本3D视觉定位需要从非结构化环境中通过自由形式自然语言定位物体。最近的视觉-语言模型（VLM）方法取得了有希望的结果，但依赖于视点依赖的推理或隐式表示，限制了组合查询的空间一致性和可解释性。我们提出了SceneGraphGrounder，一个将3D定位重新表述为在重建的3D场景图上的结构化图匹配的框架。为了实现这种表述，我们引入了一种视觉标记提示策略，使VLM能够从2D视图推断物体-物体关系，这些关系随后被提升为持久的3D场景图编码，既包含空间关系又包含语义关系。给定一个查询，我们构建查询图并与场景图进行受限对齐，确保多视图一致性和可解释的推理。在ScanRefer基准测试中，我们的方法在零样本条件下实现了与现有方法相当的性能，仅使用RGB-D输入。我们进一步通过在移动机器人上的真实世界部署验证了我们的框架，展示了其在长周期物理环境中的鲁棒空间推理能力。我们将在接受后公开我们的代码。
英文摘要：
Zero-shot 3D visual grounding requires localizing objects in unstructured environments from free-form natural language. Recent vision-language model (VLM) approaches achieve promising results but rely on view-dependent reasoning or implicit representations, limiting spatial consistency and interpretability for compositional queries. We propose SceneGraphGrounder, a framework that reformulates 3D grounding as structured graph matching over a reconstructed 3D scene graph. To enable this formulation, we introduce a visual marker prompting strategy that enables a VLM to infer object-object relationships from 2D views, which are subsequently lifted into a persistent 3D scene graph encoding both spatial and semantic relations. Given a query, we construct a query graph and perform constrained alignment with the scene graph, ensuring multi-view consistency and interpretable reasoning. Experiments on the ScanRefer benchmark demonstrate that our method achieves competitive performance among zero-shot approaches, using only RGB-D inputs. We further validate our framework through real-world deployment on a mobile robot, demonstrating robust spatial reasoning in long-horizon physical environments. We will make our code publicly available upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
