arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00637cs.CV
收藏

VIZOR: 视点不变的零样本场景图生成用于3D场景推理

VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoning

Machine Learning Lab, IIIT Hyderabad, India（IIIT海得拉巴机器学习实验室）；Fujitsu Research India, Bangalore（印度班加罗尔 Fujitsu 研究院）

Vivek Madhavaram, Vartika Sengar, Arkadipta De, Charu Sharma

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

VIZOR通过无需训练的端到端框架，生成视点不变的零样本场景图，提升3D场景推理的准确性和泛化能力。

摘要：

场景理解和推理是3D计算机视觉中的基础问题，要求模型能够识别物体、其属性以及物体之间的空间或比较关系。现有方法通过使用多种输入（如2D图像、深度图、物体标签和特定参考视图的标注关系）来创建场景图。然而，这些方法常常在泛化方面存在问题，并且会产生不准确的空间关系如“左右”，这些关系在不同视点下变得不一致。为了解决这些限制，我们提出了用于3D场景推理的视点不变零样本场景图生成（VIZOR）。VIZOR是一种无需训练的端到端框架，它直接从原始3D场景中构建密集且视点不变的3D场景图。生成的场景图是明确的，因为空间关系是相对于每个物体的正面方向定义的，因此无论参考视图如何，它们都保持一致。此外，它推断出开放词汇的关系，描述场景物体之间的空间和接近关系，而无需标注的训练数据。我们进行了广泛的定量和定性评估，以评估VIZOR在场景图生成和下游任务（如基于查询的对象定位）中的有效性。VIZOR在场景图生成方面优于最先进的方法，显示出在场景图生成中的明显改进，并在Replica和Nr3D数据集上分别实现了22%和4.81%的零样本定位准确率提升。

英文摘要：

Scene understanding and reasoning has been a fundamental problem in 3D computer vision, requiring models to identify objects, their properties, and spatial or comparative relationships among the objects. Existing approaches enable this by creating scene graphs using multiple inputs such as 2D images, depth maps, object labels, and annotated relationships from specific reference view. However, these methods often struggle with generalization and produce inaccurate spatial relationships like "left/right", which become inconsistent across different viewpoints. To address these limitations, we propose Viewpoint-Invariant Zero-shot scene graph generation for 3D scene Reasoning (VIZOR). VIZOR is a training-free, end-to-end framework that constructs dense, viewpoint-invariant 3D scene graphs directly from raw 3D scenes. The generated scene graph is unambiguous, as spatial relationships are defined relative to each object's front-facing direction, making them consistent regardless of the reference view. Furthermore, it infers open-vocabulary relationships that describe spatial and proximity relationships among scene objects without requiring annotated training data. We conduct extensive quantitative and qualitative evaluations to assess the effectiveness of VIZOR in scene graph generation and downstream tasks, such as query-based object grounding. VIZOR outperforms state-of-the-art methods, showing clear improvements in scene graph generation and achieving 22% and 4.81% gains in zero-shot grounding accuracy on the Replica and Nr3D datasets, respectively.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑