基于3D层次化场景图的开放词汇室内物体定位
Open-Vocabulary Indoor Object Grounding with 3D Hierarchical Scene Graph
AI总结：
提出OVIGo-3DHSG方法，利用3D层次化场景图结合大语言模型进行多步推理，实现开放词汇室内物体定位，在Habitat Matterport数据集上展现出高效的场景理解与稳健的定位能力。
摘要：
我们提出了OVIGo-3DHSG方法——利用3D层次化场景图进行物体的开放词汇室内定位。OVIGo-3DHSG利用一组开放词汇基础模型和传感器数据处理，将从RGB-D帧序列导出的层次化场景图表示为广泛的室内环境。这种层次化表示显式地建模了楼层、房间、位置和物体之间的空间关系。为了有效解决涉及对其他物体空间引用的复杂查询，我们将层次化场景图与大语言模型集成以进行多步推理。这种集成利用了层间（例如房间到物体）和层内（例如物体到物体）连接，增强了空间上下文理解。我们在Habitat Matterport 3D Semantic多层场景上研究了层次化表示的语义和几何准确性。与现有方法相比，我们的方法展示了高效的场景理解和稳健的物体定位。总体而言，OVIGo-3DHSG在需要空间推理和室内环境理解的应用中展现出强大潜力。相关材料可在https://github.com/linukc/OVIGo-3DHSG找到。
英文摘要：
We propose OVIGo-3DHSG method - Open-Vocabulary Indoor Grounding of objects using 3D Hierarchical Scene Graph. OVIGo-3DHSG represents an extensive indoor environment over a Hierarchical Scene Graph derived from sequences of RGB-D frames utilizing a set of open-vocabulary foundation models and sensor data processing. The hierarchical representation explicitly models spatial relations across floors, rooms, locations, and objects. To effectively address complex queries involving spatial reference to other objects, we integrate the hierarchical scene graph with a Large Language Model for multistep reasoning. This integration leverages inter-layer (e.g., room-to-object) and intra-layer (e.g., object-to-object) connections, enhancing spatial contextual understanding. We investigate the semantic and geometry accuracy of hierarchical representation on Habitat Matterport 3D Semantic multi-floor scenes. Our approach demonstrates efficient scene comprehension and robust object grounding compared to existing methods. Overall OVIGo-3DHSG demonstrates strong potential for applications requiring spatial reasoning and understanding of indoor environments. Related materials can be found at https://github.com/linukc/OVIGo-3DHSG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
