开放词汇八叉树图用于3D场景理解
Open-Vocabulary Octree-Graph for 3D Scene Understanding
AI总结：
本文提出Octree-Graph，通过CGSM和IFA算法获取3D实例及语义特征，构建适应性八叉树结构以高效表示场景，实验表明其在多种任务中具有广泛适用性和有效性。
摘要：
开放词汇3D场景理解对具身智能至关重要。近期工作利用预训练视觉-语言模型（VLMs）进行物体分割并投影到点云以构建3D地图。尽管有所进展，点云是一组无序坐标，需要大量存储空间且不直接传达占用信息或空间关系，使现有方法在下游任务如路径规划和基于文本的对象检索中效率低下。为此，我们提出Octree-Graph，一种新的开放词汇3D场景表示方法。具体而言，首先设计了Chronological Group-wise Segment Merging（CGSM）策略和Instance Feature Aggregation（IFA）算法以获取3D实例及其对应的语义特征。随后开发了自适应八叉树结构，该结构根据物体形状可调整地存储语义并表示占用情况。最后构建Octree-Graph，其中每个自适应八叉树作为图节点，边描述节点间的空间关系。在多个广泛使用的数据集上进行了大量实验，证明了本方法的通用性和有效性。代码可在此获取：https://github.com/yifeisu/OV-Octree-Graph
英文摘要：
Open-vocabulary 3D scene understanding is indispensable for embodied agents. Recent works leverage pretrained vision-language models (VLMs) for object segmentation and project them to point clouds to build 3D maps. Despite progress, a point cloud is a set of unordered coordinates that requires substantial storage space and does not directly convey occupancy information or spatial relation, making existing methods inefficient for downstream tasks, e.g., path planning and text-based object retrieval. To address these issues, we propose \textbf{Octree-Graph}, a novel scene representation for open-vocabulary 3D scene understanding. Specifically, a Chronological Group-wise Segment Merging (CGSM) strategy and an Instance Feature Aggregation (IFA) algorithm are first designed to get 3D instances and corresponding semantic features. Subsequently, an adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape. Finally, the Octree-Graph is constructed where each adaptive-octree acts as a graph node, and edges describe the spatial relations among nodes. Extensive experiments on various tasks are conducted on several widely-used datasets, demonstrating the versatility and effectiveness of our method. Code is available \href{https://github.com/yifeisu/OV-Octree-Graph}{here}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
