面向语言引导机器人导航的分层开放词汇3D场景图
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation
AI总结：
本文提出HOV-SG分层开放词汇3D场景图方法，融合视觉基础模型构建楼层-房间-物体层级语义表示，在提升开放词汇语义准确率的同时大幅压缩地图规模，并验证了其在真实多层环境中语言引导导航的有效性。
摘要：
近期的开放词汇机器人建图方法利用预训练的视觉-语言特征来丰富稠密几何地图。尽管这些地图在查询某一语言概念时能够预测逐点显著性图，但大规模环境以及超出物体级别的抽象查询仍然构成相当大的障碍，最终限制了语言引导的机器人导航。在本工作中，我们提出了HOV-SG，一种面向语言引导机器人导航的分层开放词汇3D场景图建图方法。借助开放词汇视觉基础模型，我们首先在3D中获得当前最优的开放词汇分割级地图，随后构建由楼层、房间和物体概念组成的3D场景图层级结构，每个层级都富含开放词汇特征。我们的方法能够表示多层建筑，并允许机器人利用跨楼层Voronoi图在这些建筑中进行遍历。HOV-SG在三个不同的数据集上进行了评估，在物体、房间和楼层级别上的开放词汇语义准确率超过了以往基线，同时与稠密开放词汇地图相比，表示大小减少了75%。为了证明HOV-SG的有效性和泛化能力，我们展示了在真实世界多层存储环境中成功的长期语言条件机器人导航。我们在http://hovsg.github.io/提供了代码和试验视频数据。
英文摘要：
Recent open-vocabulary robot mapping methods enrich dense geometric maps with pre-trained visual-language features. While these maps allow for the prediction of point-wise saliency maps when queried for a certain language concept, large-scale environments and abstract queries beyond the object level still pose a considerable hurdle, ultimately limiting language-grounded robotic navigation. In this work, we present HOV-SG, a hierarchical open-vocabulary 3D scene graph mapping approach for language-grounded robot navigation. Leveraging open-vocabulary vision foundation models, we first obtain state-of-the-art open-vocabulary segment-level maps in 3D and subsequently construct a 3D scene graph hierarchy consisting of floor, room, and object concepts, each enriched with open-vocabulary features. Our approach is able to represent multi-story buildings and allows robotic traversal of those using a cross-floor Voronoi graph. HOV-SG is evaluated on three distinct datasets and surpasses previous baselines in open-vocabulary semantic accuracy on the object, room, and floor level while producing a 75% reduction in representation size compared to dense open-vocabulary maps. In order to prove the efficacy and generalization capabilities of HOV-SG, we showcase successful long-horizon language-conditioned robot navigation within real-world multi-storage environments. We provide code and trial video data at http://hovsg.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
