QueSTMaps: 面向3D场景理解的可查询语义拓扑地图
QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understanding
AI总结：
针对现有3D场景理解方法难以分割拓扑区域的问题，提出QueSTMaps两步流水线，通过多通道占用表示提取拓扑地图、自注意力Transformer生成CLIP对齐特征，支持自然语言查询，在房间分割与分类任务上性能显著优于现有方法。
摘要：
规划、导航等机器人任务需要对场景进行层级化语义理解，场景可能包含多个楼层与房间。现有方法主要聚焦于用于3D场景理解的目标分割，但这类方法难以分割出场景中类似“厨房”的拓扑区域。本文提出一种两步式流水线解决该问题：首先，采用新型多通道占用表示提取室内场景的拓扑地图（即楼层平面图）；随后，基于每个房间实例包含的物体，使用自注意力Transformer生成与CLIP对齐的特征及语义标签。我们的语言-拓扑对齐机制支持自然语言查询，例如输入“做饭的地方”可定位到“厨房”。在房间分割任务上，我们的方法比当前最优方法性能提升约20%，房间分类任务提升约12%。详细的定性分析与消融实验为3D场景结构与语义联合理解问题提供了洞见。项目主页：quest-maps.github.io
英文摘要：
Robotic tasks such as planning and navigation require a hierarchical semantic understanding of a scene, which could include multiple floors and rooms. Current methods primarily focus on object segmentation for 3D scene understanding. However, such methods struggle to segment out topological regions like "kitchen" in the scene. In this work, we introduce a two-step pipeline to solve this problem. First, we extract a topological map, i.e., floorplan of the indoor scene using a novel multi-channel occupancy representation. Then, we generate CLIP-aligned features and semantic labels for every room instance based on the objects it contains using a self-attention transformer. Our language-topology alignment supports natural language querying, e.g., a "place to cook" locates the "kitchen". We outperform the current state-of-the-art on room segmentation by ~20% and room classification by ~12%. Our detailed qualitative analysis and ablation studies provide insights into the problem of joint structural and semantic 3D scene understanding. Project Page: quest-maps.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
