arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2405.02162cs.CVcs.AIcs.RO

映射未知:利用基础模型的动态标签统一提示式全景映射

Mapping the Unseen: Unified Promptable Panoptic Mapping with Dynamic Labeling using Foundation Models

1 Applied AI Institute, Moscow 121205 , Russia;(R.S.) 2 School of Computer Science, University of Lincoln, Lincoln LN6 7TS , UK 3 Independent Researcher, Dubai 500001, United Arab Emirates

Mohamad Al Mdfaa, Raghad Salameh, Geesara Kulathunga, Sergey Zagoruyko, Gonzalo Ferrer

AI总结:

UPPM通过基础模型引入动态描述符,实现开放式词汇与统一结构的协调,提升全景地图的几何和语义准确性。

摘要:

全景地图使机器人能够同时推理几何和语义信息。然而,开放式词汇模型反复生成密切相关标签,导致全景实体被分割并降低体积一致性。所提出的UPPM通过利用基础模型引入全景动态描述符,以协调开放式词汇标签与统一类别结构和几何尺寸先验。此类动态描述符的融合是在多分辨率多TSDF地图中通过语言引导的开放式词汇全景分割和语义检索进行的,从而得到一个持久且可提示的全景地图,而无需额外模型训练。基于我们的评估实验,UPPM在地图重建准确性和全景分割质量方面表现最佳。消融研究调查了UPPM每个组件(自定义NMS、模糊帧过滤和统一语义)对整体系统性能的贡献。因此,UPPM在保持开放式词汇可解释性的同时,提供了强大的几何和全景准确性。

英文摘要:

Panoptic maps enable robots to reason about both geometry and semantics. However, open-vocabulary models repeatedly produce closely related labels that split panoptic entities and degrade volumetric consistency. The proposed UPPM advances open-world scene understanding by leveraging foundation models to introduce a panoptic Dynamic Descriptor that reconciles open-vocabulary labels with unified category structure and geometric size priors. The fusion for such dynamic descriptors is performed within a multi-resolution multi-TSDF map using language-guided open-vocabulary panoptic segmentation and semantic retrieval, resulting in a persistent and promptable panoptic map without additional model training. Based on our evaluation experiments, UPPM shows the best overall performance in terms of the map reconstruction accuracy and the panoptic segmentation quality. The ablation study investigates the contribution for each component of UPPM (custom NMS, blurry-frame filtering, and unified semantics) to the overall system performance. Consequently, UPPM preserves open-vocabulary interpretability while delivering strong geometric and panoptic accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑