arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.11783cs.CVcs.RO

LatentBKI:视觉-语言潜在空间中的开放词典连续建图与可量化不确定性

LatentBKI: Open-Dictionary Continuous Mapping in Visual-Language Latent Spaces with Quantifiable Uncertainty

University of Michigan(密歇根大学)

Joey Wilson, Ruihan Xu, Yile Sun, Parker Ewen, Minghan Zhu, Kira Barton, Maani Ghaffari

AI总结:

本文提出 LatentBKI,一种基于视觉-语言潜在空间和贝叶斯核推断的开放词典连续建图算法,在 Matterport3D 和 Semantic KITTI 数据集上验证了其可量化不确定性和开放查询能力。

摘要:

本文提出了一种新颖的概率建图算法 LatentBKI,能够实现具有可量化不确定性的开放词汇建图。传统上,语义建图算法专注于固定的语义类别集合,这限制了它们在复杂机器人任务中的适用性。视觉-语言(VL)模型最近作为一种在潜在空间中联合建模语言和视觉特征的技术出现,使得能够识别超出预定义固定语义类别的语义。LatentBKI 通过贝叶斯核推断(BKI)利用附近观测的空间相关性,将来自 VL 模型的神经嵌入循环地整合到具有可量化不确定性的体素地图中。LatentBKI 在流行的 Matterport3D 和 Semantic KITTI 数据集上,与类似的显式语义建图和 VL 建图框架进行了评估,结果表明 LatentBKI 保持了连续建图的概率优势,并额外提供了开放词典查询的好处。真实世界实验证明了其在具有挑战性的室内环境中的适用性。

英文摘要:

This paper introduces a novel probabilistic mapping algorithm, LatentBKI, which enables open-vocabulary mapping with quantifiable uncertainty. Traditionally, semantic mapping algorithms focus on a fixed set of semantic categories which limits their applicability for complex robotic tasks. Vision-Language (VL) models have recently emerged as a technique to jointly model language and visual features in a latent space, enabling semantic recognition beyond a predefined, fixed set of semantic classes. LatentBKI recurrently incorporates neural embeddings from VL models into a voxel map with quantifiable uncertainty, leveraging the spatial correlations of nearby observations through Bayesian Kernel Inference (BKI). LatentBKI is evaluated against similar explicit semantic mapping and VL mapping frameworks on the popular Matterport3D and Semantic KITTI datasets, demonstrating that LatentBKI maintains the probabilistic benefits of continuous mapping with the additional benefit of open-dictionary queries. Real-world experiments demonstrate applicability to challenging indoor environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑