KM-ViPE：在线紧密耦合视觉-语言-几何融合用于开放词汇语义SLAM
KM-ViPE: Online Tightly Coupled Vision-Language-Geometry Fusion for Open-Vocabulary Semantic SLAM
AI总结：
KM-ViPE通过在线紧密耦合视觉-语言-几何融合，实现开放词汇语义SLAM，适用于动态环境和未校准单目相机。
摘要：
我们提出了KM-ViPE（知识映射视频姿态引擎），一种适用于动态环境的实时开放词汇SLAM框架，适用于未校准的单目相机。不同于需要深度传感器和离线校准的系统，KM-ViPE直接在原始RGB流上运行，使其成为用于自我中心应用和获取互联网级视频数据进行训练的理想选择。KM-ViPE通过基于高级特征的自适应稳健核紧密耦合DINO视觉特征与几何约束，能够处理移动物体和可移动静态物体（例如自我中心视图中的移动家具）。该系统通过融合与语言嵌入对齐的几何和深度视觉特征，实现同时在线定位和开放词汇语义映射。我们的结果与最先进的方法竞争，而现有解决方案要么需要离线操作，要么需要深度数据和/或里程计估计，或者缺乏动态场景的鲁棒性。KM-ViPE受益于互联网级训练，并且唯一结合了在线操作、未校准单目输入和动态场景的稳健处理，使其成为自主机器人和AR/VR应用的良好选择，并推动了具身AI的实用空间智能能力。
英文摘要：
We present KM-ViPE (Knowledge Mapping Video Pose Engine), a real-time open-vocabulary SLAM framework for uncalibrated monocular cameras in dynamic environments. Unlike systems requiring depth sensors and offline calibration, KM-ViPE operates directly on raw RGB streams, making it ideal for ego-centric applications and harvesting internet-scale video data for training. KM-ViPE tightly couples DINO visual features with geometric constraints through a high-level features based adaptive robust kernel that handles both moving objects and movable static objects (e.g., moving furniture in ego-centric views). The system performs simultaneous online localization and open-vocabulary semantic mapping by fusing geometric and deep visual features aligned with language embeddings. Our results are competitive with state-of-the-art approaches, while existing solutions either operate offline, need depth data and/or odometry estimation, or lack dynamic scene robustness. KM-ViPE benefits from internet-scale training and uniquely combines online operation, uncalibrated monocular input, and robust handling of dynamic scenes, which makes it a good fit for autonomous robotics and AR/VR applications and advances practical spatial intelligence capabilities for embodied AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
