RADIO-ViPE：面向动态环境的在线紧密耦合多模态融合用于开放词汇语义SLAM
RADIO-ViPE: Online Tightly Coupled Multi-Modal Fusion for Open-Vocabulary Semantic SLAM in Dynamic Environments
AI总结：
RADIO-ViPE通过在线多模态融合实现动态环境中开放词汇语义SLAM，无需校准RGB-D输入，结合视觉与语言嵌入提升地图一致性。
摘要：
我们提出了RADIO-ViPE（Reduce All Domains Into One -- Video Pose Engine），一种在线语义SLAM系统，能够实现几何感知的开放词汇定位，将任意自然语言查询与动态环境中局部3D区域和物体关联。与需要校准RGB-D输入的现有方法不同，RADIO-ViPE直接处理原始单目RGB视频流，无需先验相机内参、深度传感器或姿态初始化。系统紧密耦合多模态嵌入——涵盖视觉和语言——由聚合基础模型（如RADIO）生成，并与几何场景信息结合。这种耦合发生在初始化、优化和因子图连接中，以提高多模态地图的一致性。优化被封装在自适应鲁棒核中，以处理主动移动物体和主体位移的场景元素（如会话期间重新排列的家具）。实验表明，RADIO-ViPE在动态TUM-RGBD基准上取得最先进的结果，同时在依赖校准数据和静态场景假设的离线开放词汇方法中保持竞争力。RADIO-ViPE弥合了现实部署中的关键差距，使自主机器人和无约束真实视频流中的鲁棒开放词汇语义定位成为可能。项目页面：https://be2rlab.github.io/radio_vipe
英文摘要：
We present RADIO-ViPE (Reduce All Domains Into One -- Video Pose Engine), an online semantic SLAM system that enables geometry-aware open-vocabulary grounding, associating arbitrary natural language queries with localized 3D regions and objects in dynamic environments. Unlike existing approaches that require calibrated, posed RGB-D input, RADIO-ViPE operates directly on raw monocular RGB video streams, requiring no prior camera intrinsics, depth sensors, or pose initialization. The system tightly couples multi-modal embeddings -- spanning vision and language -- derived from agglomerative foundation models (e.g., RADIO) with geometric scene information. This coupling takes place in initialization, optimization and factor graph connections to improve the consistency of the map from multiple modalities. The optimization is wrapped within adaptive robust kernels, designed to handle both actively moving objects and agent-displaced scene elements (e.g., furniture rearranged during ego-centric session). Experiments demonstrate that RADIO-ViPE achieves state-of-the-art results on the dynamic TUM-RGBD benchmark while maintaining competitive performance against offline open-vocabulary methods that rely on calibrated data and static scene assumptions. RADIO-ViPE bridges a critical gap in real-world deployment, enabling robust open-vocabulary semantic grounding for autonomous robotics and unconstrained in-the-wild video streams. Project page: https://be2rlab.github.io/radio_vipe
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
