Service robots in human-centered environments such as hospitals, office buildings, and long-term care homes need to navigate while adhering to social norms to ensure the safety and comfortability of the people they are sharing the space with. Furthermore, they need to adapt to new social scenarios that can arise during robot navigation. In this paper, we present a novel Online Lifelong Vision Language architecture, OLiVia- Nav, which uniquely integrates vision-language models (VLMs) with an online lifelong learning framework for robot social navigation. We introduce a unique distillation approach, Social Context Contrastive Language Image Pre-training (SC-CLIP), to transfer the social reasoning capabilities of large VLMs to a lightweight VLM, in order for OLiVia-Nav to directly encode social and environment context during robot navigation. These encoded embeddings are used to generate and select robot social compliant trajectories. The lifelong learning capabilities of SC-CLIP enable OLiVia-Nav to update the robot trajectory planning overtime as new social scenarios are encountered. We conducted extensive real-world experiments in diverse social navigation scenarios. The results showed that OLiVia-Nav outperformed existing state-of-the-art DRL and VLM methods in terms of mean squared error, Hausdorff loss, and personal space violation duration. Ablation studies also verified the design choices for OLiVia-Nav.
Robust Lifelong Indoor LiDAR Localization using the Area Graph
ShanghaiTech University（上海科技大学）；School of Information Science and Technology (SIST)（信息科学与技术学院）；Key Laboratory of Intelligent Perception and Human-Machine Collaboration（智能感知与人机协同重点实验室）
本文提出基于 Area Graph 的终身室内 3D LiDAR 定位方法，通过语义建筑特征、点云滤波及加权点到线 ICP 实现鲁棒全局定位与位姿跟踪。
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Sony AI（索尼人工智能）
This paper presents a self-improving lifelong learning framework for a mobile robot navigating in different environments. Classical static navigation methods require environment-specific in-situ system adjustment, e.g. from human experts, or may repeat their mistakes regardless of how many times they have navigated in the same environment. Having the potential to improve with experience, learning-based navigation is highly dependent on access to training resources, e.g. sufficient memory and fast computation, and is prone to forgetting previously learned capability, especially when facing diff…
University of Toronto Institute for Aerospace Studies (UTIAS)（多伦多大学航空航天研究所）；ETH Zürich（苏黎世联邦理工学院）；Apple Inc.（苹果公司）
We present a novel method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future information of dynamic scenes. Our automated generation process creates groundtruth SOGMs from previous navigation data. We build on prior work to annotate lidar points based on their dynamic properties, which are then projected on time-stamped 2D grids: SOGMs. We design a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for robots.…
State Key Laboratory of Robotics and Intelligent Systems（机器人与智能系统国家重点实验室）；Shenyang Institute of Automation（沈阳自动化研究所）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation
State Key Laboratory of Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）
University of Montréal（蒙特利尔大学）；Mila（米拉研究所）；Element AI（Element AI公司）
Commonly, learning-based topological navigation approaches produce a local policy while preserving some loose connectivity of the space through a topological map. Nevertheless, spurious or missing edges in the topological graph often lead to navigation failure. In this work, we propose a sampling-based graph building method, which results in sparser graphs yet with higher navigation performance compared to baseline methods. We also propose graph maintenance strategies that eliminate spurious edges and expand the graph as needed, which improves lifelong navigation performance. Unlike controller…
GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation
Georgia Institute of Technology（佐治亚理工学院）；Carnegie Mellon University（卡内基梅隆大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mistral AI（米斯特拉尔人工智能公司）；University of Washington（华盛顿大学）
CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigation
Purdue University（普渡大学）；Indian Institute of Technology, Kharagpur（印度理工学院卡哈拉格普尔分校）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）；National University of Singapore（新加坡国立大学）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Berkeley（加州大学伯克利分校）；Meta AI Research（Meta AI研究院）；Mistral AI
Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong University of Science and Technology（香港科技大学）
University of Freiburg（弗赖堡大学）；University of Technology Nuremberg（纽伦堡工业大学）
Robots operating in the open world encounter various different environments that can substantially differ from each other. This domain gap also poses a challenge for Simultaneous Localization and Mapping (SLAM) being one of the fundamental tasks for navigation. In particular, learning-based SLAM methods are known to generalize poorly to unseen environments hindering their general adoption. In this work, we introduce the novel task of continual SLAM extending the concept of lifelong SLAM from a single dynamically changing environment to sequential deployments in several drastically differing en…
The role of deep learning (DL) in robotics has significantly deepened over the last decade. Intelligent robotic systems today are highly connected systems that rely on DL for a variety of perception, control, and other tasks. At the same time, autonomous robots are being increasingly deployed as part of fleets, with collaboration among robots becoming a more relevant factor. From the perspective of collaborative learning, federated learning (FL) enables continuous training of models in a distributed, privacy-preserving way. This paper focuses on vision-based obstacle avoidance for mobile robot…
This paper presents a novel robot-environment interaction in navigation tasks such that robots have neither a representation of their working space nor planning function, instead, an active environment takes charge of these aspects. This is realized by spatially deploying computing units, called cells, and making cells manage traffic in their respective physical region. Different from stigmegic approaches, cells interact with each other to manage environmental information and to construct instructions on how robots move. As a proof-of-concept, we present an architecture called AFADA and its pr…
OpenNavMap: Multi-Session Appearance-Based Topometric Mapping for Scalable Visual Navigation
Robot Perception and Learning Lab, Intelligent Robotics, Department of Computer Science, University College London（机器人感知与学习实验室，智能机器人，计算机科学系，伦敦大学学院）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）；Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Institute of Robotics and Automatic Information System, Nankai University（机器人与自动信息系统研究所，南开大学）；College of control science and engineering, Zhejiang University（控制科学与工程学院，浙江大学）；AI Centre, Department of Computer Science, University College London（人工智能中心，计算机科学系，伦敦大学学院）
R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Space
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Esslingen University of Applied Sciences（埃斯林根应用科学大学）；Dr. Ing. h.c. F. Porsche AG（德意志联邦汽车工业协会）；University of Michigan（密歇根大学）；Voxel51 Inc.（Voxel51公司）
