We present Multiple Scenario Verifiable Reinforcement Learning via Policy Extraction (MSVIPER), a new method for policy distillation to decision trees for improved robot navigation. MSVIPER learns an "expert" policy using any Reinforcement Learning (RL) technique involving learning a state-action mapping and then uses imitation learning to learn a decision-tree policy from it. We demonstrate that MSVIPER results in efficient decision trees and can accurately mimic the behavior of the expert policy. Moreover, we present efficient policy distillation and tree-modification techniques that take advantage of the decision tree structure to allow improvements to a policy without retraining. We use our approach to improve the performance of RL-based robot navigation algorithms for indoor and outdoor scenes. We demonstrate the benefits in terms of reduced freezing and oscillation behaviors (by up to 95\% reduction) for mobile robots navigating among dynamic obstacles and reduced vibrations and oscillation (by up to 17\%) for outdoor robot navigation on complex, uneven terrains.
TAME-Nav: Temporal Asymmetric Mixture-of-Experts Distillation for Agile and Versatile Robot Navigation
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东方理工大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Science and Technology of China（中国科学技术大学）
ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation
Dept. of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA（电气与计算机工程系，马里兰大学，College Park, MD, USA）；Dept. of Computer Science, University of Maryland, College Park, MD, USA（计算机科学系，马里兰大学，College Park, MD, USA）；James Clark School of Engineering, University of Maryland, College Park, MD, USA（James Clark工程学院，马里兰大学，College Park, MD, USA）
We focus on the problem of teaching a robot to solve tasks presented sequentially, i.e., in a continual learning scenario. The robot should be able to solve all tasks it has encountered, without forgetting past tasks. We provide preliminary work on applying Reinforcement Learning to such setting, on 2D navigation tasks for a 3 wheel omni-directional robot. Our approach takes advantage of state representation learning and policy distillation. Policies are trained using learned features as input, rather than raw observations, allowing better sample efficiency. Policy distillation is used to comb…
ENSTA Paris（巴黎高等先进技术学院）；Institute Polytechnique de Paris（巴黎理工学院）；INRIA（法国国家信息与自动化研究所）；Softbank Robotics Europe（软银机器人欧洲公司）；Thales（泰雷兹集团）
In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an external signal. In the case of continual reinforcement learning a third challenge arises: learning tasks sequentially without forgetting the previous ones. In this paper, we tackle these challenges by proposing DisCoRL, an approach combining state representation learning and policy distillation. We experiment on a sequence of three simulated 2D navigation tasks with a 3 wheel omni-…
Contextual information in human environments, such as signs, symbols, and objects provide important information for robots to use for exploration and navigation. To identify and segment contextual information from complex images obtained in these environments, data-driven methods such as Convolutional Neural Networks (CNNs) are used. However, these methods require large amounts of human labeled data which are slow and time-consuming to obtain. Weakly supervised methods address this limitation by generating pseudo segmentation labels (PSLs). In this paper, we present the novel Weakly Supervised…
Training automated agents to complete complex tasks in interactive environments is challenging: reinforcement learning requires careful hand-engineering of reward functions, imitation learning requires specialized infrastructure and access to a human expert, and learning from intermediate forms of supervision (like binary preferences) is time-consuming and extracts little information from each human intervention. Can we overcome these challenges by building agents that learn from rich, interactive feedback instead? We propose a new supervision paradigm for interactive learning based on "teacha…
C-NAV: Towards Self-Evolving Continual Object Navigation in Open World
Beihang University（北京航空航天大学）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（香港智能科学与工业研究院人工智能与机器人中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新研究院）；Beijing Jiaotong University（北京交通大学）
SCALE: Self-Correcting Visual Navigation for Mobile Robots via Anti-Novelty Estimation
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Huawei（华为）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Shenzhen Huawei Cloud Computing Technologies Co., Ltd.（华为云计算技术有限公司（深圳））
Rectified Schrödinger Bridge Matching for Few-Step Visual Navigation
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；College of Computer Science, Chongqing University（重庆大学计算机学院）；Department of Computer Science, University of Liverpool（利物浦大学计算机科学系）；Changchun GenY Technology Co., Ltd.（长春GenY科技有限公司）
Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School（一般人工智能国家重点实验室，北京大学深圳研究生院）；Oxford Robotics Institute, University of Oxford（牛津大学机器人研究所）；Institute for Machine Learning, Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系机器学习研究所）
We tackle the problem of cooperative visual exploration where multiple agents need to jointly explore unseen regions as fast as possible based on visual signals. Classical planning-based methods often suffer from expensive computation overhead at each step and a limited expressiveness of complex cooperation strategy. By contrast, reinforcement learning (RL) has recently become a popular paradigm for tackling this challenge due to its modeling capability of arbitrarily complex strategies and minimal inference overhead. In this paper, we extend the state-of-the-art single-agent visual navigation…
Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization
Tsinghua University（清华大学）；Tencent AI Lab（腾讯人工智能实验室）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Research Institute of Tsinghua University in Shenzhen（清华大学深圳研究院）
Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations
Southern University of Science and Technology（南方科技大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Manifold Tech Limited（曼孚科技股份有限公司）
