Learning Social Navigation from Demonstrations with Conditional Neural Processes
Bogazici University（博阿齐奇大学）
Yigit Yildirim, Emre Ugur
备注：
Author submitted manuscript, accepted for publication in the Special Issue of Interaction Studies, titled "Socially Acceptable Robot Behavior: Approaches for Learning, Adaptation and Evaluation"
Sociability is essential for modern robots to increase their acceptability in human environments. Traditional techniques use manually engineered utility functions inspired by observing pedestrian behaviors to achieve social navigation. However, social aspects of navigation are diverse, changing across different types of environments, societies, and population densities, making it unrealistic to use hand-crafted techniques in each domain. This paper presents a data-driven navigation architecture that uses state-of-the-art neural architectures, namely Conditional Neural Processes, to learn global and local controllers of the mobile robot from observations. Additionally, we leverage a state-of-the-art, deep prediction mechanism to detect situations not similar to the trained ones, where reactive controllers step in to ensure safe navigation. Our results demonstrate that the proposed framework can successfully carry out navigation tasks regarding social norms in the data. Further, we showed that our system produces fewer personal-zone violations, causing less discomfort.
The Hong Kong University of Science and Technology（香港科技大学）
In autonomous driving, navigation through unsignaled intersections with many traffic participants moving around is a challenging task. To provide a solution to this problem, we propose a novel branched network G-CIL for the navigation policy learning. Specifically, we firstly represent such dynamic environments as graph-structured data and propose an effective strategy for edge definition to aggregate surrounding information for the ego-vehicle. Then graph convolutional neural networks are used as the perception module to capture global and geometric features from the environment. To generate …
Northwestern Polytechnical University（西北工业大学）；Cranfield University（克兰菲尔德大学）；City, University of London（伦敦大学城市学院）
In this paper, a deep reinforcement learning (DRL) method is proposed to address the problem of UAV navigation in an unknown environment. However, DRL algorithms are limited by the data efficiency problem as they typically require a huge amount of data before they reach a reasonable performance. To speed up the DRL training process, we developed a novel learning framework which combines imitation learning and reinforcement learning and building upon Twin Delayed DDPG (TD3) algorithm. We newly introduced both policy and Q-value network are learned using the expert demonstration during the imita…
Teaching Robots Like Dogs: Learning Agile Navigation from Luring, Gesture, and Speech
Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Department of Computer Science, ETH Zurich（计算机科学系，苏黎世联邦理工学院）；Department of Mechanical and Aerospace Engineering, UCLA（机械与航空航天工程系，加州大学洛杉矶分校）
Learning Autonomy: Off-Road Navigation Enhanced by Human Input
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University（J. Mike Walker ’66 机械工程系，德克萨斯A&M大学）；Department of Engineering technology and Industrial Distribution, Texas A&M University（工程技术与工业分销系，德克萨斯A&M大学）
This paper describes a system whereby a robot detects and track human-meaningful navigational cues as it navigates in an indoor environment. It is intended as the sensor front-end for a mobile robot system that can communicate its navigational context with human users. From simulated LiDAR scan data we construct a set of 2D occupancy grid bitmaps, then hand-label these with human-scale navigational features such as closed doors, open corridors and intersections. We train a Convolutional Neural Network (CNN) to recognize these features on input bitmaps. In our demonstration system, these featur…
Army Research Laboratory（陆军研究实验室）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Existing autonomous robot navigation systems allow robots to move from one point to another in a collision-free manner. However, when facing new environments, these systems generally require re-tuning by expert roboticists with a good understanding of the inner workings of the navigation system. In contrast, even users who are unversed in the details of robot navigation algorithms can generate desirable navigation behavior in new environments via teleoperation. In this paper, we introduce APPLD, Adaptive Planner Parameter Learning from Demonstration, that allows existing navigation systems to …
City University of Hong Kong（香港城市大学）；HKUST（香港科技大学）；ETH Zurich（苏黎世联邦理工学院）
We present a learning-based mapless motion planner by taking the sparse 10-dimensional range findings and the target position with respect to the mobile robot coordinate frame as input and the continuous steering commands as output. Traditional motion planners for mobile ground robots with a laser range sensor mostly depend on the obstacle map of the navigation environment where both the highly precise laser sensor and the obstacle map building work of the environment are indispensable. We show that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be tr…
Zhejiang University（浙江大学）；Netease Fuxi Robot Department（网易伏羲机器人部）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhejiang University of Technology（浙江工业大学）；University of Shanghai for Science and Technology（上海理工大学）
Reliable navigation systems have a wide range of applications in robotics and autonomous driving. Current approaches employ an open-loop process that converts sensor inputs directly into actions. However, these open-loop schemes are challenging to handle complex and dynamic real-world scenarios due to their poor generalization. Imitating human navigation, we add a reasoning process to convert actions back to internal latent states, forming a two-stage closed loop of perception, decision-making, and reasoning. Firstly, VAE-Enhanced Demonstration Learning endows the model with the understanding …
University of Bonn（波恩大学）；Fraunhofer FKIE（弗劳恩霍夫通信、信息处理与人机工程研究所）
For the most comfortable, human-aware robot navigation, subjective user preferences need to be taken into account. This paper presents a novel reinforcement learning framework to train a personalized navigation controller along with an intuitive virtual reality demonstration interface. The conducted user study provides evidence that our personalized approach significantly outperforms classical approaches with more comfortable human-robot experiences. We achieve these results using only a few demonstration trajectories from non-expert users, who predominantly appreciate the intuitive demonstrat…
ETH Zurich（苏黎世联邦理工学院）；Max Planck ETH Center for Learning Systems（马克斯·普朗克-苏黎世联邦理工学习系统中心）
This work presents a case study of a learning-based approach for target driven map-less navigation. The underlying navigation model is an end-to-end neural network which is trained using a combination of expert demonstrations, imitation learning (IL) and reinforcement learning (RL). While RL and IL suffer from a large sample complexity and the distribution mismatch problem, respectively, we show that leveraging prior expert demonstrations for pre-training can reduce the training time to reach at least the same level of performance compared to plain RL by a factor of 5. We present a thorough ev…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Google（谷歌）
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human robot coexistence. To address this challenge, imitation learning is a promising framework, since …
CeDR-Nav: Cross-Embodiment Mapless Navigation via Offline Demonstration Recasting
The Hong Kong Polytechnic University（香港理工大学）；Eastern Institute of Technology（宁波工程学院）；University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）
University of California, Berkeley（加州大学伯克利分校）；Robotics at Google（谷歌机器人学）
Imitation learning is a popular approach for training visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for a long time. Here, we propose a zero-shot imitation learning approach for training a visual navigation policy on legged robots from human (third-person perspective) demonstrations, enabling high-quality navigation and cost-effective data collection. However, imitation learning from third-person demonstrations raises unique challenges. First, these demo…
Enhanced Robotic Navigation in Deformable Environments using Learning from Demonstration and Dynamic Modulation
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany（慕尼黑机器人与机器智能研究所（MIRMI），慕尼黑技术大学（TUM））；Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE（穆罕默德·本·扎耶德人工智能大学）
University of Massachusetts Dartmouth（马萨诸塞大学达特茅斯分校）；United States Army Research Laboratory（美国陆军研究实验室）
Traditional imitation learning provides a set of methods and algorithms to learn a reward function or policy from expert demonstrations. Learning from demonstration has been shown to be advantageous for navigation tasks as it allows for machine learning non-experts to quickly provide information needed to learn complex traversal behaviors. However, a minimal set of demonstrations is unlikely to capture all relevant information needed to achieve the desired behavior in every possible future operational environment. Due to distributional shift among environments, a robot may encounter features t…
University of California, San Diego（加利福尼亚大学圣地亚哥分校）
This paper focuses on inverse reinforcement learning (IRL) to enable safe and efficient autonomous navigation in unknown partially observable environments. The objective is to infer a cost function that explains expert-demonstrated navigation behavior while relying only on the observations and state-control trajectory used by the expert. We develop a cost function representation composed of two parts: a probabilistic occupancy encoder, with recurrent dependence on the observation sequence, and a cost encoder, defined over the occupancy features. The representation parameters are optimized by d…
This paper focuses on inverse reinforcement learning (IRL) for autonomous robot navigation using semantic observations. The objective is to infer a cost function that explains demonstrated behavior while relying only on the expert's observations and state-control trajectory. We develop a map encoder, which infers semantic class probabilities from the observation sequence, and a cost encoder, defined as deep neural network over the semantic features. Since the expert cost is not directly observable, the representation parameters can only be optimized by differentiating the error between demonst…
Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning
The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
Learning Social Navigation from Positive and Negative Demonstrations and Rule-Based Specifications
Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Artificial Intelligence, Yonsei University（人工智能系，yonsei大学）；Robotics Institute, School of Computer Science at Carnegie Mellon University（机器人研究所，卡内基梅隆大学计算机科学学院）；Mobinn；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Statistics（统计学系）
Learning from Demonstration with Failure Awareness for Safe Robot Navigation
Department of Mechanical Engineering, The Hong Kong Polytechnic University（香港理工大学机械工程系）；College of Information Science and Technology, Eastern Institute of Technology（东华理工大学信息科学学院）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（香港理工大学航空与航空工程系）
Co-training with Ego-centric Video and Demonstration for Robot Navigation Task
Department of Informatics, Graduate School of Informatics, Kyoto University（信息学系，京都大学研究生院）；Spatial Robotics Research Center, Fujitsu Limited（空间机器人研究中心，富士通有限公司）
Visual Servoing has been effectively used to move a robot into specific target locations or to track a recorded demonstration. It does not require manual programming, but it is typically limited to settings where one demonstration maps to one environment state. We propose a modular approach to extend visual servoing to scenarios with multiple demonstration sequences. We call this conditional servoing, as we choose the next demonstration conditioned on the observation of the robot. This method presents an appealing strategy to tackle multi-step problems, as individual demonstrations can be comb…
DreamFlow: Local Navigation Beyond Observation via Conditional Flow Matching in the Latent Space
Robotics Program, KAIST（韩国科学技术院机器人学系）；School of Electrical Engineering, KAIST（韩国科学技术院电子工程学院）；URobotics（URobotics公司）；KRAFTON（KRAFTON公司）；Dept. of Electrical Engineering and Computer Science, UC Berkeley（伯克利大学电子工程与计算机科学系）
IntentNav: Learning Spatial-Visual Object Navigation from Human Demonstrations
Nanyang Technological University（南洋理工大学）；Carnegie Mellon University（卡内基梅隆大学）；The Chinese University of Hong Kong（香港中文大学）；A*STAR Institute for Infocomm Research (I2R)（新加坡科技研究局资讯通信研究院）
US Army Research Laboratory（美国陆军研究实验室）；Carnegie Mellon University（卡内基梅隆大学）
When working alongside human collaborators in dynamic and unstructured environments, such as disaster recovery or military operation, fast field adaptation is necessary for an unmanned ground vehicle (UGV) to perform its duties or learn novel tasks. In these scenarios, personnel and equipment are constrained, making training with minimal human supervision a desirable learning attribute. We address the problem of making UGVs more reliable and adaptable teammates with a novel framework that uses visual perception and inverse optimal control to learn traversal costs for environment features. Thro…
In recent years, great efforts have been devoted to deep imitation learning for autonomous driving control, where raw sensory inputs are directly mapped to control actions. However, navigating through densely populated intersections remains a challenging task due to uncertainty caused by uncertain traffic participants. We focus on autonomous navigation at crowded intersections that require interaction with pedestrians. A multi-task conditional imitation learning framework is proposed to adapt both lateral and longitudinal control tasks for safe and efficient interaction. A new benchmark called…
Bio-inspired vehicles are currently leading the way in the quest to produce a vehicle capable of flight and underwater navigation. However, a fully functional vehicle has not yet been realized. We present the first fully functional vehicle platform operating in air and underwater with seamless transition between both mediums. These unique capabilities combined with the hovering, high maneuverability and reliability of multirotor vehicles, results in a disruptive technology for both civil and military application including air/water search and rescue, inspection, repairs and survey missions amo…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Sony AI（索尼人工智能）
While imitation learning for vision based autonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or worse yet the demonstrator does not have access to the platform at all? Is imitation learning for vision based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent …
Aalto University（阿尔托大学）；The University of Queensland（昆士兰大学）；Queensland Brain Institute（昆士兰脑研究所）；Queensland University of Technology（昆士兰科技大学）；University of Oulu（奥卢大学）
With the advent of consumer-grade products for presenting an immersive virtual environment (VE), there is a growing interest in utilizing VEs for testing human navigation behavior. However, preparing a VE still requires a high level of technical expertise in computer graphics and virtual reality, posing a significant hurdle to embracing the emerging technology. To address this issue, this paper presents Delayed Feedback based Immersive Navigation Environment (DeFINE), a framework that allows for easy creation and administration of navigation tasks within customizable VEs via intuitive graphica…
Johns Hopkins University（约翰斯·霍普金斯大学）；Wilmer Eye Institute at the Johns Hopkins Hospital（约翰斯·霍普金斯医院威尔默眼科研究所）
A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina in order to perform the tool-navigation task, which can be prone to human error. To alleviate such uncertainty, prior work has introduced ways to assist the surgeon by estimating the tool-tip distance to the retina and providing haptic or auditory fee…
SGN-CIRL: Scene Graph-based Navigation with Curriculum, Imitation, and Reinforcement Learning
Moscow Institute of Physics and Technology (MIPT)（莫斯科物理技术学院）；Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心）；Artificial Intelligence Research Institute (AIRI)（人工智能研究院）
Crowd-FM: Learned Optimal Selection of Conditional Flow Matching-generated Trajectories for Crowd Navigation
Robotics Research Center, IIIT Hyderabad, India（IIIT海得拉巴机器人研究中心，印度）；Nanyang Technological University, Singapore（新加坡南洋理工大学）；University of Tartu, Estonia（爱沙尼亚塔尔图大学）；Inria, Université de Lorraine, France（法国里尔大学Inria研究院）
LDP: A Local Diffusion Planner for Efficient Robot Navigation and Collision Avoidance
Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）；School of Data Science, USTC（中国科学技术大学数据科学学院）；School of Computer Science and Technology, USTC（中国科学技术大学计算机科学与技术学院）
