We propose a multi-head attention mechanism as a blending layer in a neural network model that translates natural language to a high level behavioral language for indoor robot navigation. We follow the framework established by (Zang et al., 2018a) that proposes the use of a navigation graph as a knowledge base for the task. Our results show significant performance gains when translating instructions on previously unseen environments, therefore, improving the generalization capabilities of the model.
Stanford University（斯坦福大学）；Pontificia Universidad Católica de Chile（智利天主教大学）；Yale University（耶鲁大学）
Inspired by research in psychology, we introduce a behavioral approach for visual navigation using topological maps. Our goal is to enable a robot to navigate from one location to another, relying only on its visual input and the topological map of the environment. We propose using graph neural networks for localizing the agent in the map, and decompose the action space into primitive behaviors implemented as convolutional or recurrent neural networks. Using the Gibson simulator, we verify that our approach outperforms relevant baselines and is able to navigate in both seen and unseen environm…
Pontificia Universidad Catolica de Chile（智利天主教大学）；Stanford University（斯坦福大学）
We present a semantically rich graph representation for indoor robotic navigation. Our graph representation encodes: semantic locations such as offices or corridors as nodes, and navigational behaviors such as enter office or cross a corridor as edges. In particular, our navigational behaviors operate directly from visual inputs to produce motor controls and are implemented with deep learning architectures. This enables the robot to avoid explicit computation of its precise location or the geometry of the environment, and enables navigation at a higher level of semantic abstraction. We evaluat…
NORM-Nav: Zero-Shot Mobile Robot Navigation with Natural Language Behavioral Constraints
College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Institute of Systems Engineering and Collaborative Laboratory for Intelligent Science and Systems, Macau University of Science and Technology（澳门大学系统工程与智能科学与系统联合实验室）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）
Learning visuomotor control policies in robotic systems is a fundamental problem when aiming for long-term behavioral autonomy. Recent supervised-learning-based vision and motion perception systems, however, are often separately built with limited capabilities, while being restricted to few behavioral skills such as passive visual odometry (VO) or mobile robot visual localization. Here we propose an approach to unify those successful robot perception systems for active target-driven navigation tasks via reinforcement learning (RL). Our method temporally incorporates compact motion and visual p…
EnQuery: Ensemble Policies for Diverse Query-Generation in Preference Alignment of Robot Navigation
University of Bonn（波恩大学）；Center for Robotics at the University of Bonn（波恩大学机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Politecnico di Torino（都灵理工大学）；Boston University（波士顿大学）
Drawing inspiration from biology, we describe the way in which visual sensing with a monocular camera can provide a reliable signal for navigation of mobile robots. The work takes inspiration from a classic paper by Lee and Reddish (Nature, 1981, https://doi.org/10.1038/293293a0) in which they outline a behavioral strategy pursued by diving sea birds based on a visual cue called time-to-contact. A closely related concept of time-to-transit, tau, is defined, and it is shown that idealized steering laws based on monocular camera perceptions of tau can reliably and robustly steer a mobile vehicle…
Learning Robot Visual Navigation in Crowds via Intention-Aware Scene Representations
Shenzhen Key Laboratory of Robotics Perception and Intelligence, Department of Electronic and Electrical Engineering, SUSTech（南方科技大学电子与电气工程系深圳市机器人感知与智能重点实验室）；Jiaxing Research Institute, SUSTech（南方科技大学嘉兴研究院）；Research Institute of MA&EI, Peng Cheng Laboratory（鹏城实验室机器人与人工智能研究所）
When the navigational environment is known, it can be represented as a graph where landmarks are nodes, the robot behaviors that move from node to node are edges, and the route is a set of behavioral instructions. The route path from source to destination can be viewed as a class of combinatorial optimization problems where the path is a sequential subset from a set of discrete items. The pointer network is an attention-based recurrent network that is suitable for such a task. In this paper, we utilize a modified R-NET with gated attention and self-matching attention translating natural langua…
Unified Multimodal Vessel Trajectory Prediction with Explainable Navigation Intention
School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机科学与人工智能学院）；School of Navigation, Wuhan University of Technology（武汉理工大学航海学院）；Hubei Key Laboratory of Internet of Intelligence, School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院智能互联网关键实验室）；College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）
Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation
Hochschule Bonn-Rhein-Sieg, Germany（波恩-莱茵-锡格应用科学大学，德国）；University of Bonn, Germany（波恩大学，德国）；Lamarr Institute for Machine Learning and Artificial Intelligence, Germany（拉马尔机器学习与人工智能研究所，德国）
Nagoya University（名古屋大学）；RIKEN Center for Advanced Intelligence Project（日本理化学研究所先进智能项目）；Doshisha University（同志社大学）；Tokai University（东海大学）；Kyushu University（九州大学）
Extracting the interaction rules of biological agents from movement sequences pose challenges in various domains. Granger causality is a practical framework for analyzing the interactions from observed time-series data; however, this framework ignores the structures and assumptions of the generative process in animal behaviors, which may lead to interpretational problems and sometimes erroneous assessments of causality. In this paper, we propose a new framework for learning Granger causality from multi-animal trajectories via augmented theory-based behavioral models with interpretable data-dri…
Using touch devices to navigate in virtual 3D environments such as computer assisted design (CAD) models or geographical information systems (GIS) is inherently difficult for humans, as the 3D operations have to be performed by the user on a 2D touch surface. This ill-posed problem is classically solved with a fixed and handcrafted interaction protocol, which must be learned by the user. We propose to automatically learn a new interaction protocol allowing to map a 2D user input to 3D actions in virtual environments using reinforcement learning (RL). A fundamental problem of RL methods is the …
Johns Hopkins University/Applied Physics Laboratory（约翰斯·霍普金斯大学/应用物理实验室）；Kavli Neuroscience Discovery Institute, Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Dynamical systems models for controlling multi-agent swarms have demonstrated advances toward resilient, decentralized navigation algorithms. We previously introduced the NeuroSwarms controller, in which agent-based interactions were modeled by analogy to neuronal network interactions, including attractor dynamics and phase synchrony, that have been theorized to operate within hippocampal place-cell circuits in navigating rodents. This complexity precludes linear analyses of stability, controllability, and performance typically used to study conventional swarm models. Further, tuning dynamical…
Modeling multi-agent systems requires understanding how agents interact. Such systems are often difficult to model because they can involve a variety of types of interactions that layer together to drive rich social behavioral dynamics. Here we introduce a method for accurately modeling multi-agent systems. We present Interaction Modeling with Multiplex Attention (IMMA), a forward prediction model that uses a multiplex latent graph to represent multiple independent types of interactions and attention to account for relations of different strengths. We also introduce Progressive Layer Training,…
