Recently, visual-language navigation (VLN) -- entailing robot agents to follow navigation instructions -- has shown great advance. However, existing literature put most emphasis on interpreting instructions into actions, only delivering "dumb" wayfinding agents. In this article, we devise LANA, a language-capable navigation agent which is able to not only execute human-written navigation commands, but also provide route descriptions to humans. This is achieved by simultaneously learning instruction following and generation with only one single model. More specifically, two encoders, respectively for route and language encoding, are built and shared by two decoders, respectively, for action prediction and instruction generation, so as to exploit cross-task knowledge and capture task-specific characteristics. Throughout pretraining and fine-tuning, both instruction following and generation are set as optimization objectives. We empirically verify that, compared with recent advanced task-specific solutions, LANA attains better performances on both instruction following and route description, with nearly half complexity. In addition, endowed with language generation capability, LANA can explain to humans its behaviors and assist human's wayfinding. This work is expected to foster future efforts towards building more trustworthy and socially-intelligent navigation robots.
Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Laboratory for Computational Sensing and Robotics, Johns Hopkins University（约翰斯·霍普金斯大学计算感知与机器人实验室）
Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments
New Laboratory of Pattern Recognition (NLPR)（模式识别新实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Robotics X, Tencent（腾讯Robotics X实验室）
Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation
University of New South Wales, Sydney, Australia（新南威尔士大学）；Macquarie University, Sydney, Australia（麦考瑞大学）；Data61, CSIRO, Sydney, Australia（Data61, CSIRO）；The School of Computer Science（计算机科学学院）
Neighbor-view Enhanced Model for Vision and Language Navigation
Center for Research on Intelligent Perception and Computing, Institution of Automation, Chinese Academy of Sciences（中国科学院自动化研究所智能感知与计算研究中心）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；University of Adelaide（阿德莱德大学）；Center for Excellence in Brain Science and Intelligence Technology (CEBSIT)（脑科学与智能技术卓越创新中心（CEBSIT））；Chinese Academy of Sciences, Artificial Intelligence Research (CAS-AIR)（中国科学院人工智能研究院（CAS-AIR））
GeoScaffold: Learning Compact Geometric Latents via Reconstruction for Efficient Vision-Language Navigation
Nanjing University（南京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；MAICRO（迈克罗）
