Landmarks are central to how people navigate, but most navigation technologies do not incorporate them into their representations. We propose the landmark graph generation task (creating landmark-based spatial representations from natural language) and introduce a fully end-to-end neural approach to generate these graphs. We evaluate our models on the SAIL route instruction dataset, as well as on a small set of real-world delivery instructions that we collected, and we show that our approach yields high quality results on both our task and the related robotic navigation task.
Controllable Navigation Instruction Generation with Chain of Thought Prompting
School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
Landmark-Guided Knowledge for Vision-and-Language Navigation
School of Computer Science and Technology, Xinjiang University, Urumqi, China(计算机科学与技术学院,新疆大学,乌鲁木齐,中国);No. 59 Middle School of Urumqi, Urumqi, China(乌鲁木齐第五十九中学,乌鲁木齐,中国)
Correctable Landmark Discovery via Large Models for Vision-Language Navigation
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);PengCheng Laboratory(鹏城实验室);Dataa Robotics(达闼机器人公司);Shenzhen Institute of Advanced Technology(深圳先进技术研究院)
Dual-Anchoring: Addressing State Drift in Vision-Language Navigation
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence(人机混合增强智能国家重点实验室);National Engineering Research Center for Visual Information and Applications(视觉信息与应用国家工程研究中心);Institute of Artificial Intelligence and Robotics(人工智能与机器人研究院);Xi’an Jiaotong University(西安交通大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Johns Hopkins University(约翰霍普金斯大学);Joy Future Academy, JD(京东探索研究院)
From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)
Visuospatial navigation from the bottom-up: without vestibular integration, distance prediction, or maps
Institute for Theoretical Biology, Department of Biology, Humboldt Universität zu Berlin(理论生物学研究所,生物学系,洪堡柏林大学);Science of Intelligence, Research Cluster of Excellence, Berlin, Germany(智能科学,卓越研究集群,柏林,德国);Bernstein Center for Computational Neuroscience, Berlin, Germany(计算神经科学伯恩斯坦中心,柏林,德国)