Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
Defense Innovation Institute, Chinese Academy of Military Sciences(军事科学院国防创新研究院);Intelligent Game and Decision Laboratory(智能博弈与决策实验室);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院)
Xingyu Xia, Lekai Zhou, Yujie Tang, Xiaozhou Zhu, Hai Zhu, Wen Yao
Aerial vision-and-language navigation (Aerial VLN) aims to enable unmanned aerial vehicles (UAVs) to interpret natural language instructions and autonomously navigate complex three-dimensional environments by grounding language in visual perception. This survey provides a critical and analytical review of the Aerial VLN field, with particular attention to the recent integration of large language models (LLMs) and vision-language models (VLMs). We first formally introduce the Aerial VLN problem and define two interaction paradigms: single-instruction and dialog-based, as foundational axes. We then organize the body of Aerial VLN methods into a taxonomy of five architectural categories: sequence-to-sequence and attention-based methods, end-to-end LLM/VLM methods, hierarchical methods, multi-agent methods, and dialog-based navigation methods. For each category, we systematically analyze design rationales, technical trade-offs, and reported performance. We critically assess the evaluation infrastructure for Aerial VLN, including datasets, simulation platforms, and metrics, and identify their gaps in scale, environmental diversity, real-world grounding, and metric coverage. We consolidate cross-method comparisons on shared benchmarks and analyze key architectural trade-offs, including discrete versus continuous actions, end-to-end versus hierarchical designs, and the simulation-to-reality gap. Finally, we synthesize seven concrete open problems: long-horizon instruction grounding, viewpoint robustness, scalable spatial representation, continuous 6-DoF action execution, onboard deployment, benchmark standardization, and multi-UAV swarm navigation, with specific research directions grounded in the evidence presented throughout the survey.
School of Automation Science and Engineering, South China University of Technology(华南理工大学自动化科学与工程学院);State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所机器人重点实验室);State Key Laboratory of Robotics and System, School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)机械工程与自动化学院机器人与系统重点实验室)
ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning
Pengcheng Laboratory(鹏城实验室);School of Computer Science and Cyber Engineering(计算机科学与网络工程学院);Guangzhou University(广州大学);Southern University of Science and Technology(南方科技大学)
Deep Learning-based Robust Autonomous Navigation of Aerial Robots in Dense Forests
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland(遥感与摄影测量系,芬兰地理空间研究 institute FGI,芬兰国家土地测绘局)
ViVa-SAFELAND: a New Freeware for Safe Validation of Vision-based Navigation in Aerial Vehicles
Center for Research in Mathematics CIMAT(数学研究中心(CIMAT));Centro de Investigación y de Estudios Avanzados del IPN, CINVESTAV-IPN(墨西哥理工学院高级研究中心(CINVESTAV-IPN))
OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency
School of Artificial Intelligence, Sun Yat-Sen University(中山大学人工智能学院);Southern University of Science and Technology(南方科技大学);The Hong Kong University of Science and Technology(香港科技大学)
Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation
National Key Laboratory of Big Data and Decision(大数据与决策国家重点实验室);National University of Defense Technology(国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);Information Support Force Engineering University(信息支援力量工程大学)
Openfly: A comprehensive platform for aerial vision-language navigation
Shanghai AI Laboratory(上海人工智能实验室);Northwestern Polytechnical University(西北工业大学);Beihang University(北航);Shanghai Jiao Tong University(上海交通大学);The University of Hong Kong(香港大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);East China University of Science and Technology(东华大学);Fudan University(复旦大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);TeleAI
School of Electrical Engineering and Computer Science, The University of Queensland(昆士兰大学电气工程与计算机科学学院);CSIRO Robotics, Data61(联邦科学与工业研究组织机器人学与Data61研究院)
RINGO: Real-time Navigation with a Guiding Trajectory for Aerial Manipulators in Unknown Environments
Institute of Robotics and Automatic Information System(机器人与自动信息系统研究所);College of Artificial Intelligence(人工智能学院);Tianjin Key Laboratory of Intelligent Robotics(天津市智能机器人重点实验室);Nankai University(南开大学);Institute of Intelligence Technology and Robotic Systems(智能技术与机器人系统研究所);Shenzhen Research Institute of Nankai University(南开大学深圳研究院)
GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigation
College of Systems Engineering, National University of Defense Technology(系统工程学院,国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);BNRist, Tsinghua University(清华大学BNRist)
Agile in the Face of Delay: Asynchronous End-to-End Learning for Real-World Aerial Navigation
School of Intelligence and Engineering(智能与工程学院);Guangdong Provincial Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics(广东省智能变形机制与自适应机器人重点实验室);Harbin Institute of Technology(哈尔滨工业大学)
Vision-based Navigation of Unmanned Aerial Vehicles in Orchards: An Imitation Learning Approach
Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, USA(生物与农业工程系,加州大学戴维斯分校);Department of Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA, USA(机械与航空航天工程系,加州大学戴维斯分校)
Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Deep Reinforcement Learning Through Environmental Generalization
Universidad Tecnológica del Uruguay (UTEC)(乌拉圭技术大学);Centro de Ciências Computacionais (C3) of Universidade Federal do Rio Grande (FURG)(里奥格兰德联邦大学计算科学中心);Universidade Federal de Santa Maria (UFSM)(圣玛丽亚联邦大学)