How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspace
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Northeastern University(东北大学);National University of Defense Technology(国防科技大学);Shandong University(山东大学);BNRist, Tsinghua University(清华大学北京信息科学与技术国家研究中心)
Large multimodal models (LMMs) show strong visual-linguistic reasoning but their capacity for spatial decision-making and action remains unclear. In this work, we investigate whether LMMs can achieve embodied spatial action like human through a challenging scenario: goal-oriented navigation in urban 3D spaces. We first spend over 500 hours constructing a dataset comprising 5,037 high-quality goal-oriented navigation samples, with an emphasis on 3D vertical actions and rich urban semantic information. Then, we comprehensively assess 17 representative models, including non-reasoning LMMs, reasoning LMMs, agent-based methods, and vision-language-action models. Experiments show that current LMMs exhibit emerging action capabilities, yet remain far from human-level performance. Furthermore, we reveal an intriguing phenomenon: navigation errors do not accumulate linearly but instead diverge rapidly from the destination after a critical decision bifurcation. The limitations of LMMs are investigated by analyzing their behavior at these critical decision bifurcations. Finally, we experimentally explore four promising directions for improvement: geometric perception, cross-view understanding, spatial imagination, and long-term memory. The project is available at: https://github.com/serenditipy-AC/Embodied-Navigation-Bench.
AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航空信息研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院);Key Laboratory of Target Cognition and Application Technology(TCAT), Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航空信息研究所目标认知与应用技术重点实验室)
CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);Tongji University(同济大学);Nanjing Institute of Geography and Limnology, Chinese Academy of Sciences(中国科学院南京地理与湖泊研究所);Research Centre for Data Science & Artificial Intelligence(数据科学与人工智能研究中心)
SimWorld-Robotics: Synthesizing Photorealistic and Dynamic Urban Environments for Multimodal Robot Navigation and Collaboration
University of Virginia(弗吉尼亚大学);UC San Diego(加州大学圣地亚哥分校);Johns Hopkins University(约翰霍普金斯大学);Carnegie Mellon University(卡内基梅隆大学);University of Michigan(密歇根大学)
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments
College of Systems Engineering, National University of Defense Technology(系统工程学院,国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);BNRist, Tsinghua University(清华大学北京研究院);Department of Electronic Engineering, Tsinghua University(电子工程系,清华大学)
CityX: Controllable Procedural Content Generation for Unbounded 3D Cities
University of Chinese Academy of Sciences(中国科学院大学);China University of Geosciences (Beijing)(中国地质大学(北京));Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);Shandong University(山东大学);University of Science and Technology Beijing(北京科技大学)
PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
Shanghai Jiao Tong University(上海交通大学);Beihang University(北京航空航天大学);The Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学);University of Cambridge(剑桥大学);Intelligent Game and Decision Lab (IGDL)(智能博弈与决策实验室)
Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning
Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education)(电磁波信息科学重点实验室(教育部));School of Information Science and Technology(信息科学与技术学院);Fudan University(复旦大学)