Advances in Embodied Navigation Using Large Language Models: A Survey
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China（北京邮电大学人工智能学院）；Aerospace Information Research Institute, Chinese Academy of Science（中国科学院航空航天信息研究所）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, China（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)（山东省教育厅计算能力网络与信息安全重点实验室）
Jinzhou Lin, Han Gao, Xuxiang Feng, Rongtao Xu, Changwei Wang, Man Zhang, Li Guo, Shibiao Xu
备注：
Submited to IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS
In recent years, the rapid advancement of Large Language Models (LLMs) such as the Generative Pre-trained Transformer (GPT) has attracted increasing attention due to their potential in a variety of practical applications. The application of LLMs with Embodied Intelligence has emerged as a significant area of focus. Among the myriad applications of LLMs, navigation tasks are particularly noteworthy because they demand a deep understanding of the environment and quick, accurate decision-making. LLMs can augment embodied intelligence systems with sophisticated environmental perception and decision-making support, leveraging their robust language and image-processing capabilities. This article offers an exhaustive summary of the symbiosis between LLMs and embodied intelligence with a focus on navigation. It reviews state-of-the-art models, research methodologies, and assesses the advantages and disadvantages of existing embodied navigation models and datasets. Finally, the article elucidates the role of LLMs in embodied intelligence, based on current research, and forecasts future directions in the field. A comprehensive list of studies in this survey is available at https://github.com/Rongtao-Xu/Awesome-LLM-EN.
State Key Laboratory of Robotics and Intelligent Systems（机器人与智能系统国家重点实验室）；Shenyang Institute of Automation（沈阳自动化研究所）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Tongji University（同济大学）；Nanjing Institute of Geography and Limnology, Chinese Academy of Sciences（中国科学院南京地理与湖泊研究所）；Research Centre for Data Science & Artificial Intelligence（数据科学与人工智能研究中心）
NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）
EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning
Shanghai Jiao Tong University（上海交通大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Peng Cheng Laboratory（鹏城实验室）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Mohamed Bin Zayed University of Artificial Intelligence（Mohamed Bin Zayed人工智能大学）；Yinwang Intelligent Technology Co.（亿维智能技术有限公司）
How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspace
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Northeastern University（东北大学）；National University of Defense Technology（国防科技大学）；Shandong University（山东大学）；BNRist, Tsinghua University（清华大学北京信息科学与技术国家研究中心）
PM-Nav: Priori-Map Guided Embodied Navigation in Functional Buildings
Faculty of Robot Science and Engineering at Northeastern University（东北大学机器人科学与工程学院）；Foshan Graduate School of Innovation at Northeastern University（东北大学佛山创新研究生学院）；School of Aeronautic Science and Engineering at Beihang University（北航航空科学与工程学院）
A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI
Department of Computer Science, City University of Hong Kong（城市大学计算机科学系）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电子与电气工程学院）；Department of Informatics, Universität Hamburg（汉堡大学信息学院）
Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation
University of British Columbia（不列颠哥伦比亚大学）；National University of Singapore（新加坡国立大学）；University of Washington（华盛顿大学）；Centre for Frontier AI Research, A*STAR（新加坡科技研究局前沿人工智能研究中心）
Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigation
Southwestern University of Finance（西南财经大学）；Department of Informatics,Universitat Hamburg, Hamburg, Germany（汉堡大学信息学院）；University of Electronic Science（电子科技大学）
Meta AI Research（Meta人工智能研究院）；Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
像图像-目标导航这样的现实长期任务涉及探索和利用阶段。由于目标的图像，一个具体的代理必须探索发现目标，即，使用学习的先验知识进行有效搜索。一旦目标被发现，智能体必须准确地校准到目标的最后一英里的导航。与任何强大的系统一样，探索性目标发现和利用性最后一英里导航之间的切换可以更好地从错误中恢复。根据这些直观的导轨，我们提出了SLING，以提高现有的图像目标导航系统的性能。作为对现有方法的补充，我们专注于最后一英里导航，并利用神经描述符来利用问题的底层几何结构。通过简单但有效的开关，我们可以轻松地将SLING与启发式、强化学习和神经模块化策略联系起来。在标准化的图像目标导航基准（Hahn et al. 2021）上，我们提高了策略，场景和情节复杂性的性能，将最先进的成功率从45%提高到55%。除了真实感模拟，我们在三个物理场景中进行了真实的机器人实验，并发现这些改进可以很好地转移到真实环境中。
FloNa: Floor Plan Guided Embodied Visual Navigation
Beijing Institute of Technology（北京理工大学）；Yangtze Delta Region Academy of Beijing Institute of Technology（北京理工大学长三角研究院）；Beijing Racobit Electronic Information Technology Co., Ltd.（北京睿科比特电子信息技术有限公司）
NavTrust: Benchmarking Trustworthiness for Embodied Navigation
Trustworthy Autonomous Systems Laboratory at the University of California, Riverside（加州大学河滨分校可信自主系统实验室）；University of Michigan（密歇根大学）；Workday；University of Southern California（南加州大学）；Texas A&M University（德克萨斯A&M大学）；Lehigh University（莱斯大学）
Australian National University（澳大利亚国立大学）；Princeton University（普林斯顿大学）；Carnegie Mellon University（卡内基梅隆大学）；Intel Labs（英特尔实验室）；UC Berkeley（加州大学伯克利分校）；George Mason University（乔治梅森大学）；Facebook（脸书）；Allen Institute for AI（艾伦人工智能研究所）
pFedNavi: Structure-Aware Personalized Federated Vision-Language Navigation for Embodied AI
Openmind (Wuhu) Intelligent Robot Co., Ltd.（Openmind（芜湖）智能机器人有限公司）；Shanghai Key Laboratory of Scalable Computing and Systems（上海可扩展计算与系统重点实验室）；National Natural Science Foundation of China（国家自然科学基金委员会）；US National Science Foundation（美国国家科学基金会）
RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigation
MBZUAI（默罕默德·本·扎耶德人工智能大学）；Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区）；AAII, UTS（悉尼科技大学人工智能与创新研究所）；University of Science and Technology of China（中国科学技术大学）
