The ability to accurately locate and navigate to a specific object is a crucial capability for embodied agents that operate in the real world and interact with objects to complete tasks. Such object navigation tasks usually require large-scale training in visual environments with labeled objects, which generalizes poorly to novel objects in unknown environments. In this work, we present a novel zero-shot object navigation method, Exploration with Soft Commonsense constraints (ESC), that transfers commonsense knowledge in pre-trained models to open-world object navigation without any navigation experience nor any other training on the visual environments. First, ESC leverages a pre-trained vision and language model for open-world prompt-based grounding and a pre-trained commonsense language model for room and object reasoning. Then ESC converts commonsense knowledge into navigation actions by modeling it as soft logic predicates for efficient exploration. Extensive experiments on MP3D, HM3D, and RoboTHOR benchmarks show that our ESC method improves significantly over baselines, and achieves new state-of-the-art results for zero-shot object navigation (e.g., 288% relative Success Rate improvement than CoW on MP3D).
Correctable Landmark Discovery via Large Models for Vision-Language Navigation
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；PengCheng Laboratory（鹏城实验室）；Dataa Robotics（达闼机器人公司）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）
OpenObject-NAV: Open-Vocabulary Object-Oriented Navigation Based on Dynamic Carrier-Relationship Scene Graph
School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）；School of Mechanical Engineering, University of Nottingham Ningbo China（宁波诺丁汉大学机械工程学院）
Augmented Commonsense Knowledge for Remote Object Grounding
Australian Institute for Machine Learning (AIML), University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Australian National University（澳大利亚国立大学）；Macquarie University（麦考瑞大学）；Griffith University（格里菲斯大学）
VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation
Polytechnic of Turin（都灵理工大学）；Simon Fraser University（Simon Fraser大学）；University of Verona（威尼斯大学）；University of Reykjavik（雷克雅未克大学）；University of California, Irvine（加州大学尔湾分校）
iCORPP: Interleaved Commonsense Reasoning and Probabilistic Planning on Robots
The State University of New York at Binghamton（纽约州立大学宾汉姆顿分校）；Sony AI, Sony Corporation of America（索尼美国公司索尼人工智能部门）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledge
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China（机器人系统国家重点实验室，哈尔滨工业大学，哈尔滨）；School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore（机械与航空航天工程学院，南洋理工大学，新加坡）
AirDreamer: Generalist Drone Navigation with World Models
Institute for AI Industry Research, Tsinghua University, Beijing, China（人工智能产业研究院，清华大学，北京，中国）；Department of Electronic Engineering, Tsinghua University, Beijing, China（电子工程系，清华大学，北京，中国）；School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China（遥感与信息工程学院，武汉大学，武汉，中国）
