VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation
Polytechnic of Turin(都灵理工大学);Simon Fraser University(Simon Fraser大学);University of Verona(威尼斯大学);University of Reykjavik(雷克雅未克大学);University of California, Irvine(加州大学尔湾分校)
Francesco Taioli, Shiping Yang, Sonia Raychaudhuri, Marco Cristani, Unnat Jain, Angel X Chang
Language-driven object navigation requires agents to interpret natural language descriptions of target objects, which combine intrinsic and extrinsic attributes for instance recognition and commonsense navigation. Existing methods either (i) use end-to-end trained models with vision-language embeddings, which struggle to generalize beyond training data and lack action-level explainability, or (ii) rely on modular zero-shot pipelines with large language models (LLMs) and open-set object detectors, which suffer from error propagation, high computational cost, and difficulty integrating their reasoning back into the navigation policy. To this end, we propose a compact 3B-parameter Vision-Language-Action (VLA) agent that performs human-like embodied reasoning for both object recognition and action selection, removing the need for stitched multi-model pipelines. Instead of raw embedding matching, our agent employs explicit image-grounded reasoning to directly answer "Is this the target object?" and "Why should I take this action?" The reasoning process unfolds in three stages: "think", "think summary", and "action", yielding improved explainability, stronger generalization, and more efficient navigation. Code and dataset available upon acceptance.
ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation
Tianmushan Laboratory, Beihang University(北航之梦实验室,北京航空航天大学);Hangzhou International Innovation Institute, Beihang University(杭州国际创新研究院,北京航空航天大学);Foshan Graduate School of Innovation, Northeastern University(佛山创新研究生学院,东北大学);School of Aeronautic Science and Engineering, Beihang University(航空科学与工程学院,北京航空航天大学);Faculty of Robot Science and Engineering, Northeastern University(机器人科学与工程学院,东北大学)
Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledge
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China(机器人系统国家重点实验室,哈尔滨工业大学,哈尔滨);School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore(机械与航空航天工程学院,南洋理工大学,新加坡)
EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning
Shanghai Jiao Tong University(上海交通大学);Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);Peng Cheng Laboratory(鹏城实验室);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Mohamed Bin Zayed University of Artificial Intelligence(Mohamed Bin Zayed人工智能大学);Yinwang Intelligent Technology Co.(亿维智能技术有限公司)
MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chains
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区智能系统工程学院);School of Computer Science and Engineering(计算机科学与工程学院)
针对基于LLM的VLN方法记忆构建不足、策略单一的问题,本文提出MC-GPT,通过拓扑记忆地图、Navigation Chain of Thoughts及集成流水线增强导航与推理可解释性。
NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models
AIML, University of Adelaide(阿德莱德大学AIML);Adobe Research(奥多比研究院);University of North Carolina, Chapel Hill(北卡罗来纳大学教堂山分校);University of California, Santa Cruz(加利福尼亚大学圣克鲁兹分校)
Uni. of Bristol(布里斯托大学);Uni. of Michigan(密歇根大学);Uni. of Toronto(多伦多大学)
We introduce VISOR, a new dataset of pixel annotations and a benchmark suite for segmenting hands and active objects in egocentric video. VISOR annotates videos from EPIC-KITCHENS, which comes with a new set of challenges not encountered in current video segmentation datasets. Specifically, we need to ensure both short- and long-term consistency of pixel-level annotations as objects undergo transformative interactions, e.g. an onion is peeled, diced and cooked - where we aim to obtain accurate pixel-level annotations of the peel, onion pieces, chopping board, knife, pan, as well as the acting …
Zero-shot Object Navigation with Vision-Language Models Reasoning
New York University Abu Dhabi(纽约大学阿布扎比分校);Center for Artificial Intelligence and Robotics(人工智能与机器人中心);Tandon School of Engineering(坦登工程学院);China Academic of Electronics and Information Technology(中国电子信息产业发展研究院);School of Software, Tsinghua University(清华大学软件学院)
NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning
Sun Yat-sen University(中山大学);Shanghai Jiao Tong University(上海交通大学);Peng Cheng Laboratory(鹏城实验室);University of Hong Kong(香港大学);Dataa Robotics(达闼机器人公司);University of Science and Technology of China(中国科学技术大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning
University of Science and Technology of China(中国科学技术大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);Stanford University(斯坦福大学);Amap, Alibaba Group(阿里巴巴集团高德地图);Shanghai AI Laboratory(上海人工智能实验室)
CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);Tongji University(同济大学);Nanjing Institute of Geography and Limnology, Chinese Academy of Sciences(中国科学院南京地理与湖泊研究所);Research Centre for Data Science & Artificial Intelligence(数据科学与人工智能研究中心)
The Sword, Shield, and Achilles' Heel: Characterizing the Linguistic Inductive Bias of Large Language Models for Spatial Reasoning in Navigation Planning
East China Normal University(东华大学);Information Engineering University(信息工程大学);Zhengzhou University(郑州大学)
Autonomous Navigation of Cloud-Controlled Quadcopters in Confined Spaces Using Multi-Modal Perception and LLM-Driven High Semantic Reasoning
Department of Mechanical Engineering(机械工程系);Rajshahi University of Engineering and Technology(拉贾沙希工程与技术大学);Department of Industrial and Production Engineering(工业与生产工程系);Shahjalal University of Science and Technology(沙赫jalal科学与技术大学);Bangladesh University of Engineering and Technology(孟加拉工程与技术大学);Department of Urban and Regional Planning(城市与区域规划系);Department of Computer Science Engineering(计算机科学与工程系);United International University(联合国际大学)
GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models
Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Tongji University(同济大学);The University of Hong Kong(香港大学)
HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models
School of Computer Science and Technology, Xinjiang University(新疆大学计算机科学与技术学院);Joint Research Laboratory for Embodied Intelligence, Xinjiang University(新疆大学具身智能联合研究实验室);Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University(新疆大学丝绸之路多语言认知计算国际联合研究实验室)
Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation
City University of Hong Kong, Dongguan Campus(香港城市大学东莞校区);The University of Melbourne(墨尔本大学);Tsinghua University(清华大学);Baidu Inc.(百度公司);University of Chinese Academy of Science(中国科学院大学);National University of Singapore(新加坡国立大学)
Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigation
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Department of Computer Science, Emory University(埃默里大学计算机科学系);Department of Computer Science, University of Wisconsin-Madison(威斯康星大学麦迪逊分校计算机科学系);Tsinghua University(清华大学);Innovation and Research and Development Department, BoardWare Information System Company(BoardWare信息系统公司创新与研发部);School of Mathematics, Peking University(北京大学数学学院)
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and dataset, where an agent must first follow navigation instructions in a real-life visual urban environment, and then identify a location described in natural language to find a hidden object at the goal position. The data contains 9,326 examples of English instructions and spatial descriptions paired with demonstrations. Empirical analysis shows the data presents an open challenge to existing methods, and qualitative linguistic analysis shows t…
Monash University(莫纳什大学);University of Chinese Academy of Sciences(中国科学院大学);Sun Yat-sen University(中山大学);Dark Matter AI Inc.(暗物质人工智能公司)
Vision-Language Navigation (VLN) is a task where agents learn to navigate following natural language instructions. The key to this task is to perceive both the visual scene and natural language sequentially. Conventional approaches exploit the vision and language features in cross-modal grounding. However, the VLN task remains challenging, since previous works have neglected the rich semantic information contained in the environment (such as implicit navigation graphs or sub-trajectory semantics). In this paper, we introduce Auxiliary Reasoning Navigation (AuxRN), a framework with four self-su…
CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigation
Purdue University(普渡大学);Indian Institute of Technology, Kharagpur(印度理工学院卡哈拉格普尔分校);DEVCOM Army Research Lab(DEVCOM陆军研究实验室);National University of Singapore(新加坡国立大学)
Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning
Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education)(电磁波信息科学重点实验室(教育部));School of Information Science and Technology(信息科学与技术学院);Fudan University(复旦大学)