Navigating socially in human environments requires more than satisfying geometric constraints, as collision-free paths may still interfere with ongoing activities or conflict with social norms. Addressing this challenge calls for analyzing interactions between agents and incorporating common-sense reasoning into planning. This paper presents a social robot navigation framework that integrates geometric planning with contextual social reasoning. The system first extracts obstacles and human dynamics to generate geometrically feasible candidate paths, then leverages a fine-tuned vision-language model (VLM) to evaluate these paths, informed by contextually grounded social expectations, selecting a socially optimized path for the controller. This task-specific VLM distills social reasoning from large foundation models into a smaller and efficient model, allowing the framework to perform real-time adaptation in diverse human-robot interaction contexts. Experiments in four social navigation contexts demonstrate that our method achieves the best overall performance with the lowest personal space violation duration, the minimal pedestrian-facing time, and no social zone intrusions. Project page: https://path-etiquette.github.io
University of Freiburg(弗赖堡大学);Google Research(谷歌研究院);University of Technology Nuremberg(纽伦堡工业大学)
Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visual-language models pretrained on Internet-scale data (e.g., image captions). While this is useful for matching images to natural language descriptions of object goals, it remains disjoint from the process of mapping the environment, so that it lacks the spatial precision of classic geometric maps. To address this problem, we propose VLMaps, a spatial map representation that directly fuses pretrained visual-language features with a 3D reconstruction of the physical world. VLMaps can be a…
ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation
Dept. of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA(电气与计算机工程系,马里兰大学,College Park, MD, USA);Dept. of Computer Science, University of Maryland, College Park, MD, USA(计算机科学系,马里兰大学,College Park, MD, USA);James Clark School of Engineering, University of Maryland, College Park, MD, USA(James Clark工程学院,马里兰大学,College Park, MD, USA)
WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN
Institute of Artificial Intelligence, China Telecom(中国电信人工智能研究院);Zhejiang University(浙江大学);Tongji University(同济大学);Shanghai Jiao Tong University(上海交通大学)
AirHunt: Bridging VLM Semantics and Continuous Planning for Efficient Aerial Object Navigation
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Department of Mechanical and Energy Engineering, Southern University of Science and Technology(南方科技大学机械与能源工程系);Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);Department of Physics, Southern University of Science and Technology(南方科技大学物理系)
Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control
Graduate School of Information Science and Technology, Hokkaido University(弘前大学信息科学与技术研究生院);Graduate School of Information Science and Technology, The University of Tokyo(东京大学信息科学与技术研究生院)
ZeST: A VLM-based Zero-Shot Traversability Navigation for Unknown Environments
Field Robotics Engineering and Science Hub (FRESH), Illinois Autonomous Farm, University of Illinois at Urbana-Champaign (UIUC), IL(伊利诺伊大学厄巴纳-香槟分校);Mobile Robotics Group, São Carlos School of Engineering, University of São Paulo (EESC-USP), São Carlos, SP, Brazil(圣保罗大学)
PIGEON: VLM-Driven Object Navigation via Points of Interest Selection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院);Peking University(北京大学);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior
University of Technology Nuremberg(纽伦堡工业大学);ETH Zürich(苏黎世联邦理工学院);Inria, Ecole Normale Supérieure, CNRS, PSL Research University(法国国家信息与自动化研究所,巴黎高等师范学校,法国国家科学研究中心,巴黎文理研究大学)
CoINS: Counterfactual Interactive Navigation via Skill-Aware VLM
School of Advanced Manufacturing and Robotics, Peking University(北京大学先进制造与机器人学院);Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong(香港中文大学机械与自动化工程系);College of Design and Engineering, National University Of Singapore(新加坡国立大学设计与工程学院);Department of Computer Science, The University of Manchester(曼彻斯特大学计算机科学系)
NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
Peking University(北京大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);CASIA(中国科学院自动化研究所);University of Adelaide(阿德莱德大学);Australian National University(澳大利亚国立大学)