现有的大多数社交机器人导航技术要么利用手工制定的规则，要么借助人类演示，将机器人感知与符合社交规范的动作关联起来。然而，在将感知有效转化为符合社交规范的动作方面仍存在显著差距，这一过程本应如同人类在动态环境中自然开展推理一样。鉴于视觉语言模型（VLM）近期取得的成功，我们提出利用语言来弥补感知与具备社交意识的机器人动作之间类人推理的缺口。我们构建了一个视觉语言数据集——社交机器人可解释交互导航数据集（Social robot Navigation via Explainable Interactions, SNEI），该数据集基于非结构化拥挤公共空间中的2K段人机社交交互，包含4万条人工标注的视觉问答（VQA），涵盖感知、预测、思维链推理、动作及解释多个维度。我们使用SNEI微调了视觉语言模型Social-LLaVA，以验证该数据集的实际应用价值。在50组视觉问答的15项不同人类评分的平均值对比中，Social-LLaVA的表现优于GPT-4V、Gemini等当前最先进的模型。将Social-LLaVA部署在移动机器人上后，机器人可实现类人推理，这标志着通过语言推理实现动态公共空间中符合社交规范的机器人导航迈出了充满前景的一步。
英文摘要：
Most existing social robot navigation techniques either leverage hand-crafted rules or human demonstrations to connect robot perception to socially compliant actions. However, there remains a significant gap in effectively translating perception into socially compliant actions, much like how human reasoning naturally occurs in dynamic environments. Considering the recent success of Vision-Language Models (VLMs), we propose using language to bridge the gap in human-like reasoning between perception and socially aware robot actions. We create a vision-language dataset, Social robot Navigation via Explainable Interactions (SNEI), featuring 40K human-annotated Visual Question Answers (VQAs) based on 2K human-robot social interactions in unstructured, crowded public spaces, spanning perception, prediction, chain-of-thought reasoning, action, and explanation. We fine-tune a VLM, Social-LLaVA, using SNEI to demonstrate the practical application of our dataset. Social-LLaVA outperforms state-of-the-art models like GPT-4V and Gemini, based on the average of fifteen different human-judge scores across 50 VQA. Deployed onboard a mobile robot, Social-LLaVA enables human-like reasoning, marking a promising step toward socially compliant robot navigation in dynamic public spaces through language reasoning.
Artificial Intelligence Research Institute, Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心人工智能研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）
Exploration is an important step in autonomous navigation of robotic systems. In this paper we introduce a series of enhancements for exploration algorithms in order to use them with vision-based simultaneous localization and mapping (vSLAM) methods. We evaluate developed approaches in photo-realistic simulator in two modes: with ground-truth depths and neural network reconstructed depth maps as vSLAM input. We evaluate standard metrics in order to estimate exploration coverage.
Enhancing Glass Surface Reconstruction via Depth Prior for Robot Navigation
Shenzhen Key Laboratory of Robotics and Computer Vision（机器人与计算机视觉深圳重点实验室）；Southern University of Science and Technology（南方科技大学）；CKS Robotics Institute（CKS机器人研究院）
SlipNet: Enhancing Slip Cost Mapping for Autonomous Navigation on Heterogeneous and Deformable Terrains
Khalifa University Center for Autonomous and Robotic Systems (KUCARS)（哈利法大学自主与机器人系统中心）；Advanced Research and Innovation Center (ARIC)（先进研究与创新中心）；Khalifa University（哈利法大学）
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models
Shenzhen campus of Sun Yat-sen University（中山大学深圳校区）；Huawei（华为）；The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）
GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning
Beijing University of Posts and Telecommunications（北京邮电大学）；StepFun；Waseda University（早稻田大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Enhancing Vision-Language Navigation with Multimodal Event Knowledge from Real-World Indoor Tour Videos
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）；Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)（马尔代夫 bin Zayed 大学人工智能学院）；Hangzhou City University（杭州城市学院）
SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Southeast University（东南大学）；Ant Group, Alibaba（蚂蚁集团）
IntelliMove: Enhancing Robotic Planning with Semantic Mapping
S.A.S. euroDAO（S.A.S. 欧元道）；LIRMM（蒙彼利埃计算机科学、机器人学、微电子与微电子实验室）；CNRS（法国国家科学研究中心）；Université de Montpellier（蒙彼利埃大学）；Université de Picardie Jules Verne（儒勒·凡尔纳皮卡第大学）
DynaNav: Dynamic Feature and Layer Selection for Efficient Visual Navigation
College of Design and Engineering, National University of Singapore（设计与工程学院，新加坡国立大学）；PEAK-Lab, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）PEAK实验室）
Prior Does Matter: Visual Navigation via Denoising Diffusion Bridge Models
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能系统工程学院）
VISA: Group-wise Visual Token Selection and Aggregation via Graph Summarization for Efficient MLLMs Inference
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室，教育部，厦门大学）；Tencent Youtu Lab（腾讯优图实验室）
FSUNav: A Cerebrum-Cerebellum Architecture for Fast, Safe, and Universal Zero-Shot Goal-Oriented Navigation
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；College of Information Science and Technology, Eastern Institute of Technology（东方理工学院信息科学与技术学院）
