Navigation is an essential ability for mobile agents to be completely autonomous and able to perform complex actions. However, the problem of navigation for agents with limited (or no) perception of the world, or devoid of a fully defined motion model, has received little attention from research in AI and Robotics. One way to tackle this problem is to use guided navigation, in which other autonomous agents, endowed with perception, can combine their distinct viewpoints to infer the localisation and the appropriate commands to guide a sensory deprived agent through a particular path. Due to the limited knowledge about the physical and perceptual characteristics of the guided agent, this task should be conducted on a level of abstraction allowing the use of a generic motion model, and high-level commands, that can be applied by any type of autonomous agents, including humans. The main task considered in this work is, given a group of autonomous agents perceiving their common environment with their independent, egocentric and local vision sensors, the development and evaluation of algorithms capable of producing a set of high-level commands (involving qualitative directions: e.g. move left, go straight ahead) capable of guiding a sensory deprived robot to a goal location.
Natural Language-Driven Viewpoint Navigation for Volume Exploration via Semantic Block Representation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Sun Yat-sen University（中山大学）；Guangdong Province Key Laboratory of Computational Science（广东省计算科学重点实验室）；National Supercomputer Center in Guangzhou（广州国家超级计算机中心）
SRM Institute of Science and Technology（SRM科学技术学院）
This work presents a decentralized motion planning framework for addressing the task of multi-robot navigation using deep reinforcement learning. A custom simulator was developed in order to experimentally investigate the navigation problem of 4 cooperative non-holonomic robots sharing limited state information with each other in 3 different settings. The notion of decentralized motion planning with common and shared policy learning was adopted, which allowed robust training and testing of this approach in a stochastic environment since the agents were mutually independent and exhibited asynch…
Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environments
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；The University of Queensland（昆士兰大学）；Zhejiang University of Technology（浙江工业大学）
DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memory
SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University（上海交通大学巴黎精英技术研究所）；Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化系）；MoE Key Lab of Artificial Intelligence and AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能与AI研究所）
Cornell University（康奈尔大学）；Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
This paper presents a novel method for qualitative mapping of large scale spaces. The proposed framework makes use of a graphical representation of the world in order to build a map consisting of qualitative constraints on the geometric relationships between landmark triplets. A novel measurement method based on camera imagery is presented which extends previous work from the field of Qualitative Spatial Reasoning. Measurements are fused into the map using a deterministic, iterative graph update. A Branch-and-Bound approach is taken to solve a set of non-convex feasibility problems required fo…
Hierarchical Scoring with 3D Gaussian Splatting for Instance Image-Goal Navigation
NYUAD Center for Artificial Intelligence and Robotics (CAIR)（纽约大学阿布扎比分校人工智能与机器人中心）；New York University Abu Dhabi（纽约大学阿布扎比分校）；New York University（纽约大学）；Embodied AI and Robotics (AIR) Lab（具身人工智能与机器人实验室）；School of Software, Tsinghua University（清华大学软件学院）
Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation
National Key Laboratory of Human Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University（西安交通大学人机混合增强智能全国重点实验室）；School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）
We propose a meta-ability decoupling (MAD) paradigm, which brings together various object navigation methods in an architecture system, allowing them to mutually enhance each other and evolve together. Based on the MAD paradigm, we design a multiple thinking (MT) model that leverages distinct thinking to abstract various meta-abilities. Our method decouples meta-abilities from three aspects: input, encoding, and reward while employing the multiple thinking collaboration (MTC) module to promote mutual cooperation between thinking. MAD introduces a novel qualitative and quantitative interpretabi…
Handle Object Navigation as Weighted Traveling Repairman Problem
Centre for Advanced Robotics Technology Innovation (CARTIN), School of Electrical and Electronic Engineering, Nanyang Technological University（先进机器人技术创新中心（CARTIN）、电子与电气工程学院、南洋理工大学）
View Invariant Learning for Vision-Language Navigation in Continuous Environments
Department of Electrical and Computer Engineering, University of Waterloo（电气与计算机工程系，滑铁卢大学）；Department of Civil and Environmental Engineering, University of Waterloo（土木与环境工程系，滑铁卢大学）；Department of Electrical and Computer Engineering, Northwestern University（电气与计算机工程系，西北大学）
Technion - Israel Institute of Technology（以色列理工学院）
Simultaneous localization and mapping (SLAM) are essential in numerous robotics applications, such as autonomous navigation. Traditional SLAM approaches infer the metric state of the robot along with a metric map of the environment. While existing algorithms exhibit good results, they are still sensitive to measurement noise, sensor quality, and data association and are still computationally expensive. Alternatively, some navigation and mapping missions can be achieved using only qualitative geometric information, an approach known as qualitative spatial reasoning (QSR). We contribute a novel …
Funnel lane concept is a qualitative visual navigation method which helps robots to autonomously navigate by using a recorded video. A visual path is extracted from the video by extracting some keyframes from the video. The robot uses this visual path for its navigation. Funnel lane unlike some other methods does not make use of traditional calculations of Jacobians, homographies, fundamental matrices, or the focus of expansion, and does not require any camera calibration. However, funnel lane has some shortcomings. One problem is that funnel lane gives no information about the radius of rotat…
MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation
Fujian Key Laboratory of Urban Intelligent Sensing and Computing（福建智能感知与计算重点实验室）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（多媒体可信感知与高效计算重点实验室）；Beihang University（北航）；Nanyang Technological University（南洋理工大学）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Academy（中关村学院）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
A robot guide dog has compelling advantages over animal guide dogs for its cost-effectiveness, potential for mass production, and low maintenance burden. However, despite the long history of guide dog robot research, previous studies were conducted with little or no consideration of how the guide dog handler and the guide dog work as a team for navigation. To develop a robotic guiding system that is genuinely beneficial to blind or visually impaired individuals, we performed qualitative research, including interviews with guide dog handlers and trainers and first-hand blindfold walking experie…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Sony AI（索尼人工智能）
While imitation learning for vision based autonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or worse yet the demonstrator does not have access to the platform at all? Is imitation learning for vision based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent …
Institut de Robòtica i Informàtica Industrial (CSIC-UPC)（机器人与工业信息研究所（CSIC-UPC））
This paper proposes a Social Reward Sources (SRS) design for a Human-Robot Collaborative Navigation (HRCN) task: human-robot collaborative search. It is a flexible approach capable of handling the collaborative task, human-robot interaction and environment restrictions, all integrated on a common environment. We modelled task rewards based on unexplored area observability and isolation and evaluated the model through different levels of human-robot communication. The models are validated through quantitative evaluation against both agents' individual performance and qualitative surveying of pa…
RSRNav: Reasoning Spatial Relationship for Image-Goal Navigation
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（国家人类-机器混合增强智能重点实验室、视觉信息与应用国家工程研究中心、人工智能与机器人研究所，西安交通大学）
From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation
Australian Institute for Machine Learning, Adelaide University（澳大利亚机器学习研究所，阿德莱德大学）；Ruyi Dynamics（如意动力）；Fudan University（复旦大学）；Swiss Federal Institute of Technology Lausanne (EPFL)（瑞士洛桑联邦理工学院）
Reasoning Systems for Semantic Navigation in Mobile Robots
University of Carlos III of Madrid（卡洛斯三世大学）；Technical University of Cartagena（卡塔赫纳理工大学）；Institute of Artificial Intelligent at the University of Bremen（不来梅大学人工智能研究所）
Human navigation in built environments depends on symbolic spatial information which has unrealised potential to enhance robot navigation capabilities. Information sources such as labels, signs, maps, planners, spoken directions, and navigational gestures communicate a wealth of spatial information to the navigators of built environments; a wealth of information that robots typically ignore. We present a robot navigation system that uses the same symbolic spatial information employed by humans to purposefully navigate in unseen built environments with a level of performance comparable to human…
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Zhejiang Normal University（浙江师范大学）
Vision-and-Language Navigation in Continuous Environments (VLN-CE), which links language instructions to perception and control in the real world, is a core capability of embodied robots. Recently, large-scale pretrained foundation models have been leveraged as shared priors for perception, reasoning, and action, enabling zero-shot VLN without task-specific training. However, existing zero-shot VLN methods depend on costly perception and passive scene understanding, collapsing control to point-level choices. As a result, they are expensive to deploy, misaligned in action semantics, and short-s…
Swinburne University of Technology（斯威本科技大学）；Melbourne University（墨尔本大学）
Vision guided navigation requires processing complex visual information to inform task-orientated decisions. Applications include autonomous robots, self-driving cars, and assistive vision for humans. A key element is the extraction and selection of relevant features in pixel space upon which to base action choices, for which Machine Learning techniques are well suited. However, Deep Reinforcement Learning agents trained in simulation often exhibit unsatisfactory results when deployed in the real-world due to perceptual differences known as the $\textit{reality gap}$. An approach that is yet t…
