The process of satisfying daily demands is a fundamental aspect of humans' daily lives. With the advancement of embodied AI, robots are increasingly capable of satisfying human demands. Demand-driven navigation (DDN) is a task in which an agent must locate an object to satisfy a specified demand instruction, such as ``I am thirsty.'' The previous study typically assumes that each demand instruction requires only one object to be fulfilled and does not consider individual preferences. However, the realistic human demand may involve multiple objects. In this paper, we introduce the Multi-object Demand-driven Navigation (MO-DDN) benchmark, which addresses these nuanced aspects, including multi-object search and personal preferences, thus making the MO-DDN task more reflective of real-life scenarios compared to DDN. Building upon previous work, we employ the concept of ``attribute'' to tackle this new task. However, instead of solely relying on attribute features in an end-to-end manner like DDN, we propose a modular method that involves constructing a coarse-to-fine attribute-based exploration agent (C2FAgent). Our experimental results illustrate that this coarse-to-fine exploration strategy capitalizes on the advantages of attributes at various decision-making levels, resulting in superior performance compared to baseline methods. Code and video can be found at https://sites.google.com/view/moddn.
Team Xiaomi EV-AD VLA: Caption-Guided Retrieval System for Cross-Modal Drone Navigation -- Technical Report for IROS 2025 RoboSense Challenge Track 4
Tsinghua University（清华大学）；Xiaomi EV（小米电动车）；Georgia Institute of Technology（佐治亚理工学院）；National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong（香港中文大学）；Renmin University of China（中国人民大学）
AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild
Southeast University（东南大学）；Purple Mountain Labs（紫金山实验室）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）
Traversability prediction is a fundamental perception capability for autonomous navigation. The diversity of data in different domains imposes significant gaps to the prediction performance of the perception model. In this work, we make efforts to reduce the gaps by proposing a novel coarse-to-fine unsupervised domain adaptation (UDA) model - CALI. Our aim is to transfer the perception model with high data efficiency, eliminate the prohibitively expensive data labeling, and improve the generalization capability during the adaptation from easy-to-obtain source domains to various challenging tar…
Stairway to Success: An Online Floor-Aware Zero-Shot Object-Goal Navigation Framework via LLM-Driven Coarse-to-Fine Exploration
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）
In this work, we consider the problem of learning a perception model for monocular robot navigation using few annotated images. Using a Vision Transformer (ViT) pretrained with a label-free self-supervised method, we successfully train a coarse image segmentation model for the Duckietown environment using 70 training images. Our model performs coarse image segmentation at the 8x8 patch level, and the inference resolution can be adjusted to balance prediction granularity and real-time perception constraints. We study how best to adapt a ViT to our task and environment, and find that some lightw…
Department of Intelligent Robotics, Sungkyunkwan University（智能机器人学系，全北国立大学）；NAVER LABS；Department of Robotics and Mechatronics Engineering, DGIST（机器人与机电工程系，韩国科学技术院）
Enhancing Vision-Language Navigation with Multimodal Event Knowledge from Real-World Indoor Tour Videos
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）；Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)（马尔代夫 bin Zayed 大学人工智能学院）；Hangzhou City University（杭州城市学院）
Zhejiang University（浙江大学）；University of Leeds（利兹大学）
We present a novel traffic trajectory editing method which uses spatio-temporal keyframes to control vehicles during the simulation to generate desired traffic trajectories. By taking self-motivation, path following and collision avoidance into account, the proposed force-based traffic simulation framework updates vehicle's motions in both the Frenet coordinates and the Cartesian coordinates. With the way-points from users, lane-level navigation can be generated by reference path planning. With a given keyframe, the coarse-to-fine optimization is proposed to efficiently generate the plausible …
Robust Subpixel Localization of Diagonal Markers in Large-Scale Navigation via Multi-Layer Screening and Adaptive Matching
College of Aerospace Science and Engineering（航空航天科学与工程学院）；National University of Defense Technology（国防科技大学）；Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation（湖南省图像测量与视觉导航重点实验室）
Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School（一般人工智能国家重点实验室，北京大学深圳研究生院）；Oxford Robotics Institute, University of Oxford（牛津大学机器人研究所）；Institute for Machine Learning, Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系机器学习研究所）
HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models
School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院）；Joint Research Laboratory for Embodied Intelligence, Xinjiang University（新疆大学具身智能联合研究实验室）；Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University（新疆大学丝绸之路多语言认知计算国际联合研究实验室）
Time-to-contact (TTC), the time for an object to collide with the observer's plane, is a powerful tool for path planning: it is potentially more informative than the depth, velocity, and acceleration of objects in the scene -- even for humans. TTC presents several advantages, including requiring only a monocular, uncalibrated camera. However, regressing TTC for each pixel is not straightforward, and most existing methods make over-simplifying assumptions about the scene. We address this challenge by estimating TTC via a series of simpler, binary classifications. We predict with low latency whe…
Simultaneous localization and mapping (SLAM) frameworks for autonomous navigation rely on robust data association to identify loop closures for back-end trajectory optimization. In the case of autonomous underwater vehicles (AUVs) equipped with multibeam echosounders (MBES), data association is particularly challenging due to the scarcity of identifiable landmarks in the seabed, the large drift in dead-reckoning navigation estimates to which AUVs are prone and the low resolution characteristic of MBES data. Deep learning solutions to loop closure detection have shown excellent performance on d…
University of Maryland, College Park（马里兰大学帕克分校）；Amazon Alexa AI（亚马逊Alexa人工智能部门）
Household environments are visually diverse. Embodied agents performing Vision-and-Language Navigation (VLN) in the wild must be able to handle this diversity, while also following arbitrary language instructions. Recently, Vision-Language models like CLIP have shown great performance on the task of zero-shot object recognition. In this work, we ask if these models are also capable of zero-shot language grounding. In particular, we utilize CLIP to tackle the novel problem of zero-shot VLN using natural language referring expressions that describe target objects, in contrast to past work that u…
LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；TermiTech（TermiTech公司）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）
AnyImageNav: Any-View Geometry for Precise Last-Meter Image-Goal Navigation
NYUAD Center for Artificial Intelligence and Robotics (CAIR)（纽约大学阿布扎比分校人工智能与机器人中心）；New York University Abu Dhabi, Electrical Engineering（纽约大学阿布扎比分校电气工程系）；Embodied AI and Robotics (AIR) Lab, NYU Abu Dhabi（纽约大学阿布扎比分校具身人工智能与机器人实验室）
LiDAR-VGGT: Cross-Modal Coarse-to-Fine Fusion for Globally Consistent and Metric-Scale Dense Mapping
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University（工业控制技术国家重点实验室，系统与控制研究院，浙江大学）；Differential Robot Technology Co., Ltd.（差分机器人技术有限公司）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）
Heuristic search-based motion planning algorithms typically discretise the search space in order to solve the shortest path problem. Their performance is closely related to this discretisation. A fine discretisation allows for better approximations of the continuous search space, but makes the search for a solution more computationally costly. A coarser resolution might allow the algorithms to find solutions quickly at the expense of quality. For large state spaces, it can be beneficial to search for solutions across multiple resolutions even though defining the discretisations is challenging.…
Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加州理工学院）；Swiss Federal Institute of Technology（瑞士联邦理工学院）；ETH Zürich（苏黎世联邦理工学院）；FieldAI Inc.（FieldAI公司）；Stanford University（斯坦福大学）；University of California Berkeley（加州大学伯克利分校）
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所机器人学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；Biorobotics Lab, Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所生物机器人实验室）
Place recognition and loop closure detection are challenging for long-term visual navigation tasks. SeqSLAM is considered to be one of the most successful approaches to achieving long-term localization under varying environmental conditions and changing viewpoints. It depends on a brute-force, time-consuming sequential matching method. We propose MRS-VPR, a multi-resolution, sampling-based place recognition method, which can significantly improve the matching efficiency and accuracy in sequential matching. The novelty of this method lies in the coarse-to-fine searching pipeline and a particle …
NeRF-Loc: Transformer-Based Object Localization Within Neural Radiance Fields
Stanford University（斯坦福大学）；Chinese University of Hong Kong（香港中文大学）；UC Berkeley（加州大学伯克利分校）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Baidu Inc.（百度公司）；Baidu Research（百度研究院）
