User-Feedback-Driven Adaptation for Vision-and-Language Navigation
Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed大学人工智能学院);School of Electrical Engineering and Computer Science, The University of Queensland(电气工程与计算机科学学院,昆士兰大学);University of Science and Technology of China(中国科学技术大学);School of Computer Science, The University of Adelaide(计算机科学学院,阿德莱德大学)
Real-world deployment of Vision-and-Language Navigation (VLN) agents is constrained by the scarcity of reliable supervision after offline training. While recent adaptation methods attempt to mitigate distribution shifts via environment-driven self-supervision (e.g., entropy minimization), these signals are often noisy and can cause the agent to amplify its own mistakes during long-horizon sequential decision-making. In this paper, we propose a paradigm shift that positions user feedback, specifically episode-level success confirmations and goal-level corrections, as a primary and general-purpose supervision signal for VLN. Unlike internal confidence scores, user feedback is intent-aligned and in-situ consistent, directly correcting the agent's decoupling from user instructions. To effectively leverage this supervision, we introduce a user-feedback-driven learning framework featuring a topology-aware trajectory construction pipeline. This mechanism lifts sparse, goal-level corrections into dense path-level supervision by generating feasible paths on the agent's incrementally built topological graph, enabling sample-efficient imitation learning without requiring step-by-step human demonstrations. Furthermore, we develop a persistent memory bank mechanism for warm-start initialization, supporting the reuse of previously acquired topology and cached representations across navigation sessions. Extensive experiments on the GSA-R2R benchmark demonstrate that our approach transforms sparse interaction into robust supervision, consistently outperforming environment-driven baselines while exhibiting strong adaptability across diverse instruction styles.
Hierarchical Belief Modeling for Zero-Shot Opponent Adaptation in Partially Observable Multi-Agent Navigation
Tsinghua University(清华大学);Stevens Institute of Technology(史蒂文斯理工学院);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Texas A&M University(德克萨斯A&M大学)
DeFINE: Delayed Feedback based Immersive Navigation Environment for Studying Goal-Directed Human Navigation
Aalto University(阿尔托大学);The University of Queensland(昆士兰大学);Queensland Brain Institute(昆士兰脑研究所);Queensland University of Technology(昆士兰科技大学);University of Oulu(奥卢大学)
HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback
Northwest A&F University(西北农林科技大学);Tencent AI Lab(腾讯AI实验室);The University of Melbourne(墨尔本大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Seeing Through Uncertainty: Free-Energy-Inspired Real-Time Adaptation for Robust Visual Navigation
Department of Computer Science, The University of Manchester(曼彻斯特大学计算机科学系);Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR)(信息与通信研究院(I2R)、科技研究局(A*STAR));College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation
University of British Columbia(不列颠哥伦比亚大学);National University of Singapore(新加坡国立大学);University of Washington(华盛顿大学);Centre for Frontier AI Research, A*STAR(新加坡科技研究局前沿人工智能研究中心)