We consider the problem of embodied visual navigation given an image-goal (ImageNav) where an agent is initialized in an unfamiliar environment and tasked with navigating to a location 'described' by an image. Unlike related navigation tasks, ImageNav does not have a standardized task definition which makes comparison across methods difficult. Further, existing formulations have two problematic properties; (1) image-goals are sampled from random locations which can lead to ambiguity (e.g., looking at walls), and (2) image-goals match the camera specification and embodiment of the agent; this rigidity is limiting when considering user-driven downstream applications. We present the Instance-specific ImageNav task (InstanceImageNav) to address these limitations. Specifically, the goal image is 'focused' on some particular object instance in the scene and is taken with camera parameters independent of the agent. We instantiate InstanceImageNav in the Habitat Simulator using scenes from the Habitat-Matterport3D dataset (HM3D) and release a standardized benchmark to measure community progress.
MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Science(中国科学院大学人工智能学院);Australian Institute for Machine Learning, The University of Adelaide(阿德莱德大学澳大利亚机器学习研究所);Tongji University(同济大学)
Beijing Institute of Technology(北京理工大学);ETH Zurich(苏黎世联邦理工学院);University of Technology Sydney(悉尼科技大学);Yangtze Delta Region Academy of Beijing Institute of Technology(北京理工大学长三角研究院)
NavBench: Probing Multimodal Large Language Models for Embodied Navigation
The University of Adelaide(阿德莱德大学);The University of Queensland(昆士兰大学);CSIRO Data61(澳大利亚联邦科学与工业研究组织数据61研究中心);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Tongji University(同济大学)
Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation
Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);UAS Esslingen(埃森嫩大学);TU Wien(维也纳技术大学);Dr. Ing. h.c. F. Porsche AG(保时捷股份有限公司);University of Michigan(密歇根大学);Voxel51 Inc.(Voxel51公司)
Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation
Tsinghua University(清华大学);Beijing Institute of Technology(北京理工大学);Beihang University(北航);State Key Laboratory of General Artificial Intelligence, BIGAI, China(国家一般人工智能重点实验室, BIGAI, 中国)
Consistent Attack: Universal Adversarial Perturbation on Embodied Vision Navigation
Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology, Tsinghua-Bosch Joint Center for Machine Learning, Institute for Artificial Intelligence, Tsinghua University, Beijing 100084, China(计算机科学与技术系,北京信息科学国家研究中心,清华-博世联合机器学习中心,人工智能研究院,清华大学,北京)
Meta AI Research(Meta人工智能研究院);Carnegie Mellon University(卡内基梅隆大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
像图像-目标导航这样的现实长期任务涉及探索和利用阶段。由于目标的图像,一个具体的代理必须探索发现目标,即,使用学习的先验知识进行有效搜索。一旦目标被发现,智能体必须准确地校准到目标的最后一英里的导航。与任何强大的系统一样,探索性目标发现和利用性最后一英里导航之间的切换可以更好地从错误中恢复。根据这些直观的导轨,我们提出了SLING,以提高现有的图像目标导航系统的性能。作为对现有方法的补充,我们专注于最后一英里导航,并利用神经描述符来利用问题的底层几何结构。通过简单但有效的开关,我们可以轻松地将SLING与启发式、强化学习和神经模块化策略联系起来。在标准化的图像目标导航基准(Hahn et al. 2021)上,我们提高了策略,场景和情节复杂性的性能,将最先进的成功率从45%提高到55%。除了真实感模拟,我们在三个物理场景中进行了真实的机器人实验,并发现这些改进可以很好地转移到真实环境中。
Instruction-as-State: Environment-Guided and State-Conditioned Semantic Understanding for Embodied Navigation
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence(人机混合增强智能国家重点实验室);The Institute of Artificial Intelligence and Robotics(人工智能与机器人研究所);School of Information Science and Technology(信息科学与技术学院);Zhongguancun Academy(中关村学院)
Embodied Active Learning under Limited Annotation and Navigation Budget for Object Detection
LAAS-CNRS, Universite de Toulouse, CNRS(法国国家科学研究中心图卢兹分部LAAS、图卢兹大学、法国国家科学研究中心);Department of Electrical Engineering and Automation at Aalto University(阿尔托大学电气工程与自动化系)
VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation
Adelaide University(阿德莱德大学);Responsible AI Research Centre, Australian Institute for Machine Learning(负责任人工智能研究中心、澳大利亚机器学习研究所);Tongji University(同济大学);ManyCore;Zhejiang University(浙江大学);Shanghai AI Lab(上海人工智能实验室);CSIRO Data61
IndustryNav: Evaluating Spatial Reasoning and Safety of Embodied Agents in Dynamic Industrial Navigation
Michigan State University(密歇根州立大学);Independent Researcher(独立研究者);University of California, Irvine(加州大学尔湾分校);Ohio State University(俄亥俄州立大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of California, Santa Barbara(加州大学圣巴巴拉分校);University of Pittsburgh(匹兹堡大学);University of Virginia(弗吉尼亚大学);Arizona State University(亚利桑那州立大学);Purdue University Northwest(普渡大学西北分校)
NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM
School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院);College of Information Science and Technology, Beijing University of Chemical Technology(北京化工大学信息科学与技术学院)
FloNa: Floor Plan Guided Embodied Visual Navigation
Beijing Institute of Technology(北京理工大学);Yangtze Delta Region Academy of Beijing Institute of Technology(北京理工大学长三角研究院);Beijing Racobit Electronic Information Technology Co., Ltd.(北京睿科比特电子信息技术有限公司)
NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigation
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院)
NavTrust: Benchmarking Trustworthiness for Embodied Navigation
Trustworthy Autonomous Systems Laboratory at the University of California, Riverside(加州大学河滨分校可信自主系统实验室);University of Michigan(密歇根大学);Workday;University of Southern California(南加州大学);Texas A&M University(德克萨斯A&M大学);Lehigh University(莱斯大学)