Navigation tasks in photorealistic 3D environments are challenging because they require perception and effective planning under partial observability. Recent work shows that map-like memory is useful for long-horizon navigation tasks. However, a focused investigation of the impact of maps on navigation tasks of varying complexity has not yet been performed. We propose the multiON task, which requires navigation to an episode-specific sequence of objects in a realistic environment. MultiON generalizes the ObjectGoal navigation task and explicitly tests the ability of navigation agents to locate previously observed goal objects. We perform a set of multiON experiments to examine how a variety of agent models perform across a spectrum of navigation task complexities. Our experiments show that: i) navigation performance degrades dramatically with escalating task complexity; ii) a simple semantic map agent performs surprisingly well relative to more complex neural image feature map agents; and iii) even oracle map agents achieve relatively low performance, indicating the potential for future work in training embodied navigation agents using maps. Video summary: https://youtu.be/yqTlHNIcgnY
We present SGoLAM, short for simultaneous goal localization and mapping, which is a simple and efficient algorithm for Multi-Object Goal navigation. Given an agent equipped with an RGB-D camera and a GPS/Compass sensor, our objective is to have the agent navigate to a sequence of target objects in realistic 3D environments. Our pipeline fully leverages the strength of classical approaches for visual navigation, by decomposing the problem into two key components: mapping and goal localization. The mapping module converts the depth observations into an occupancy map, and the goal localization mo…
Navigation research is attracting renewed interest with the advent of learning-based methods. However, this new line of work is largely disconnected from well-established classic navigation approaches. In this paper, we take a step towards coordinating these two directions of research. We set up classic and learning-based navigation systems in common simulated environments and thoroughly evaluate them in indoor spaces of varying complexity, with access to different sensory modalities. Additionally, we measure human performance in the same environments. We find that a classic pipeline, when pro…
We propose a novel benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation. Aquatic navigation is an extremely challenging task due to the non-stationary environment and the uncertainties of the robotic platform, hence it is crucial to consider the safety aspect of the problem, by analyzing the behavior of the trained network to avoid dangerous situations (e.g., collisions). To this end, we consider a value-based and policy-gradient Deep Reinforcement Learning (DRL) and we propose a crossover-based strategy that combines gradient-based and gradient-free DRL to impr…
Georgia Tech（佐治亚理工学院）；Allen Institute of AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）
As an attempt towards assessing the robustness of embodied navigation agents, we propose RobustNav, a framework to quantify the performance of embodied navigation agents when exposed to a wide variety of visual - affecting RGB inputs - and dynamics - affecting transition dynamics - corruptions. Most recent efforts in visual navigation have typically focused on generalizing to novel target environments with similar appearance and dynamics characteristics. With RobustNav, we find that some standard embodied navigation agents significantly underperform (or fail) in the presence of visual or dynam…
Benchmarking Interaction, Beyond Policy: a Reproducible Benchmark for Collaborative Instance Object Navigation
Sapienza University of Rome（罗马大学）；University of Verona（维罗纳大学）；Polytechnic of Turin（都灵理工大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；Reykjavik University（雷克雅未克大学）
NavTrust: Benchmarking Trustworthiness for Embodied Navigation
Trustworthy Autonomous Systems Laboratory at the University of California, Riverside（加州大学河滨分校可信自主系统实验室）；University of Michigan（密歇根大学）；Workday；University of Southern California（南加州大学）；Texas A&M University（德克萨斯A&M大学）；Lehigh University（莱斯大学）
IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarking
The University of Hong Kong（香港大学）；Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；University of Michigan（密歇根大学）；University of Macau（澳门大学）
Metric ground navigation addresses the problem of autonomously moving a robot from one point to another in an obstacle-occupied planar environment in a collision-free manner. It is one of the most fundamental capabilities of intelligent mobile robots. This paper presents a standardized testbed with a set of environments and metrics to benchmark difficulty of different scenarios and performance of different systems of metric ground navigation. Current benchmarks focus on individual components of mobile robot navigation, such as perception and state estimation, but the navigation performance as …
Chalmers University of Technology（查尔姆斯理工大学）；University College London（伦敦大学学院）
There are various trajectory planners for mobile manipulators. It is often challenging to compare their performance under similar circumstances due to differences in hardware, dissimilarity of tasks and objectives, as well as uncertainties in measurements and operating environments. In this paper, we propose a simulation framework to evaluate the performance of the local trajectory planners to generate smooth, and dynamically and kinematically feasible trajectories for mobile manipulators in the same environment. We focus on local planners as they are key components that provide smooth traject…
Australian National University（澳大利亚国立大学）；Princeton University（普林斯顿大学）；Carnegie Mellon University（卡内基梅隆大学）；Intel Labs（英特尔实验室）；UC Berkeley（加州大学伯克利分校）；George Mason University（乔治梅森大学）；Facebook（脸书）；Allen Institute for AI（艾伦人工智能研究所）
Skillful mobile operation in three-dimensional environments is a primary topic of study in Artificial Intelligence. The past two years have seen a surge of creative work on navigation. This creative output has produced a plethora of sometimes incompatible task definitions and evaluation protocols. To coordinate ongoing and future research in this area, we have convened a working group to study empirical methodology in navigation research. The present document summarizes the consensus recommendations of this working group. We discuss different problem statements and the role of generalization, …
MetricNet: Recovering Metric Scale in Generative Navigation Policies
Artificial Intelligence and Robotics Lab, Department of Computer Science and Artificial Intelligence, University of Technology Nuremberg（技术大学纽伦堡计算机科学与人工智能系人工智能与机器人实验室）；Inria, Ecole Normale Supérieure, CNRS, PSL Research University（法国国家信息与自动化技术研究院、巴黎高等师范学校、国家科学研究中心、巴黎-萨克勒研究大学）
RVN-Bench: A Benchmark for Reactive Visual Navigation
Department of Electrical and Computer Engineering, Seoul National University (SNU) and Automation and Systems Research Institute (ASRI) and Sequor Robotics Inc.（电气与计算机工程系，首尔国立大学（SNU）和自动化与系统研究所（ASRI）以及Sequor机器人公司）；Department of Electrical and Computer Engineering, Seoul National University (SNU) and Automation and Systems Research Institute (ASRI)（电气与计算机工程系，首尔国立大学（SNU）和自动化与系统研究所（ASRI））；Interdisciplinary Program in Artificial Intelligence, Seoul National University (SNU) and Automation and Systems Research Institute (ASRI) and Sequor Robotics Inc.（人工智能跨学科项目，首尔国立大学（SNU）和自动化与系统研究所（ASRI）以及Sequor机器人公司）；Sequor Robotics Inc.（Sequor机器人公司）
EEG-Driven Intention Decoding: Offline Deep Learning Benchmarking on a Robotic Rover
Department of Computer Science, Imam Mohammad Ibn Saud Islamic University（计算机科学系，伊玛目穆罕默德·伊本·沙特伊斯兰大学）；Department of Computer Science, King Saud University（计算机科学系，国王沙特大学）
UC San Diego（加州大学圣地亚哥分校）；NEC Labs America（NEC美国实验室）；University of Pennsylvania（宾夕法尼亚大学）
We address key challenges in long-horizon embodied exploration and navigation by proposing a new object transport task and a novel modular framework for temporally extended navigation. Our first contribution is the design of a novel Long-HOT environment focused on deep exploration and long-horizon planning where the agent is required to efficiently find and pick up target objects to be carried and dropped at a goal location, with load constraints and optional access to a container if it finds one. Further, we propose a modular hierarchical transport policy (HTP) that builds a topological graph…
MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation
Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）；University of California, Los Angeles（加州大学洛杉矶分校）；University of California, Berkeley（加州大学伯克利分校）
Université de Nantes（南特大学）；Beijing University of Technology（北京工业大学）
Free-Viewpoint Video (FVV) systems allow the viewers to freely change the viewpoints of the scene. In such systems, view synthesis and compression are the two main sources of artifacts influencing the perceived quality. To assess this influence, quality evaluation studies are often carried out using conventional displays and generating predefined navigation trajectories mimicking the possible movement of the viewers when exploring the content. Nevertheless, as different trajectories may lead to different conclusions in terms of visual quality when benchmarking the performance of the systems, m…
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；U.S. Army Combat Capabilities Development Command Soldier Center (DEVCOM-SC)（美国陆军作战能力发展司令部士兵中心）
This report reviews all results derived from performance benchmarking conducted during Phase I of the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell, using the test methods specified in the DECISIVE Test Methods Handbook v1.1 for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. Usin…
This work introduces a simulator-based benchmark for visual localization in the autonomous navigation context. The dynamic benchmark enables investigation of how variables such as the time of day, weather, and camera perspective affect the navigation performance of autonomous agents that utilize visual localization for closed-loop control. The experimental part of the paper studies the effects of four such variables by evaluating state-of-the-art visual localization methods as part of the motion planning module of an autonomous navigation stack. The results show major variation in the suitabil…
Facebook AI Research（脸书人工智能研究院）；Simon Fraser University（西蒙弗雷泽大学）；Facebook Reality Labs（脸书现实实验室）；Georgia Institute of Technology（佐治亚理工学院）；Intel Labs（英特尔实验室）；UC Berkeley（加州大学伯克利分校）
We present Habitat, a platform for research in embodied artificial intelligence (AI). Habitat enables training embodied agents (virtual robots) in highly efficient photorealistic 3D simulation. Specifically, Habitat consists of: (i) Habitat-Sim: a flexible, high-performance 3D simulator with configurable agents, sensors, and generic 3D dataset handling. Habitat-Sim is fast -- when rendering a scene from Matterport3D, it achieves several thousand frames per second (fps) running single-threaded, and can reach over 10,000 fps multi-process on a single GPU. (ii) Habitat-API: a modular high-level l…
UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；State Key Laboratory of Complex & Critical Software Environment, Beihang University（北航复杂与关键软件环境国家重点实验室）；School of Computer Science, Institute for Artificial Intelligence, State Key Laboratory of General Artificial Intelligence, Peking University（北京大学计算机学院）；City University of Macau（澳门城市大学）；National University of Singapore（新加坡国立大学）；Beijing Institute for General Artificial Intelligence (BIGAI)（北京通用人工智能研究院）
TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings
Computational Intelligence and Operations Laboratory (CIOL), Bangladesh（计算智能与运筹实验室（CIOL），孟加拉国）；Shahjalal University of Science and Technology (SUST), Sylhet, Bangladesh（沙赫jalal科学与技术大学（SUST），沙赫里尔，孟加拉国）；North South University (NSU), Dhaka, Bangladesh（北南大学（NSU），达卡，孟加拉国）；Qatar Computing Research Institute (QCRI), Doha, Qatar（卡塔尔计算研究中心（QCRI），多哈，卡塔尔）
DreamSat-Bench: Development and Initial Testing of a Testbed for AI-Based Pose Estimation from 3D Reconstruction
Massachusetts Institute of Technology（麻省理工学院）；Politecnico di Milano（米兰理工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Universidad Politécnica de Madrid（马德里理工大学）
