Underwater target localization uses real-time sensory measurements to estimate the position of underwater objects of interest, providing critical feedback information for underwater robots. While acoustic sensing is the most acknowledged method in underwater robots and possibly the only effective approach for long-range underwater target localization, such a sensing modality generally suffers from low resolution, high cost and high energy consumption, thus leading to a mediocre performance when applied to close-range underwater target localization. On the other hand, optical sensing has attracted increasing attention in the underwater robotics community for its advantages of high resolution and low cost, holding a great potential particularly in close-range underwater target localization. However, most existing studies in underwater optical sensing are restricted to specific types of targets due to the limited training data available. In addition, these studies typically focus on the design of estimation algorithms and ignore the influence of illumination conditions on the sensing performance, thus hindering wider applications in the real world. To address the aforementioned issues, this paper proposes a novel target localization method that assimilates both optical and acoustic sensory measurements to estimate the 3D positions of close-range underwater targets. A test platform with controllable illumination conditions is designed and developed to experimentally investigate the proposed multi-modal sensing approach. A large vision model is applied to process the optical imaging measurements, eliminating the requirement for training data acquisition, thus significantly expanding the scope of potential applications. Extensive experiments are conducted, the results of which validate the effectiveness of the proposed underwater target localization method.
University of Coimbra（科英布拉大学）；Ingeniarius, Ltd.（Ingeniarius有限公司）
Autonomous Underwater Vehicles (AUVs) have the ability to operate in harsh underwater environments without endangering human lives in the process. Nevertheless, just like their ground and aerial counterparts, AUVs need to be able to estimate their own position. Yet, unlike ground and aerial robots, estimating the pose of AUVs is very challenging, with only a few high-cost technological solutions available in the market. In this paper, we present the development of a realistic underwater acoustic model, implemented within the Robot Operating System (ROS) and the Gazebo simulator framework, for …
Minnesota Robotics Institute（明尼苏达机器人研究所）；University of Minnesota–Twin Cities（明尼苏达大学双城分校）
This paper presents an evaluation of a number of probabilistic algorithms for localization of autonomous underwater vehicles (AUVs) using bathymetry data. The algorithms, based on the principles of the Bayes filter, work by fusing bathymetry information with depth and altitude data from an AUV. Four different Bayes filter-based algorithms are used to design the localization algorithms: the Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), Particle Filter (PF), and Marginalized Particle Filter (MPF). We evaluate the performance of these four Bayesian bathymetry-based AUV localization…
Having accurate localization capabilities is one of the fundamental requirements of autonomous robots. For underwater vehicles, the choices for effective localization are limited due to limitations of GPS use in water and poor environmental visibility that makes camera-based methods ineffective. Popular inertial navigation methods for underwater localization using Doppler-velocity log sensors, sonar, high-end inertial navigation systems, or acoustic positioning systems require bulky expensive hardware which are incompatible with low cost, bio-inspired underwater robots. In this paper, we intro…
Bearing only cooperative localization has been used successfully on aerial and ground vehicles. In this paper we present an extension of the approach to the underwater domain. The focus is on adapting the technique to handle the challenging visibility conditions underwater. Furthermore, data from inertial, magnetic, and depth sensors are utilized to improve the robustness of the estimation. In addition to robotic applications, the presented technique can be used for cave mapping and for marine archeology surveying, both by human divers. Experimental results from different environments, includi…
This article focuses on feature-based underwater localization and navigation for autonomous underwater vehicles (AUVs) using 2D imaging sonar measurements. The sparsity of underwater acoustic features and the loss of elevation angle in sonar images may introduce wrong feature matches or insufficient features for optimization-based underwater localization (i.e. under-constrained/degeneracy cases). This motivates us to propose a novel inertial-aided sliding window optimization framework to improve the estimation accuracy and the robustness to front-end outliers. Concretely, we first discriminate…
Localization between a swarm of AUVs can be entirely estimated through the use of range measurements between neighboring AUVs via a class of techniques commonly referred to as sensor network localization. However, the localization quality depends on network topology, with degenerate topologies, referred to as low-rigidity configurations, leading to ambiguous or highly uncertain localization results. This paper presents tools for rigidity-based analysis, planning, and control of a multi-AUV network which account for sensor noise and limited sensing range. We evaluate our long-term planning fram…
Learning-Based Leader Localization for Underwater Vehicles With Optical-Acoustic-Pressure Sensor Fusion
Department of Advanced Manufacturing and Robotics, College of Engineering, and the State Key Laboratory of Turbulence and Complex Systems（先进制造与机器人系，工程学院，湍流与复杂系统国家重点实验室）
Dehazing-aided Multi-Rate Multi-Modal Pose Estimation Framework for Mitigating Visual Disturbances in Extreme Underwater Domain
Center for Autonomous Robotic Systems, Khalifa University（哈利法大学自主机器人系统中心）；College of Engineering, Abu Dhabi University（阿布扎比大学工程学院）；Autonomous Robotics Research Center, Technology Innovation Institute（技术创新研究院自主机器人研究中心）
An Underwater, Fault-Tolerant, Laser-Aided Robotic Multi-Modal Dense SLAM System for Continuous Underwater In-Situ Observation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
ONERA, Université Paris Saclay（法国航空航天实验室（ONERA），巴黎萨克雷大学）；LIRMM, Université de Montpellier, CNRS（蒙彼利埃计算机、机器人与微电子实验室，蒙彼利埃大学，法国国家科学研究中心）；DRASSM, Ministry of Culture（法国文化部水下与海底考古研究局）
This paper presents a new underwater dataset acquired from a visual-inertial-pressure acquisition system and meant to be used to benchmark visual odometry, visual SLAM and multi-sensors SLAM solutions. The dataset is publicly available and contains ground-truth trajectories for evaluation.
University of South Carolina（南卡罗来纳大学）；Dartmouth College（达特茅斯学院）
This paper presents a novel tightly-coupled keyframe-based Simultaneous Localization and Mapping (SLAM) system with loop-closing and relocalization capabilities targeted for the underwater domain. Our previous work, SVIn, augmented the state-of-the-art visual-inertial state estimation package OKVIS to accommodate acoustic data from sonar in a non-linear optimization-based framework. This paper addresses drift and loss of localization -- one of the main problems affecting other packages in underwater domain -- by providing the following main contributions: a robust initialization method to refi…
In the context of robotic underwater operations, the visual degradations induced by the medium properties make difficult the exclusive use of cameras for localization purpose. Hence, most localization methods are based on expensive navigational sensors associated with acoustic positioning. On the other hand, visual odometry and visual SLAM have been exhaustively studied for aerial or terrestrial applications, but state-of-the-art algorithms fail underwater. In this paper we tackle the problem of using a simple low-cost camera for underwater localization and propose a new monocular visual odome…
German Research Center for Artificial Intelligence（德国人工智能研究中心）
This paper presents a manifold based Unscented Kalman Filter that applies a novel strategy for inertial, model-aiding and Acoustic Doppler Current Profiler (ADCP) measurement incorporation. The filter is capable of observing and utilizing the Earth rotation for heading estimation with a tactical grade IMU, and utilizes information from the vehicle model during DVL drop outs. The drag and thrust model-aiding accounts for the correlated nature of vehicle model parameter error by applying them as states in the filter. ADCP-aiding provides further information for the model-aiding in the case of DV…
University of South Carolina（南卡罗来纳大学）；McGill University（麦吉尔大学）；Dartmouth College（达特茅斯学院）
In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image-to-image tra…
We present a new dataset, dedicated to the development of simultaneous localization and mapping methods for underwater vehicles navigating close to the seabed. The data sequences composing this dataset are recorded in three different environments: a harbor at a depth of a few meters, a first archaeological site at a depth of 270 meters and a second site at a depth of 380 meters. The data acquisition is performed using Remotely Operated Vehicles equipped with a monocular monochromatic camera, a low-cost inertial measurement unit, a pressure sensor and a computing unit, all embedded in a single …
Direct localization (DLOC) methods, which use the observed data to localize a source at an unknown position in a one-step procedure, generally outperform their indirect two-step counterparts (e.g., using time-difference of arrivals). However, underwater acoustic DLOC methods require prior knowledge of the environment, and are computationally costly, hence slow. We propose, what is to the best of our knowledge, the first data-driven DLOC method. Inspired by classical and contemporary optimal model-based DLOC solutions, and leveraging the capabilities of convolutional neural networks (CNNs), we …
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Autonomous Underwater Vehicles (AUVs) and Remotely Operated Vehicles (ROVs) are used for a wide variety of missions related to exploration and scientific research. Successful navigation by these systems requires a good localization system. Kalman filter based localization techniques have been prevalent since the early 1960s and extensive research has been carried out using them, both in development and in design. It has been found that the use of a dynamic model (instead of a kinematic model) in the Kalman filter can lead to more accurate predictions, as the dynamic model takes the forces acti…
MBARI（蒙特雷湾水族馆研究所）；Institut de Ciències del Mar, CSIC（西班牙国家研究委员会海洋科学研究所）；Universitat Politècnica de Catalunya, Barcelona Tech.（加泰罗尼亚理工大学（巴塞罗那理工））
Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optim…
Underwater Visual-Inertial-Acoustic-Depth SLAM with DVL Preintegration for Degraded Environments
School of Ocean Engineering and Technology & Southern Marine science and Engineering Guangdong Laboratory (Zhuhai), Sun Yat-sen University（海洋工程学院及南方海洋科学与工程广东实验室（珠海），中山大学）；Guangdong Provincial Key Laboratory of Information Technology for Deep Water Acoustics（广东省信息技术创新深水声学重点实验室）；Key Laboratory of Comprehensive Observation of Polar Environment (Sun Yat-sen University), Ministry of Education（极地环境综合观测重点实验室（中山大学），教育部）；School of Marine Science and Technology Domain, Beijing Institute of Technology (Zhuhai)（海洋科学与技术学院，北京理工大学（珠海））
Italian National Agency for New Technologies, Energy and Sustainable Economic Development（意大利新技术、能源与可持续经济发展局）
The subject is the localization problem of an underwater swarm of autonomous underwater robots (AUV), in the frame of the HARNESS project; by localization, we mean the relative swarm configuration, i.e., the geometrical shape of the group. The result is achieved by using the signals that the robots exchange. The swarm is organized by rules and conceived to perform tasks, ranging from environmental monitoring to terrorism-attack surveillance. Two methods of determining the shape of the swarm, both based on trilateration calculation, are proposed. The first method focuses on the robot's speed. I…
An essential task for a multi-robot system is generating a common understanding of the environment and relative poses between robots. Cooperative tasks can be executed only when a vehicle has knowledge of its own state and the states of the team members. However, this has primarily been achieved with direct rendezvous between underwater robots, via inter-robot ranging. We propose a novel distributed multi-robot simultaneous localization and mapping (SLAM) framework for underwater robots using imaging sonar-based perception. By passing only scene descriptors between robots, we do not need to pa…
Underwater Dense Mapping with the First Compact 3D Sonar
Mechanical Engineering Department, University of Delaware（德克萨斯大学机械工程系）；Dartmouth College（达特茅斯学院）；Aquaculture Robotics and Automation Group, SINTEF Ocean（SINTEF海洋研究所水产机器人与自动化小组）
Localization is a key requirement for mobile robot autonomy and human-robot interaction. Vision-based localization is accurate and flexible, however, it incurs a high computational burden which limits its application on many resource-constrained platforms. In this paper, we address the problem of performing real-time localization in large-scale 3D point cloud maps of ever-growing size. While most systems using multi-modal information reduce localization time by employing side-channel information in a coarse manner (eg. WiFi for a rough prior position estimate), we propose to inter-weave the ma…
Autonomous Underwater Vehicles (AUVs) are platforms used for research and exploration of marine environments. However, these types of vehicles face many challenges that hinder their widespread use in the industry. One of the main limitations is obtaining accurate position estimation, due to the lack of GPS signal underwater. This estimation is usually done with Kalman filters. However, new developments in the neuroscience field have shed light on the mechanisms by which mammals are able to obtain a reliable estimation of their current position based on external and internal motion cues. A new …
Autonomous underwater vehicles (AUVs) rely on a variety of sensors - acoustic, inertial and visual - for intelligent decision making. Due to its non-intrusive, passive nature, and high information content, vision is an attractive sensing modality, particularly at shallower depths. However, factors such as light refraction and absorption, suspended particles in the water, and color distortion affect the quality of visual data, resulting in noisy and distorted images. AUVs that rely on visual sensing thus face difficult challenges, and consequently exhibit poor performance on vision-driven tasks…
Visibility underwater is challenging, and degrades as the distance between the subject and camera increases, making vision tasks in the forward-looking direction more difficult. We have collected underwater forward-looking stereo-vision and visual-inertial image sets in the Mediterranean and Red Sea. To our knowledge there are no other public datasets in the underwater environment acquired with this camera-sensor orientation published with ground-truth. These datasets are critical for the development of several underwater applications, including obstacle avoidance, visual odometry, 3D tracking…
