Underwater Dense Mapping with the First Compact 3D Sonar
Mechanical Engineering Department, University of Delaware（德克萨斯大学机械工程系）；Dartmouth College（达特茅斯学院）；Aquaculture Robotics and Automation Group, SINTEF Ocean（SINTEF海洋研究所水产机器人与自动化小组）
Chinmay Burgul, Yewei Huang, Michalis Chatzispyrou, Ioannis Rekleitis, Alberto Quattrini Li, Marios Xanthidis
In the past decade, the adoption of compact 3D range sensors, such as LiDARs, has driven the developments of robust state-estimation pipelines, making them a standard sensor for aerial, ground, and space autonomy. Unfortunately, poor propagation of electromagnetic waves underwater, has limited the visibility-independent sensing options of underwater state-estimation to acoustic range sensors, which provide 2D information including, at-best, spatially ambiguous information. This paper, to the best of our knowledge, is the first study examining the performance, capacity, and opportunities arising from the recent introduction of the first compact 3D sonar. Towards that purpose, we introduce calibration procedures for extracting the extrinsics between the 3D sonar and a camera and we provide a study on acoustic response in different surfaces and materials. Moreover, we provide novel mapping and SLAM pipelines tested in deployments in underwater cave systems and other geometrically and acoustically challenging underwater environments. Our assessment showcases the unique capacity of 3D sonars to capture consistent spatial information allowing for detailed reconstructions and localization in datasets expanding to hundreds of meters. At the same time it highlights remaining challenges related to acoustic propagation, as found also in other acoustic sensors. Datasets collected for our evaluations would be released and shared with the community to enable further research advancements.
RUSSO: Robust Underwater SLAM with Sonar Optimization against Visual Degradation
School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院）；School of Electrical and Information Engineering, Tianjin University（天津大学电气自动化与信息工程学院）
University of South Carolina（南卡罗来纳大学）；Dartmouth College（达特茅斯学院）
This paper presents a novel tightly-coupled keyframe-based Simultaneous Localization and Mapping (SLAM) system with loop-closing and relocalization capabilities targeted for the underwater domain. Our previous work, SVIn, augmented the state-of-the-art visual-inertial state estimation package OKVIS to accommodate acoustic data from sonar in a non-linear optimization-based framework. This paper addresses drift and loss of localization -- one of the main problems affecting other packages in underwater domain -- by providing the following main contributions: a robust initialization method to refi…
Stevens Institute of Technology（斯蒂文斯理工学院）；Exyn Technologies（埃克森科技公司）
Simultaneous localization and mapping (SLAM) is a critical capability for any autonomous underwater vehicle (AUV). However, robust, accurate state estimation is still a work in progress when using low-cost sensors. We propose enhancing a typical low-cost sensor package using widely available and often free prior information; overhead imagery. Given an AUV's sonar image and a partially overlapping, globally-referenced overhead image, we propose using a convolutional neural network (CNN) to generate a synthetic overhead image predicting the above-surface appearance of the sonar image contents. W…
An essential task for a multi-robot system is generating a common understanding of the environment and relative poses between robots. Cooperative tasks can be executed only when a vehicle has knowledge of its own state and the states of the team members. However, this has primarily been achieved with direct rendezvous between underwater robots, via inter-robot ranging. We propose a novel distributed multi-robot simultaneous localization and mapping (SLAM) framework for underwater robots using imaging sonar-based perception. By passing only scene descriptors between robots, we do not need to pa…
University of Electronic Science and Technology of China（电子科技大学）；Dartmouth College（达特茅斯学院）
Underwater navigation presents several challenges, including unstructured unknown environments, lack of reliable localization systems (e.g., GPS), and poor visibility. Furthermore, good-quality obstacle detection sensors for underwater robots are scant and costly; and many sensors like RGB-D cameras and LiDAR only work in-air. To enable reliable mapless underwater navigation despite these challenges, we propose a low-cost end-to-end navigation system, based on a monocular camera and a fixed single-beam echo-sounder, that efficiently navigates an underwater robot to waypoints while avoiding nea…
VIMS: A Visual-Inertial-Magnetic-Sonar SLAM System in Underwater Environments
Key Laboratory of Ocean Observation-Imaging Testbed of Zhejiang Province, Zhejiang University（浙江省海洋观测-成像测试床重点实验室，浙江大学）；Interdisciplinary Student Training Platform for Marine areas, Zhejiang University（海洋领域交叉人才培养平台，浙江大学）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科技大学）；The Institute of Deep-Sea Science and Engineering, Chinese Academy of Sciences（深海科学与工程研究院，中国科学院）
Stevens Institute of Technology（史蒂文斯理工学院）；Amazon Robotics（亚马逊机器人）；Exyn Technologies（Exyn科技）；SRI International（斯坦福国际研究院）
We consider the problem of autonomous mobile robot exploration in an unknown environment, taking into account a robot's coverage rate, map uncertainty, and state estimation uncertainty. This paper presents a novel exploration framework for underwater robots operating in cluttered environments, built upon simultaneous localization and mapping (SLAM) with imaging sonar. The proposed system comprises path generation, place recognition forecasting, belief propagation and utility evaluation using a virtual map, which estimates the uncertainty associated with map cells throughout a robot's workspace…
Kongju National University（公州大学）；ETRI（韩国电子通信研究院）；KAIST（韩国科学技术院）
Among underwater perceptual sensors, imaging sonar has been highlighted for its perceptual robustness underwater. The major challenge of imaging sonar, however, arises from the difficulty in defining visual features despite limited resolution and high noise levels. Recent developments in deep learning provide a powerful solution for computer-vision researches using optical images. Unfortunately, deep learning-based approaches are not well established for imaging sonars, mainly due to the scant data in the training phase. Unlike the abundant publically available terrestrial images, obtaining un…
This article focuses on feature-based underwater localization and navigation for autonomous underwater vehicles (AUVs) using 2D imaging sonar measurements. The sparsity of underwater acoustic features and the loss of elevation angle in sonar images may introduce wrong feature matches or insufficient features for optimization-based underwater localization (i.e. under-constrained/degeneracy cases). This motivates us to propose a novel inertial-aided sliding window optimization framework to improve the estimation accuracy and the robustness to front-end outliers. Concretely, we first discriminate…
Matching sonar images with high accuracy has been a problem for a long time, as sonar images are inherently hard to model due to reflections, noise and viewpoint dependence. Autonomous Underwater Vehicles require good sonar image matching capabilities for tasks such as tracking, simultaneous localization and mapping (SLAM) and some cases of object detection/recognition. We propose the use of Convolutional Neural Networks (CNN) to learn a matching function that can be trained from labeled sonar data, after pre-processing to generate matching and non-matching pairs. In a dataset of 39K training …
I-X and Department of Electrical and Electronic Engineering, Imperial College London, UK（I-X 和 电气与电子工程系，帝国理工学院伦敦分校）；Frontier Robotics, The National Robotarium, Edinburgh UK（前沿机器人技术，国家机器人中心，爱丁堡）
SHRUMS: Sensor Hallucination for Real-time Underwater Motion Planning with a Compact 3D Sonar
Department of Computer Science and Engineering, Texas A&M University（计算机科学与工程系，德克萨斯A&M大学）；Aquaculture Robotics and Automation, SINTEF Ocean（水产养殖机器人与自动化，SINTEF海洋）
We propose a novel approach to handling the ambiguity in elevation angle associated with the observations of a forward looking multi-beam imaging sonar, and the challenges it poses for performing an accurate 3D reconstruction. We utilize a pair of sonars with orthogonal axes of uncertainty to independently observe the same points in the environment from two different perspectives, and associate these observations. Using these concurrent observations, we can create a dense, fully defined point cloud at every time-step to aid in reconstructing the 3D geometry of underwater scenes. We will evalua…
An Underwater, Fault-Tolerant, Laser-Aided Robotic Multi-Modal Dense SLAM System for Continuous Underwater In-Situ Observation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Recent work has achieved dense 3D reconstruction with wide-aperture imaging sonar using a stereo pair of orthogonally oriented sonars. This allows each sonar to observe a spatial dimension that the other is missing, without requiring any prior assumptions about scene geometry. However, this is achieved only in a small region with overlapping fields-of-view, leaving large regions of sonar image observations with an unknown elevation angle. Our work aims to achieve large-scale 3D reconstruction more efficiently using this sensor arrangement. We propose dividing the world into semantic classes to…
EROAS: 3D Efficient Reactive Obstacle Avoidance System for Autonomous Underwater Vehicles using 2.5D Forward-Looking Sonar
Department of Aerospace Eng.（航空航天工程系）；Indian Institute of Science（印度科学研究所）；Department of Electrical and Electronics Eng.（电子与电气工程系）；Birla Institute of Technology and Science（比拉理工学院）；Department of Computer Science and Eng.（计算机科学与工程系）；Shiv Nadar University（施瓦纳大学）
Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges
OceanScan Marine Systems & Technology（OceanScan海洋系统与技术公司）；INESC INOV-Lab（INESC INOV实验室）；ISRC (ISEP/P.PORTO)（ISRC（波尔图高等工程学院/波尔图大学））；Faculty of Engineering, University of Porto (FEUP)（波尔图大学工程学院（FEUP））
Brazilian Institute of Robotics, SENAI CIMATEC（巴西机器人研究所，SENAI CIMATEC）；Federal University of Bahia（巴伊亚联邦大学）；Kraken Robotik GmbH（克拉肯机器人有限公司）
Simulating sonar devices requires modeling complex underwater acoustics, simultaneously rendering time-efficient data. Existing methods focus on basic implementation of one sonar type, where most of sound properties are disregarded. In this context, this work presents a multi-device sonar simulator capable of processing an underwater scene by a hybrid pipeline on GPU: Rasterization computes the primary intersections, while only the reflective areas are ray-traced. Our proposed system launches few rays when compared to a full ray-tracing based method, achieving a significant performance gain wi…
Calibration and Comparative Analysis of Forward-Looking Sonar and 3D Sonar for Enhanced Underwater Object Recognition
College of Engineering at the University of Florida（佛罗里达大学工程学院）；Seoul National University（首尔大学）；University of South Florida（南佛罗里达大学）；University of Miami（迈阿密大学）
Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）；Faculty of Maritime Studies and Transport, University of Ljubljana（卢布尔雅那大学海事研究与运输学院）；OZON Research Group, TIMTEC（TIMTEC公司OZON研究组）；MAT Systemics, University of California, Santa Barbara（加州大学圣塔芭芭拉分校MAT系统学）
Synthetic Enclosed Echoes: A New Dataset to Mitigate the Gap Between Simulated and Real-World Sonar Data
NAUTEC - Intelligent robotics and automation group, FURG - Federal University of Rio Grande（NAUTEC智能机器人与自动化小组，联邦里约格兰德大学）；ICMC - Institute of Mathematics and Computer Sciences, USP - University of São Paulo（数学与计算机科学学院，圣保罗大学）
On-board Sonar Data Classification for Path Following in Underwater Vehicles using Fast Interval Type-2 Fuzzy Extreme Learning Machine
Hamlyn Centre for Robotic Surgery, Imperial College London（机器人手术哈明中心，帝国理工学院伦敦分校）；Center for Engineering and Industrial Development (CIDESI)（工程与工业发展中心（CIDESI））
Multi-Session Multimodal Underwater Mapping with Acoustic and Optical Imaging
Computer Vision and Robotics Research Institute (ViCOROB), University of Girona（赫罗纳大学计算机视觉与机器人研究所（ViCOROB））；University of Zagreb Faculty of Electrical Engineering and Computing（萨格勒布大学电气工程与计算学院）
University of Florida（佛罗里达大学）；Cornell University（康奈尔大学）；Naval Surface Warfare Center, Panama City Division（巴拿马城分部海军水面作战中心）；Naval Undersea Warfare Center, Newport（纽波特海军水下作战中心）
We propose a novel method to generate underwater object imagery that is acoustically compliant with that generated by side-scan sonar using the Unreal Engine. We describe the process to develop, tune, and generate imagery to provide representative images for use in training automated target recognition (ATR) and machine learning algorithms. The methods provide visual approximations for acoustic effects such as back-scatter noise and acoustic shadow, while allowing fast rendering with C++ actor in UE for maximizing the size of potential ATR training datasets. Additionally, we provide analysis o…
