使用声导航与测距（SOund Navigation and Ranging, SONAR）图像进行地点识别是水下环境中同时定位与建图（simultaneous localization and mapping, SLAM）的一项重要任务。本文提出了一种鲁棒高效的基于成像SONAR的地点识别、SONAR上下文与回环检测方法。与现有方法不同，我们的方法无需先验知识或训练，根据原始SONAR测量值的特性对几何信息进行编码。我们还设计了分层搜索流程以快速检索候选SONAR帧，并采用自适应偏移与填充实现对旋转和平移变化的鲁棒匹配。此外，我们可通过自适应偏移推导初始位姿，并将其应用于基于迭代最近点（iterative closest point, ICP）的回环因子。我们在模拟开阔水域、真实水箱及真实水下环境等多种水下序列中评估了SONAR上下文的性能。所提方法在多个数据集与评估指标上展现出地点识别的鲁棒性与性能提升。补充材料可在https://github.com/sparolab/sonar_context.git获取。
英文摘要：
Place recognition using SOund Navigation and Ranging (SONAR) images is an important task for simultaneous localization and mapping(SLAM) in underwater environments. This paper proposes a robust and efficient imaging SONAR based place recognition, SONAR context, and loop closure method. Unlike previous methods, our approach encodes geometric information based on the characteristics of raw SONAR measurements without prior knowledge or training. We also design a hierarchical searching procedure for fast retrieval of candidate SONAR frames and apply adaptive shifting and padding to achieve robust matching on rotation and translation changes. In addition, we can derive the initial pose through adaptive shifting and apply it to the iterative closest point (ICP) based loop closure factor. We evaluate the performance of SONAR context in the various underwater sequences such as simulated open water, real water tank, and real underwater environments. The proposed approach shows the robustness and improvements of place recognition on various datasets and evaluation metrics. Supplementary materials are available at https://github.com/sparolab/sonar_context.git.
RUSSO: Robust Underwater SLAM with Sonar Optimization against Visual Degradation
School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院）；School of Electrical and Information Engineering, Tianjin University（天津大学电气自动化与信息工程学院）
University of South Carolina（南卡罗来纳大学）；Dartmouth College（达特茅斯学院）
This paper presents a novel tightly-coupled keyframe-based Simultaneous Localization and Mapping (SLAM) system with loop-closing and relocalization capabilities targeted for the underwater domain. Our previous work, SVIn, augmented the state-of-the-art visual-inertial state estimation package OKVIS to accommodate acoustic data from sonar in a non-linear optimization-based framework. This paper addresses drift and loss of localization -- one of the main problems affecting other packages in underwater domain -- by providing the following main contributions: a robust initialization method to refi…
Underwater Dense Mapping with the First Compact 3D Sonar
Mechanical Engineering Department, University of Delaware（德克萨斯大学机械工程系）；Dartmouth College（达特茅斯学院）；Aquaculture Robotics and Automation Group, SINTEF Ocean（SINTEF海洋研究所水产机器人与自动化小组）
Stevens Institute of Technology（斯蒂文斯理工学院）；Exyn Technologies（埃克森科技公司）
Simultaneous localization and mapping (SLAM) is a critical capability for any autonomous underwater vehicle (AUV). However, robust, accurate state estimation is still a work in progress when using low-cost sensors. We propose enhancing a typical low-cost sensor package using widely available and often free prior information; overhead imagery. Given an AUV's sonar image and a partially overlapping, globally-referenced overhead image, we propose using a convolutional neural network (CNN) to generate a synthetic overhead image predicting the above-surface appearance of the sonar image contents. W…
An essential task for a multi-robot system is generating a common understanding of the environment and relative poses between robots. Cooperative tasks can be executed only when a vehicle has knowledge of its own state and the states of the team members. However, this has primarily been achieved with direct rendezvous between underwater robots, via inter-robot ranging. We propose a novel distributed multi-robot simultaneous localization and mapping (SLAM) framework for underwater robots using imaging sonar-based perception. By passing only scene descriptors between robots, we do not need to pa…
University of Electronic Science and Technology of China（电子科技大学）；Dartmouth College（达特茅斯学院）
Underwater navigation presents several challenges, including unstructured unknown environments, lack of reliable localization systems (e.g., GPS), and poor visibility. Furthermore, good-quality obstacle detection sensors for underwater robots are scant and costly; and many sensors like RGB-D cameras and LiDAR only work in-air. To enable reliable mapless underwater navigation despite these challenges, we propose a low-cost end-to-end navigation system, based on a monocular camera and a fixed single-beam echo-sounder, that efficiently navigates an underwater robot to waypoints while avoiding nea…
VIMS: A Visual-Inertial-Magnetic-Sonar SLAM System in Underwater Environments
Key Laboratory of Ocean Observation-Imaging Testbed of Zhejiang Province, Zhejiang University（浙江省海洋观测-成像测试床重点实验室，浙江大学）；Interdisciplinary Student Training Platform for Marine areas, Zhejiang University（海洋领域交叉人才培养平台，浙江大学）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科技大学）；The Institute of Deep-Sea Science and Engineering, Chinese Academy of Sciences（深海科学与工程研究院，中国科学院）
Stevens Institute of Technology（史蒂文斯理工学院）；Amazon Robotics（亚马逊机器人）；Exyn Technologies（Exyn科技）；SRI International（斯坦福国际研究院）
We consider the problem of autonomous mobile robot exploration in an unknown environment, taking into account a robot's coverage rate, map uncertainty, and state estimation uncertainty. This paper presents a novel exploration framework for underwater robots operating in cluttered environments, built upon simultaneous localization and mapping (SLAM) with imaging sonar. The proposed system comprises path generation, place recognition forecasting, belief propagation and utility evaluation using a virtual map, which estimates the uncertainty associated with map cells throughout a robot's workspace…
Kongju National University（公州大学）；ETRI（韩国电子通信研究院）；KAIST（韩国科学技术院）
Among underwater perceptual sensors, imaging sonar has been highlighted for its perceptual robustness underwater. The major challenge of imaging sonar, however, arises from the difficulty in defining visual features despite limited resolution and high noise levels. Recent developments in deep learning provide a powerful solution for computer-vision researches using optical images. Unfortunately, deep learning-based approaches are not well established for imaging sonars, mainly due to the scant data in the training phase. Unlike the abundant publically available terrestrial images, obtaining un…
This article focuses on feature-based underwater localization and navigation for autonomous underwater vehicles (AUVs) using 2D imaging sonar measurements. The sparsity of underwater acoustic features and the loss of elevation angle in sonar images may introduce wrong feature matches or insufficient features for optimization-based underwater localization (i.e. under-constrained/degeneracy cases). This motivates us to propose a novel inertial-aided sliding window optimization framework to improve the estimation accuracy and the robustness to front-end outliers. Concretely, we first discriminate…
Matching sonar images with high accuracy has been a problem for a long time, as sonar images are inherently hard to model due to reflections, noise and viewpoint dependence. Autonomous Underwater Vehicles require good sonar image matching capabilities for tasks such as tracking, simultaneous localization and mapping (SLAM) and some cases of object detection/recognition. We propose the use of Convolutional Neural Networks (CNN) to learn a matching function that can be trained from labeled sonar data, after pre-processing to generate matching and non-matching pairs. In a dataset of 39K training …
I-X and Department of Electrical and Electronic Engineering, Imperial College London, UK（I-X 和 电气与电子工程系，帝国理工学院伦敦分校）；Frontier Robotics, The National Robotarium, Edinburgh UK（前沿机器人技术，国家机器人中心，爱丁堡）
SHRUMS: Sensor Hallucination for Real-time Underwater Motion Planning with a Compact 3D Sonar
Department of Computer Science and Engineering, Texas A&M University（计算机科学与工程系，德克萨斯A&M大学）；Aquaculture Robotics and Automation, SINTEF Ocean（水产养殖机器人与自动化，SINTEF海洋）
We propose a novel approach to handling the ambiguity in elevation angle associated with the observations of a forward looking multi-beam imaging sonar, and the challenges it poses for performing an accurate 3D reconstruction. We utilize a pair of sonars with orthogonal axes of uncertainty to independently observe the same points in the environment from two different perspectives, and associate these observations. Using these concurrent observations, we can create a dense, fully defined point cloud at every time-step to aid in reconstructing the 3D geometry of underwater scenes. We will evalua…
Recent work has achieved dense 3D reconstruction with wide-aperture imaging sonar using a stereo pair of orthogonally oriented sonars. This allows each sonar to observe a spatial dimension that the other is missing, without requiring any prior assumptions about scene geometry. However, this is achieved only in a small region with overlapping fields-of-view, leaving large regions of sonar image observations with an unknown elevation angle. Our work aims to achieve large-scale 3D reconstruction more efficiently using this sensor arrangement. We propose dividing the world into semantic classes to…
EROAS: 3D Efficient Reactive Obstacle Avoidance System for Autonomous Underwater Vehicles using 2.5D Forward-Looking Sonar
Department of Aerospace Eng.（航空航天工程系）；Indian Institute of Science（印度科学研究所）；Department of Electrical and Electronics Eng.（电子与电气工程系）；Birla Institute of Technology and Science（比拉理工学院）；Department of Computer Science and Eng.（计算机科学与工程系）；Shiv Nadar University（施瓦纳大学）
Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges
OceanScan Marine Systems & Technology（OceanScan海洋系统与技术公司）；INESC INOV-Lab（INESC INOV实验室）；ISRC (ISEP/P.PORTO)（ISRC（波尔图高等工程学院/波尔图大学））；Faculty of Engineering, University of Porto (FEUP)（波尔图大学工程学院（FEUP））
Brazilian Institute of Robotics, SENAI CIMATEC（巴西机器人研究所，SENAI CIMATEC）；Federal University of Bahia（巴伊亚联邦大学）；Kraken Robotik GmbH（克拉肯机器人有限公司）
Simulating sonar devices requires modeling complex underwater acoustics, simultaneously rendering time-efficient data. Existing methods focus on basic implementation of one sonar type, where most of sound properties are disregarded. In this context, this work presents a multi-device sonar simulator capable of processing an underwater scene by a hybrid pipeline on GPU: Rasterization computes the primary intersections, while only the reflective areas are ray-traced. Our proposed system launches few rays when compared to a full ray-tracing based method, achieving a significant performance gain wi…
Calibration and Comparative Analysis of Forward-Looking Sonar and 3D Sonar for Enhanced Underwater Object Recognition
College of Engineering at the University of Florida（佛罗里达大学工程学院）；Seoul National University（首尔大学）；University of South Florida（南佛罗里达大学）；University of Miami（迈阿密大学）
Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）；Faculty of Maritime Studies and Transport, University of Ljubljana（卢布尔雅那大学海事研究与运输学院）；OZON Research Group, TIMTEC（TIMTEC公司OZON研究组）；MAT Systemics, University of California, Santa Barbara（加州大学圣塔芭芭拉分校MAT系统学）
Synthetic Enclosed Echoes: A New Dataset to Mitigate the Gap Between Simulated and Real-World Sonar Data
NAUTEC - Intelligent robotics and automation group, FURG - Federal University of Rio Grande（NAUTEC智能机器人与自动化小组，联邦里约格兰德大学）；ICMC - Institute of Mathematics and Computer Sciences, USP - University of São Paulo（数学与计算机科学学院，圣保罗大学）
Visibility underwater is challenging, and degrades as the distance between the subject and camera increases, making vision tasks in the forward-looking direction more difficult. We have collected underwater forward-looking stereo-vision and visual-inertial image sets in the Mediterranean and Red Sea. To our knowledge there are no other public datasets in the underwater environment acquired with this camera-sensor orientation published with ground-truth. These datasets are critical for the development of several underwater applications, including obstacle avoidance, visual odometry, 3D tracking…
On-board Sonar Data Classification for Path Following in Underwater Vehicles using Fast Interval Type-2 Fuzzy Extreme Learning Machine
Hamlyn Centre for Robotic Surgery, Imperial College London（机器人手术哈明中心，帝国理工学院伦敦分校）；Center for Engineering and Industrial Development (CIDESI)（工程与工业发展中心（CIDESI））
Multi-Session Multimodal Underwater Mapping with Acoustic and Optical Imaging
Computer Vision and Robotics Research Institute (ViCOROB), University of Girona（赫罗纳大学计算机视觉与机器人研究所（ViCOROB））；University of Zagreb Faculty of Electrical Engineering and Computing（萨格勒布大学电气工程与计算学院）
