This work reviews the problem of object detection in underwater environments. We analyse and quantify the shortcomings of conventional state-of-the-art (SOTA) algorithms in the computer vision community when applied to this challenging environment, as well as providing insights and general guidelines for future research efforts. First, we assessed if pretraining with the conventional ImageNet is beneficial when the object detector needs to be applied to environments that may be characterised by a different feature distribution. We then investigate whether two-stage detectors yields to better performance with respect to single-stage detectors, in terms of accuracy, intersection of union (IoU), floating operation per second (FLOPS), and inference time. Finally, we assessed the generalisation capability of each model to a lower quality dataset to simulate performance on a real scenario, in which harsher conditions ought to be expected. Our experimental results provide evidence that underwater object detection requires searching for "ad-hoc" architectures than merely training SOTA architectures on new data, and that pretraining is not beneficial.
Nankai University（南开大学）；Tianjin University of Finance and Economics（天津财经大学）；Tsinghua University（清华大学）
We have made a dataset of camouflage object detection mainly for complex seabed scenes, and named it UnderWater RGB&Sonar,or UW-RS for short. The UW-RS dataset contains a total of 1972 image data. The dataset mainly consists of two parts, namely underwater optical data part (UW-R dataset) and underwater sonar data part (UW-S dataset).
A General Underwater Object Detector (GUOD) should perform well on most of underwater circumstances. However, with limited underwater dataset, conventional object detection methods suffer from domain shift severely. This paper aims to build a GUOD with small underwater dataset with limited types of water quality. First, we propose a data augmentation method Water Quality Transfer (WQT) to increase domain diversity of the original small dataset. Second, for mining the semantic information from data generated by WQT, DG-YOLO is proposed, which consists of three parts: YOLOv3, DIM and IRM penalty…
With the development of underwater object grabbing technology, underwater object recognition and segmentation of high accuracy has become a challenge. The existing underwater object detection technology can only give the general position of an object, unable to give more detailed information such as the outline of the object, which seriously affects the grabbing efficiency. To address this problem, we label and establish the first underwater semantic segmentation dataset of real scene(DUT-USEG:DUT Underwater Segmentation Dataset). The DUT- USEG dataset includes 6617 images, 1487 of which have …
Jacobs University Bremen gGmbH（不来梅雅各布大学）；National Research Council of Italy (CNR)（意大利国家研究委员会）；University of Zagreb（萨格勒布大学）
In this article we present a novel underwater dataset collected from several field trials within the EU FP7 project "Cognitive autonomous diving buddy (CADDY)", where an Autonomous Underwater Vehicle (AUV) was used to interact with divers and monitor their activities. To our knowledge, this is one of the first efforts to collect a large dataset in underwater environments targeting object classification, segmentation and human pose estimation tasks. The first part of the dataset contains stereo camera recordings (~10K) of divers performing hand gestures to communicate and interact with an AUV i…
University of Leicester（莱斯特大学）；Ocean University of China（中国海洋大学）
Underwater object detection technique is of great significance for various applications in underwater the scenes. However, class imbalance issue is still an unsolved bottleneck for current underwater object detection algorithms. It leads to large precision discrepancies among different classes that the dominant classes with more training data achieve higher detection precisions while the minority classes with fewer training data achieves much lower detection precisions. In this paper, we propose a novel class-wise style augmentation (CWSA) algorithm to generate a class-balanced underwater data…
AquaFeat+: an Underwater Vision Learning-based Enhancement Method for Object Detection, Classification, and Tracking
Centro de Ciências Computacionais (C3), Universidade Federal do Rio Grande (FURG), Brazil（巴西里约格兰德联邦大学计算科学研究中心（C3））；Instituto de Ciências Exatas e Tecnologia (ICET), Universidade Federal do Amazonas (UFAM), Brazil（巴西亚马逊联邦大学精确科学与技术研究所（ICET））
Repair and maintenance of underwater structures as well as marine science rely heavily on the results of underwater object detection, which is a crucial part of the image processing workflow. Although many computer vision-based approaches have been presented, no one has yet developed a system that reliably and accurately detects and categorizes objects and animals found in the deep sea. This is largely due to obstacles that scatter and absorb light in an underwater setting. With the introduction of deep learning, scientists have been able to address a wide range of issues, including safeguardi…
U-DECN: End-to-End Underwater Object Detection ConvNet with Improved DeNoising Training
National Key Laboratory of Autonomous Marine Vehicle Technology, Harbin Engineering University（自主水下机动车辆技术国家重点实验室，哈尔滨工程大学）；Hong Kong Polytechnic University（香港理工大学）
LUIEO: A Lightweight Model for Integrating Underwater Image Enhancement and Object Detection
School of Mathematics, North University of China（中北大学数学学院）；College of Computer and Information Technology, Shanxi University（山西大学计算机与信息技术学院）；School of Mathematics Sciences, Beijing Normal University（北京师范大学数学科学学院）
Rethinking Conditional Generation for Underwater Salient Object Detection
School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Shandong University（山东大学机器智能与系统控制教育部重点实验室）；School of Data Science, Lingnan University（岭南大学数据科学学院）
Object detection is a very important basic research direction in the field of computer vision and a basic method for other advanced tasks in the field of computer vision. It has been widely used in practical applications such as object tracking, video behavior recognition and underwater robotics vision. The Cascade-RCNN and Deformable Convolution Network are both classical and excellent object detection algorithms. In this report, we evaluate our Cascade-DCN based method on underwater optical image and acoustics image datasets with different engineering tricks and augumentation.
Macau University of Science and Technology（澳门科技大学）；Peking University（北京大学）
Self-attention is one of the most successful designs in deep learning, which calculates the similarity of different tokens and reconstructs the feature based on the attention matrix. Originally designed for NLP, self-attention is also popular in computer vision, and can be categorized into pixel-level attention and patch-level attention. In object detection, RoI features can be seen as patches from base feature maps. This paper aims to apply the attention module to RoI features to improve performance. Instead of employing an original self-attention module, we choose the external attention modu…
Complicated underwater environments bring new challenges to object detection, such as unbalanced light conditions, low contrast, occlusion, and mimicry of aquatic organisms. Under these circumstances, the objects captured by the underwater camera will become vague, and the generic detectors often fail on these vague objects. This work aims to solve the problem from two perspectives: uncertainty modeling and hard example mining. We propose a two-stage underwater detector named boosting R-CNN, which comprises three key components. First, a new region proposal network named RetinaRPN is proposed,…
WaterFlow: Explicit Physics-Prior Rectified Flow for Underwater Saliency Mask Generation
Hainan University, China（海南大学）；Huazhong University of Science and Technology, China（华中科技大学）；Shenzhen University, China（深圳大学）；Lingnan University, Hong Kong（岭南大学）
Deep learning-based underwater object detection (UOD) remains a major challenge due to the degraded visibility and difficulty to obtain sufficient underwater object images captured from various perspectives for training. To address these issues, this paper presents a high-precision UOD based on joint self-supervised deblurring and improved spatial transformer network. A self-supervised deblurring subnetwork is introduced into the designed multi-task learning aided object detection architecture to force the shared feature extraction module to output clean features for detection subnetwork. Aimi…
National Engineering Laboratory for Video Technology - Shenzhen Division（视频技术国家工程实验室深圳分室）
Generic object detection algorithms have proven their excellent performance in recent years. However, object detection on underwater datasets is still less explored. In contrast to generic datasets, underwater images usually have color shift and low contrast; sediment would cause blurring in underwater images. In addition, underwater creatures often appear closely to each other on images due to their living habits. To address these issues, our work investigates augmentation policies to simulate overlapping, occluded and blurred objects, and we construct a model capable of achieving better gene…
In recent years, deep learning based methods have achieved promising performance in standard object detection. However, these methods lack sufficient capabilities to handle underwater object detection due to these challenges: (1) Objects in real applications are usually small and their images are blurry, and (2) images in the underwater datasets and real applications accompany heterogeneous noise. To address these two problems, we first propose a novel neural network architecture, namely Sample-WeIghted hyPEr Network (SWIPENet), for small object detection. SWIPENet consists of high resolution …
DUT-RU International School of Information Science & Engineering, Dalian University of Technology（大连理工大学DUT-RU国际信息科学与工程学院）
Underwater object detection for robot picking has attracted a lot of interest. However, it is still an unsolved problem due to several challenges. We take steps towards making it more realistic by addressing the following challenges. Firstly, the currently available datasets basically lack the test set annotations, causing researchers must compare their method with other SOTAs on a self-divided test set (from the training set). Training other methods lead to an increase in workload and different researchers divide different datasets, resulting there is no unified benchmark to compare the perfo…
University of Florida（佛罗里达大学）；University of Minnesota, Twin Cities（明尼苏达大学双城分校）
This paper presents a holistic approach to saliency-guided visual attention modeling (SVAM) for use by autonomous underwater robots. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective salient object detection (SOD) in natural underwater images. The SVAM-Net architecture is configured in a unique way to jointly accommodate bottom-up and top-down learning within two separate branches of the network while sharing the same encoding layers. We design dedicated spatial attention modules (SAMs) along these learning pathways to exploit the…
Improving underwater semantic segmentation with underwater image quality attention and muti-scale aggregation attention
School of Computer Science and Engineering, Jiangsu University of Science and Technology（江苏科技大学计算机科学与工程学院）；School of Electronic and Informatics Engineering, Jiangsu University（江苏大学电子与信息工程学院）；School of Automation, Southeast University（东南大学自动化学院）
This study examines the effectiveness of spatio-temporal modeling and the integration of spatial attention mechanisms in deep learning models for underwater object detection. Specifically, in the first phase, the performance of temporal-enhanced YOLOv5 variant T-YOLOv5 is evaluated, in comparison with the standard YOLOv5. For the second phase, an augmented version of T-YOLOv5 is developed, through the addition of a Convolutional Block Attention Module (CBAM). By examining the effectiveness of the already pre-existing YOLOv5 and T-YOLOv5 models and of the newly developed T-YOLOv5 with CBAM. Wit…
Learning Heavily-Degraded Prior for Underwater Object Detection
DUT-RU International School of Information Science, Dalian University of Technology（大连理工大学-俄罗斯国立高等经济大学信息科学国际学院）；Dalian University of Technology（大连理工大学）
Adaptive Enhancement and Dual-Pooling Sequential Attention for Lightweight Underwater Object Detection with YOLOv10
1,2Department of Computer Science；Engineering, Dhaka University of Engineering \& Technology Gazipur, 1707, Bangladesh 3Department of Computer Science；Engineering, BRAC University Dhaka, Bangladesh
Peking University（北京大学）；Chongqing University of Technology（重庆理工大学）；Peng Cheng Laboratory（鹏城实验室）
The performance of existing underwater object detection methods degrades seriously when facing domain shift caused by complicated underwater environments. Due to the limitation of the number of domains in the dataset, deep detectors easily memorize a few seen domains, which leads to low generalization ability. There are two common ideas to improve the domain generalization performance. First, it can be inferred that the detector trained on as many domains as possible is domain-invariant. Second, for the images with the same semantic content in different domains, their hidden features should be…
BG-YOLO: A Bidirectional-Guided Method for Underwater Object Detection
School of Tropical Agriculture and Forestry, Hainan University（海南大学热带农林学院）；School of Information and Communication Engineering（信息与通信工程学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
Expose Camouflage in the Water: Underwater Camouflaged Instance Segmentation and Dataset
School of Information and Communication Engineering, Hainan University（海南大学信息与通信工程学院）；School of Electronic and Information Engineering, Guangdong Ocean University（广东海洋大学电子与信息工程学院）；School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；School of Computer Science, Nankai University（南开大学计算机科学学院）；Institute of Software, Chinese Academy of Science（中国科学院软件研究所）；School of Data Science, Lingnan University（岭南大学数据科学学院）
To assist underwater object detection for better performance, image enhancement technology is often used as a pre-processing step. However, most of the existing enhancement methods tend to pursue the visual quality of an image, instead of providing effective help for detection tasks. In fact, image enhancement algorithms should be optimized with the goal of utility improvement. In this paper, to adapt to the underwater detection tasks, we proposed a lightweight dynamic enhancement algorithm using a contribution dictionary to guide low-level corrections. Dynamic solutions are designed to captur…
Learning Spatially Adaptive Structural Coordination for Underwater Salient Object Detection
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；School of Robotics and Advanced Manufacture, Harbin Institute of Technology（机器人与先进制造学院，哈尔滨工业大学）；School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）；College of Computer Science & Visual Computing and Intelligent Perception Lab, Nankai University（计算机科学与视觉计算学院及智能感知实验室，南开大学）；School of Cyber Science and Technology, Sun Yat-sen University（网络科学与技术学院，中山大学）；School of Automation, Southeast University（自动化学院，东南大学）
Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future
University College London（伦敦大学学院）；Dalian University of Technology（大连理工大学）；Ocean University of China（中国海洋大学）；Lingnan University（岭南大学）；Southeast University（东南大学）；Nankai University（南开大学）
SU-YOLO: Spiking Neural Network for Efficient Underwater Object Detection
College of Computer and Information Technology, China Three Gorges University（三峡大学计算机与信息学院）；Wuhan University of Technology（武汉理工大学）；Peking University（北京大学）
University of Leicester（莱斯特大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Shaoxing University（绍兴文理学院）；Ocean University of China（中国海洋大学）；Hohai University（河海大学）
In recent years, deep learning based object detection methods have achieved promising performance in controlled environments. However, these methods lack sufficient capabilities to handle underwater object detection due to these challenges: (1) images in the underwater datasets and real applications are blurry whilst accompanying severe noise that confuses the detectors and (2) objects in real applications are usually small. In this paper, we propose a novel Sample-WeIghted hyPEr Network (SWIPENET), and a robust training paradigm named Curriculum Multi-Class Adaboost (CMA), to address these tw…
You Sense Only Once Beneath: Ultra-Light Real-Time Underwater Object Detection
School of Data Science and Engineering, and Xingzhi College, South China Normal University（数据科学与工程学院和行知学院，华南师范大学）；School of Physics, South China Normal University（物理学院，华南师范大学）；School of Electronics and Information Engineering, and Xingzhi College, South China Normal University（电子与信息工程学院和行知学院，华南师范大学）
SVGS-DSGAT: An IoT-Enabled Innovation in Underwater Robotic Object Detection Technology
College of Design and Engineering, National University of Singapore（新加坡国立大学设计与工程学院）；Institute of Semiconductors, Chinese Academy of Sciences（中国科学院半导体研究所）；Beijing Ratu Technology Co., Ltd.（北京瑞途科技有限公司）
K J Somaiya College Of Engineering（K J Somaiya工程学院）
The paper deals with the software and the electronics unit for an autonomous underwater vehicle. The implementation in the electronics unit is the connection and communication between SBC, pixhawk controller and other sensory hardware and actuators. The major implementation of the software unit is the algorithm for object detection based on Convolutional Neural Network (CNN) and its models. The Hyperparameters were tuned according to Odroid Xu4 for various models. The maneuvering algorithm uses the MAVLink protocol of the ArduSub project for movement and its simulation.
University of California, Santa Barbara（加州大学圣巴巴拉分校）
The quality and size of training sets often limit the performance of many state of the art object detectors. However, in many scenarios, it can be difficult to collect images for training, not to mention the costs associated with collecting annotations suitable for training these object detectors. For these reasons, on challenging video datasets such as the Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), budgets may only allow for collecting and providing partial annotations. To aid in the challenges associated with training with limited and partial annotations, we introduc…
