Robust visual recognition in underwater environments remains a significant challenge due to complex distortions such as turbidity, low illumination, and occlusion, which severely degrade the performance of standard vision systems. This paper introduces AQUA20, a comprehensive benchmark dataset comprising 8,171 underwater images across 20 marine species reflecting real-world environmental challenges such as illumination, turbidity, occlusions, etc., providing a valuable resource for underwater visual understanding. Thirteen state-of-the-art deep learning models, including lightweight CNNs (SqueezeNet, MobileNetV2) and transformer-based architectures (ViT, ConvNeXt), were evaluated to benchmark their performance in classifying marine species under challenging conditions. Our experimental results show ConvNeXt achieving the best performance, with a Top-3 accuracy of 98.82% and a Top-1 accuracy of 90.69%, as well as the highest overall F1-score of 88.92% with moderately large parameter size. The results obtained from our other benchmark models also demonstrate trade-offs between complexity and performance. We also provide an extensive explainability analysis using GRAD-CAM and LIME for interpreting the strengths and pitfalls of the models. Our results reveal substantial room for improvement in underwater species recognition and demonstrate the value of AQUA20 as a foundation for future research in this domain. The dataset is publicly available at: https://huggingface.co/datasets/taufiktrf/AQUA20.
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Each year, underwater remotely operated vehicles (ROVs) collect thousands of hours of video of unexplored ocean habitats revealing a plethora of information regarding biodiversity on Earth. However, fully utilizing this information remains a challenge as proper annotations and analysis require trained scientists time, which is both limited and costly. To this end, we present a Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), a benchmark suite and growing large-scale dataset to train, validate, and test methods for temporally localizing four underwater substrates as well as t…
CSIRO Data61（英联邦科学与工业研究组织数据61部门）；School of Electrical Engineering and Robotics, Queensland University of Technology (QUT)（昆士兰科技大学（QUT）电气工程与机器人学院）；Queensland University of Technology (QUT)（昆士兰科技大学（QUT））
Underwater surveys conducted using divers or robots equipped with customized camera payloads can generate a large number of images. Manual review of these images to extract ecological data is prohibitive in terms of time and cost, thus providing strong incentive to automate this process using machine learning solutions. In this paper, we introduce a multi-species detector and classifier for seagrasses based on a deep convolutional neural network (achieved an overall accuracy of 92.4%). We also introduce a simple method to semi-automatically label image patches and therefore minimize manual lab…
The target of this paper is to recommend a way for Automated classification of Fish species. A high accuracy fish classification is required for greater understanding of fish behavior in Ichthyology and by marine biologists. Maintaining a ledger of the number of fishes per species and marking the endangered species in large and small water bodies is required by concerned institutions. Majority of available methods focus on classification of fishes outside of water because underwater classification poses challenges such as background noises, distortion of images, the presence of other water bod…
Detecting Endangered Marine Species in Autonomous Underwater Vehicle Imagery Using Point Annotations and Few-Shot Learning
Australian Centre for Robotics, University of Sydney（悉尼大学澳大利亚机器人中心）；Marine Technology Department, Norwegian University of Science and Technology（挪威科技大学海洋技术系）；Institute for Marine and Antarctic Studies, University of Tasmania（塔斯马尼亚大学海洋与南极研究所）
Queensland University of Technology（昆士兰科技大学）；Data61, CSIRO（联邦科学与工业研究组织数据61研究所）
Monitoring coral reefs using underwater vehicles increases the range of marine surveys and availability of historical ecological data by collecting significant quantities of images. Analysis of this imagery can be automated using a model trained to perform semantic segmentation, however it is too costly and time-consuming to densely label images for training supervised models. In this letter, we leverage photo-quadrat imagery labeled by ecologists with sparse point labels. We propose a point label aware method for propagating labels within superpixel regions to obtain augmented ground truth fo…
James Cook University（詹姆斯·库克大学）；National Research Council of Italy（意大利国家研究委员会）
Given a sufficiently large training dataset, it is relatively easy to train a modern convolution neural network (CNN) as a required image classifier. However, for the task of fish classification and/or fish detection, if a CNN was trained to detect or classify particular fish species in particular background habitats, the same CNN exhibits much lower accuracy when applied to new/unseen fish species and/or fish habitats. Therefore, in practice, the CNN needs to be continuously fine-tuned to improve its classification accuracy to handle new project-specific fish species or habitats. In this work…
Closer to Ground Truth: Realistic Shape and Appearance Labeled Data Generation for Unsupervised Underwater Image Segmentation
NORCE Norwegian Research Center AS（NORCE挪威研究中心）；National University of Science and Technology "Politehnica" Bucharest（布加勒斯特理工科学技术大学）；"Simion Stoilow" Institute of Mathematics of the Romanian Academy（罗马尼亚科学院西米翁斯托伊洛夫数学研究所）
SharkTrack: an accurate, generalisable software for streamlining shark and ray underwater video analysis
Imperial College London（帝国理工学院）；Shark Measurements（鲨鱼测量公司）；Virginia Tech（弗吉尼亚理工大学）；James Cook University（詹姆斯库克大学）；Florida International University（佛罗里达国际大学）；University of Exeter（埃克塞特大学）；Anguilla National Trust（安圭拉国家信托基金）；Government of Anguilla（安圭拉政府）；Bangor University（班戈大学）；University of Leeds（利兹大学）
Ocean4D: Generative Underwater 4D Reconstruction via Medium-Aware Video Diffusion
Faculty of Information Science and Engineering, Ocean University of China（中国海洋大学信息科学与工程学部）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Overcoming Annotation Bottlenecks in Underwater Fish Segmentation: A Robust Self-Supervised Learning Approach
College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；ARC Research Hub for Supercharging Tropical Aquaculture through Genetic Solutions, James Cook University（詹姆斯库克大学ARC热带水产养殖基因解决方案研究中心）
City University of Hong Kong（香港城市大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Tianjin University（天津大学）；Beijing Jiaotong University（北京交通大学）
Underwater image enhancement has been attracting much attention due to its significance in marine engineering and aquatic robotics. Numerous underwater image enhancement algorithms have been proposed in the last few years. However, these algorithms are mainly evaluated using either synthetic datasets or few selected real-world images. It is thus unclear how these algorithms would perform on images acquired in the wild and how we could gauge the progress in the field. To bridge this gap, we present the first comprehensive perceptual study and analysis of underwater image enhancement using large…
Underwater image enhancement is such an important low-level vision task with many applications that numerous algorithms have been proposed in recent years. These algorithms developed upon various assumptions demonstrate successes from various aspects using different data sets and different metrics. In this work, we setup an undersea image capturing system, and construct a large-scale Real-world Underwater Image Enhancement (RUIE) data set divided into three subsets. The three subsets target at three challenging aspects for enhancement, i.e., image visibility quality, color casts, and higher-le…
Underwater visual perception is essentially important for underwater exploration, archeology, ecosystem and so on. The low illumination, light reflections, scattering, absorption and suspended particles inevitably lead to the critically degraded underwater image quality, which causes great challenges on recognizing the objects from the underwater images. The existing underwater enhancement methods that aim to promote the underwater visibility, heavily suffer from the poor image restoration performance and generalization ability. To reduce the difficulty of underwater image enhancement, we intr…
AquaFeat+: an Underwater Vision Learning-based Enhancement Method for Object Detection, Classification, and Tracking
Centro de Ciências Computacionais (C3), Universidade Federal do Rio Grande (FURG), Brazil（巴西里约格兰德联邦大学计算科学研究中心（C3））；Instituto de Ciências Exatas e Tecnologia (ICET), Universidade Federal do Amazonas (UFAM), Brazil（巴西亚马逊联邦大学精确科学与技术研究所（ICET））
Marine scientists use remote underwater video recording to survey fish species in their natural habitats. This helps them understand and predict how fish respond to climate change, habitat degradation, and fishing pressure. This information is essential for developing sustainable fisheries for human consumption, and for preserving the environment. However, the enormous volume of collected videos makes extracting useful information a daunting and time-consuming task for a human. A promising method to address this problem is the cutting-edge Deep Learning (DL) technology.DL can help marine scien…
Enhancing Underwater Light Field Images via Global Geometry-aware Diffusion Process
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学学院）；School of Mathematics and Statistics and the Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University（西安交通大学数学与统计学学院和教育部智能网络与网络安全重点实验室）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Macao Institute of Systems Engineering, Macau University of Science and Technology（澳门系统工程研究院，澳门科学大学）
Depth estimation is critical for any robotic system. In the past years estimation of depth from monocular images have shown great improvement, however, in the underwater environment results are still lagging behind due to appearance changes caused by the medium. So far little effort has been invested on overcoming this. Moreover, underwater, there are more limitations for using high resolution depth sensors, this makes generating ground truth for learning methods another enormous obstacle. So far unsupervised methods that tried to solve this have achieved very limited success as they relied on…
In this paper we present a complete framework for Underwater SLAM utilizing a single inexpensive sensor. Over the recent years, imaging technology of action cameras is producing stunning results even under the challenging conditions of the underwater domain. The GoPro 9 camera provides high definition video in synchronization with an Inertial Measurement Unit (IMU) data stream encoded in a single mp4 file. The visual inertial SLAM framework is augmented to adjust the map after each loop closure. Data collected at an artificial wreck of the coast of South Carolina and in caverns and caves in Fl…
Griffith University（格里菲斯大学）；Australian Rivers Institute（澳大利亚河流研究所）；University of the Sunshine Coast（阳光海岸大学）
Uses of underwater videos to assess diversity and abundance of fish are being rapidly adopted by marine biologists. Manual processing of videos for quantification by human analysts is time and labour intensive. Automatic processing of videos can be employed to achieve the objectives in a cost and time-efficient way. The aim is to build an accurate and reliable fish detection and recognition system, which is important for an autonomous robotic platform. However, there are many challenges involved in this task (e.g. complex background, deformation, low resolution and light propagation). Recent a…
In the context of robotic underwater operations, the visual degradations induced by the medium properties make difficult the exclusive use of cameras for localization purpose. Hence, most localization methods are based on expensive navigational sensors associated with acoustic positioning. On the other hand, visual odometry and visual SLAM have been exhaustively studied for aerial or terrestrial applications, but state-of-the-art algorithms fail underwater. In this paper we tackle the problem of using a simple low-cost camera for underwater localization and propose a new monocular visual odome…
Automatic Coral Detection with YOLO: A Deep Learning Approach for Efficient and Accurate Coral Reef Monitoring
Cadi Ayyad University（卡迪·阿雅德大学）；University of La Réunion（留尼汪大学）；IRD（法国国家可持续发展研究所）；IFREMER（法国海洋开发研究院）；CNRS（法国国家科学研究中心）；University of New Caledonia（新喀里多尼亚大学）；Institut de Recherche pour le Développement（法国国家可持续发展研究所）；MNHN（法国国家自然历史博物馆）；EPHE（巴黎高等研究实践学院）
Underwater automatic target recognition (UATR) has been a challenging research topic in ocean engineering. Although deep learning brings opportunities for target recognition on land and in the air, underwater target recognition techniques based on deep learning have lagged due to sensor performance and the size of trainable data. This letter proposed a framework for learning the visual representation of underwater acoustic imageries, which takes a transformer-based style transfer model as the main body. It could replace the low-level texture features of optical images with the visual features …
University of Victoria（维多利亚大学）；ASL Environmental Sciences（ASL环境科学公司）；Fisheries and Oceans Canada（加拿大渔业和海洋部）
Tracking the abundance of underwater species is crucial for understanding the effects of climate change on marine ecosystems. Biologists typically monitor underwater sites with echosounders and visualize data as 2D images (echograms); they interpret these data manually or semi-automatically, which is time-consuming and prone to inconsistencies. This paper proposes a deep learning framework for the automatic detection of schools of herring from echograms. Experiments demonstrated that our approach outperforms a traditional machine learning algorithm using hand-crafted features. Our framework co…
DUViN: Diffusion-Based Underwater Visual Navigation via Knowledge-Transferred Depth Features
Department of Electrical and Electronic Engineering, The University of Melbourne, Australia（墨尔本大学电子与电气工程系）；School of Mathematics and Statistics, The University of Melbourne, Australia（墨尔本大学数学与统计学学院）
Visibility underwater is challenging, and degrades as the distance between the subject and camera increases, making vision tasks in the forward-looking direction more difficult. We have collected underwater forward-looking stereo-vision and visual-inertial image sets in the Mediterranean and Red Sea. To our knowledge there are no other public datasets in the underwater environment acquired with this camera-sensor orientation published with ground-truth. These datasets are critical for the development of several underwater applications, including obstacle avoidance, visual odometry, 3D tracking…
Investigation of the Challenges of Underwater-Visual-Monocular-SLAM
Oceanic Machine Vision, GEOMAR Helmholtz Centre for Ocean Research Kiel（海洋机器视觉，基尔海洋研究所）；Marine Data Science, Department of Computer Science, Christian-Albrechts-Universität zu Kiel（海洋数据科学，计算机科学系，基尔亚历山大-冯-洪堡大学）；Computer Vision and Robotics Research Institute (VICOROB), University of Girona（计算机视觉与机器人研究 institute（VICOROB），加泰罗尼亚大学）
University of California, Santa Barbara（加州大学圣巴巴拉分校）
The quality and size of training sets often limit the performance of many state of the art object detectors. However, in many scenarios, it can be difficult to collect images for training, not to mention the costs associated with collecting annotations suitable for training these object detectors. For these reasons, on challenging video datasets such as the Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), budgets may only allow for collecting and providing partial annotations. To aid in the challenges associated with training with limited and partial annotations, we introduc…
This work reviews the problem of object detection in underwater environments. We analyse and quantify the shortcomings of conventional state-of-the-art (SOTA) algorithms in the computer vision community when applied to this challenging environment, as well as providing insights and general guidelines for future research efforts. First, we assessed if pretraining with the conventional ImageNet is beneficial when the object detector needs to be applied to environments that may be characterised by a different feature distribution. We then investigate whether two-stage detectors yields to better p…
Shanghai Jiao Tong University（上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；CloudWalk Technology Co., Ltd（云walk科技有限公司）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
