Underwater Image Enhancement Based on Structure-Texture Reconstruction
School of Automation and Electrical Engineering, Shenyang Ligong University（沈阳理工大学自动化与电气工程学院）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；School of Electronic and Information Engineering, Liaoning Technical University（辽宁工程技术大学电子与信息工程学院）
Aiming at the problems of color distortion, blur and excessive noise of underwater image, an underwater image enhancement algorithm based on structure-texture reconstruction is proposed. Firstly, the color equalization of the degraded image is realized by the automatic color enhancement algorithm; Secondly, the relative total variation is introduced to decompose the image into the structure layer and texture layer; Then, the best background light point is selected based on brightness, gradient discrimination, and hue judgment, the transmittance of the backscatter component is obtained by the red dark channel prior, which is substituted into the imaging model to remove the fogging phenomenon in the structure layer. Enhancement of effective details in the texture layer by multi scale detail enhancement algorithm and binary mask; Finally, the structure layer and texture layer are reconstructed to get the final image. The experimental results show that the algorithm can effectively balance the hue, saturation, and clarity of underwater image, and has good performance in different underwater environments.
Tianjin University（天津大学）；Southern University of Science and Technology（南方科技大学）
The challenges in recovering underwater images are the presence of diverse degradation factors and the lack of ground truth images. Although synthetic underwater image pairs can be used to overcome the problem of inadequately observing data, it may result in over-fitting and enhancement degradation. This paper proposes a model-based deep learning method for restoring clean images under various underwater scenarios, which exhibits good interpretability and generalization ability. More specifically, we build up a multi-variable convolutional neural network model to estimate the clean image, back…
Underwater visual perception is essentially important for underwater exploration, archeology, ecosystem and so on. The low illumination, light reflections, scattering, absorption and suspended particles inevitably lead to the critically degraded underwater image quality, which causes great challenges on recognizing the objects from the underwater images. The existing underwater enhancement methods that aim to promote the underwater visibility, heavily suffer from the poor image restoration performance and generalization ability. To reduce the difficulty of underwater image enhancement, we intr…
We present a wavelet-based dual-stream network that addresses color cast and blurry details in underwater images. We handle these artifacts separately by decomposing an input image into multiple frequency bands using discrete wavelet transform, which generates the downsampled structure image and detail images. These sub-band images are used as input to our dual-stream network that incorporates two sub-networks: the multi-color space fusion network and the detail enhancement network. The multi-color space fusion network takes the decomposed structure image as input and estimates the color corre…
Over the past few decades, underwater image enhancement has attracted increasing amount of research effort due to its significance in underwater robotics and ocean engineering. Research has evolved from implementing physics-based solutions to using very deep CNNs and GANs. However, these state-of-art algorithms are computationally expensive and memory intensive. This hinders their deployment on portable devices for underwater exploration tasks. These models are trained on either synthetic or limited real world datasets making them less practical in real-world scenarios. In this paper we propos…
Nanjing University of Information Science and Technology（南京信息工程大学）
Underwater image enhancement algorithms have attracted much attention in underwater vision task. However, these algorithms are mainly evaluated on different data sets and different metrics. In this paper, we set up an effective and pubic underwater test dataset named U45 including the color casts, low contrast and haze-like effects of underwater degradation and propose a fusion adversarial network for enhancing underwater images. Meanwhile, the well-designed the adversarial loss including Lgt loss and Lfe loss is presented to focus on image features of ground truth, and image features of the i…
Underwater image enhancement is such an important vision task due to its significance in marine engineering and aquatic robot. It is usually work as a pre-processing step to improve the performance of high level vision tasks such as underwater object detection. Even though many previous works show the underwater image enhancement algorithms can boost the detection accuracy of the detectors, no work specially focus on investigating the relationship between these two tasks. This is mainly because existing underwater datasets lack either bounding box annotations or high quality reference images, …
The goal of this work is to apply a denoising image transformer to remove the distortion from underwater images and compare it with other similar approaches. Automatic restoration of underwater images plays an important role since it allows to increase the quality of the images, without the need for more expensive equipment. This is a critical example of the important role of the machine learning algorithms to support marine exploration and monitoring, reducing the need for human intervention like the manual processing of the images, thus saving time, effort, and cost. This paper is the first …
Underwater Image Enhancement (UIE) technology aims to tackle the challenge of restoring the degraded underwater images due to light absorption and scattering. To address problems, a novel U-Net based Reinforced Swin-Convs Transformer for the Underwater Image Enhancement method (URSCT-UIE) is proposed. Specifically, with the deficiency of U-Net based on pure convolutions, we embedded the Swin Transformer into U-Net for improving the ability to capture the global dependency. Then, given the inadequacy of the Swin Transformer capturing the local attention, the reintroduction of convolutions may c…
This work proposes a method for underwater image enhancement using the principle of histogram equalization. Since underwater images have a global strong dominant colour, their colourfulness and contrast are often degraded. Before applying the histogram equalisation technique on the image, the image is converted from coloured image to a gray scale image for further operations. Histogram equalization is a technique for adjusting image intensities to enhance contrast. The colours of the image are retained using a convolutional neural network model which is trained by the datasets of underwater im…
PHISWID: Physics-Inspired Underwater Image Dataset Synthesized from RGB-D Images
Graduate School of Engineering, The University of Osaka（大阪大学工学研究科）；Graduate School of BASE, Tokyo University of Agriculture and Technology（东京大学农业技术大学院大学）
Ocean University of China（中国海洋大学）；Qingdao University（青岛大学）；China University of Geosciences（中国地质大学）
Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement net…
Underwater image enhancement has become an attractive topic as a significant technology in marine engineering and aquatic robotics. However, the limited number of datasets and imperfect hand-crafted ground truth weaken its robustness to unseen scenarios, and hamper the application to high-level vision tasks. To address the above limitations, we develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model, aiming to exploit its hierarchical feature representation as an auxiliary for the low-level underwater image enhancement. Specifical…
Images acquired during underwater activities suffer from environmental properties of the water, such as turbidity and light attenuation. These phenomena cause color distortion, blurring, and contrast reduction. In addition, irregular ambient light distribution causes color channel unbalance and regions with high-intensity pixels. Recent works related to underwater image enhancement, and based on deep learning approaches, tackle the lack of paired datasets generating synthetic ground-truth. In this paper, we present a self-supervised learning methodology for underwater image enhancement based o…
We present a novel underwater image enhancement method termed SCNet to improve the image quality meanwhile cope with the degradation diversity caused by the water. SCNet is based on normalization schemes across both spatial and channel dimensions with the key idea of learning water type desensitized features. Specifically, we apply whitening to de-correlate activations across spatial dimensions for each instance in a mini-batch. We also eliminate channel-wise correlation by standardizing and re-injecting the first two moments of the activations across channels. The normalization schemes of spa…
University of Leicester（莱斯特大学）；Ocean University of China（中国海洋大学）；Queen Mary University of London（伦敦玛丽女王大学）
Underwater image enhancement, as a pre-processing step to improve the accuracy of the following object detection task, has drawn considerable attention in the field of underwater navigation and ocean exploration. However, most of the existing underwater image enhancement strategies tend to consider enhancement and detection as two independent modules with no interaction, and the practice of separate optimization does not always help the underwater object detection task. In this paper, we propose two perceptual enhancement models, each of which uses a deep enhancement model with a detection per…
Data61, CSIRO（联邦科学与工业研究组织Data61）；Australian National University（澳大利亚国立大学）；City University of Hong Kong（香港城市大学）
The powerful representation capacity of deep learning has made it inevitable for the underwater image enhancement community to employ its potential. The exploration of deep underwater image enhancement networks is increasing over time, and hence; a comprehensive survey is the need of the hour. In this paper, our main aim is two-fold, 1): to provide a comprehensive and in-depth survey of the deep learning-based underwater image enhancement, which covers various perspectives ranging from algorithms to open issues, and 2): to conduct a qualitative and quantitative comparison of the deep algorithm…
Academy for Engineering & Technology, Fudan University（复旦大学工程与应用技术研究院）；College of Information Technology, Shanghai Ocean University（上海海洋大学信息学院）；University of Calabria（卡拉布里亚大学）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Napier Edinburgh University（爱丁堡龙比亚大学）
Underwater images play a key role in ocean exploration, but often suffer from severe quality degradation due to light absorption and scattering in water medium. Although major breakthroughs have been made recently in the general area of image enhancement and restoration, the applicability of new methods for improving the quality of underwater images has not specifically been captured. In this paper, we review the image enhancement and restoration methods that tackle typical underwater image impairments, including some extreme degradations and distortions. Firstly, we introduce the key causes o…
City University of Hong Kong（香港城市大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Tianjin University（天津大学）；Beijing Jiaotong University（北京交通大学）
Underwater image enhancement has been attracting much attention due to its significance in marine engineering and aquatic robotics. Numerous underwater image enhancement algorithms have been proposed in the last few years. However, these algorithms are mainly evaluated using either synthetic datasets or few selected real-world images. It is thus unclear how these algorithms would perform on images acquired in the wild and how we could gauge the progress in the field. To bridge this gap, we present the first comprehensive perceptual study and analysis of underwater image enhancement using large…
DACA-Net: A Degradation-Aware Conditional Diffusion Network for Underwater Image Enhancement
The Hong Kong University of Science and Technology (Guangzhou), Southern Marine Science and Engineering Guangdong Laboratory (Guangzhou)（香港科学与技术大学（广州）、广东海洋工程科学实验室（广州））；The Hong Kong University of Science（香港科学与技术大学）；Engineering Guangdong Laboratory (Guangzhou)（广东工程实验室（广州））
The experimental results of improved underwater image enhancement algorithms based on partial differential equations (PDEs) are presented in this report. This second work extends the study of previous work and incorporating several improvements into the revised algorithm. Experiments show the evidence of the improvements when compared to previously proposed approaches and other conventional algorithms found in the literature.
Due to the absorption and scattering effects of the water, underwater images tend to suffer from many severe problems, such as low contrast, grayed out colors and blurring content. To improve the visual quality of underwater images, we proposed a novel enhancement model, which is a trainable end-to-end neural model. Two parts constitute the overall model. The first one is a non-parameter layer for the preliminary color correction, then the second part is consisted of parametric layers for a self-adaptive refinement, namely the channel-wise linear shift. For better details, contrast and colorfu…
IDA-UIE: An Iterative Framework for Deep Network-based Degradation Aware Underwater Image Enhancement
Centre for Intelligent Cyber Physical Systems, Indian Institute of Technology Guwahati（印度古瓦哈蒂理工学院智能信息物理系统中心）；Dept. of Electronics and Electrical Engg., Indian Institute of Technology Guwahati（印度古瓦哈蒂理工学院电子与电气工程系）
Australian National University（澳大利亚国立大学）；Civil Aviation University of China（中国民航大学）
In an underwater scene, wavelength-dependent light absorption and scattering degrade the visibility of images, causing low contrast and distorted color casts. To address this problem, we propose a convolutional neural network based image enhancement model, i.e., UWCNN, which is trained efficiently using a synthetic underwater image database. Unlike the existing works that require the parameters of underwater imaging model estimation or impose inflexible frameworks applicable only for specific scenes, our model directly reconstructs the clear latent underwater image by leveraging on an automati…
Jimei University（集美大学）；National University of Singapore（新加坡国立大学）；Southwest University（西南大学）；Fujian Provincial Key Laboratory of Oceanic Information Perception and Intelligent Processing（福建省海洋信息感知与智能处理重点实验室）
Underwater images typically experience mixed degradations of brightness and structure caused by the absorption and scattering of light by suspended particles. To address this issue, we propose a Real-time Spatial and Frequency Domains Modulation Network (RSFDM-Net) for the efficient enhancement of colors and details in underwater images. Specifically, our proposed conditional network is designed with Adaptive Fourier Gating Mechanism (AFGM) and Multiscale Convolutional Attention Module (MCAM) to generate vectors carrying low-frequency background information and high-frequency detail features, …
The degradation in the underwater images is due to wavelength-dependent light attenuation, scattering, and to the diversity of the water types in which they are captured. Deep neural networks take a step in this field, providing autonomous models able to achieve the enhancement of underwater images. We introduce Underwater Capsules Vectors GAN UWCVGAN based on the discrete features quantization paradigm from VQGAN for this task. The proposed UWCVGAN combines an encoding network, which compresses the image into its latent representation, with a decoding network, able to reconstruct the enhancem…
This report presents the results of a multi-scale wavelet based scheme for single image de-hazing and underwater image enhancement. The scheme is fast and highly localized in addition to global enhancement of hazy images. A PDE-based formulation enables additional versatility as the iterative nature allows more flexibility for various types of images. Visual and objective results from experiments indicate that the proposed approach competes favourably or surpasses most of the state-of-the-art approaches.
UDBE: Unsupervised Diffusion-based Brightness Enhancement in Underwater Images
Centro de Ciências Computacionais（计算机科学中心）；Universidade Federal do Rio Grande（南里奥格兰德联邦大学）；Instituto de Ciências Exatas e Tecnologia（精确科学与技术研究所）；Universidade Federal do Amazonas（亚马逊联邦大学）
ASUS Intelligent Cloud Services, Asustek Computer Inc.（华硕电脑股份有限公司智能云服务）；National Taiwan University（国立台湾大学）
Underwater image enhancement is an important low-level computer vision task for autonomous underwater vehicles and remotely operated vehicles to explore and understand the underwater environments. Recently, deep convolutional neural networks (CNNs) have been successfully used in many computer vision problems, and so does underwater image enhancement. There are many deep-learning-based methods with impressive performance for underwater image enhancement, but their memory and model parameter costs are hindrances in practical application. To address this issue, we propose a lightweight adaptive f…
Underwater image enhancement is such an important low-level vision task with many applications that numerous algorithms have been proposed in recent years. These algorithms developed upon various assumptions demonstrate successes from various aspects using different data sets and different metrics. In this work, we setup an undersea image capturing system, and construct a large-scale Real-world Underwater Image Enhancement (RUIE) data set divided into three subsets. The three subsets target at three challenging aspects for enhancement, i.e., image visibility quality, color casts, and higher-le…
Owing to refraction, absorption, and scattering of light by suspended particles in water, raw underwater images suffer from low contrast, blurred details, and color distortion. These characteristics can significantly interfere with the visibility of underwater images and the result of visual tasks, such as segmentation and tracking. To address this problem, we propose a new robust adversarial learning framework via physics model based feedback control and domain adaptation mechanism for enhancing underwater images to get realistic results. A new method for simulating underwater-like training d…
A main challenge faced in the deep learning-based Underwater Image Enhancement (UIE) is that the ground truth high-quality image is unavailable. Most of the existing methods first generate approximate reference maps and then train an enhancement network with certainty. This kind of method fails to handle the ambiguity of the reference map. In this paper, we resolve UIE into distribution estimation and consensus process. We present a novel probabilistic network to learn the enhancement distribution of degraded underwater images. Specifically, we combine conditional variational autoencoder with …
End-To-End Underwater Video Enhancement: Dataset and Model
Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；Institute of Software, Chinese Academy of Science（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Underwater image enhancement has attracted much attention due to the rise of marine resource development in recent years. Benefit from the powerful representation capabilities of Convolution Neural Networks(CNNs), multiple underwater image enhancement algorithms based on CNNs have been proposed in the last few years. However, almost all of these algorithms employ RGB color space setting, which is insensitive to image properties such as luminance and saturation. To address this problem, we proposed Underwater Image Enhancement Convolution Neural Network using 2 Color Space (UICE^2-Net) that eff…
In this paper, we introduce and tackle the simultaneous enhancement and super-resolution (SESR) problem for underwater robot vision and provide an efficient solution for near real-time applications. We present Deep SESR, a residual-in-residual network-based generative model that can learn to restore perceptual image qualities at 2x, 3x, or 4x higher spatial resolution. We supervise its training by formulating a multi-modal objective function that addresses the chrominance-specific underwater color degradation, lack of image sharpness, and loss in high-level feature representation. It is also s…
In this paper, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP, a large-scale dataset of a paired and unpaired collection of underwater images (of `poor' and `good' quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experime…
Beneath the Surface: The Role of Underwater Image Enhancement in Object Detection
College of Computing, Michigan Technological University（密歇根理工大学计算学院）；Fairfield University（费尔菲尔德大学）；Institute of Computing and Cybersystems, Michigan Technological University（密歇根理工大学计算与网络系统研究所）；Department of Computer Science, Michigan Technological University（密歇根理工大学计算机科学系）
PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）
