Deep learning-based underwater object detection (UOD) remains a major challenge due to the degraded visibility and difficulty to obtain sufficient underwater object images captured from various perspectives for training. To address these issues, this paper presents a high-precision UOD based on joint self-supervised deblurring and improved spatial transformer network. A self-supervised deblurring subnetwork is introduced into the designed multi-task learning aided object detection architecture to force the shared feature extraction module to output clean features for detection subnetwork. Aiming at alleviating the limitation of insufficient photos from different perspectives, an improved spatial transformer network is designed based on perspective transformation, adaptively enriching image features within the network. The experimental results show that the proposed UOD approach achieved 47.9 mAP in URPC2017 and 70.3 mAP in URPC2018, outperforming many state-of-the-art UOD methods and indicating the designed method is more suitable for UOD.
UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, The University of Sydney（悉尼大学计算机学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）
Shanghai Advanced Research Institute, Chinese Academy of Sciences（中国科学院上海高等研究院）
Currently, many blind deblurring methods assume blurred images are noise-free and perform unsatisfactorily on the blurry images with noise. Unfortunately, noise is quite common in real scenes. A straightforward solution is to denoise images before deblurring them. However, even state-of-the-art denoisers cannot guarantee to remove noise entirely. Slight residual noise in the denoised images could cause significant artifacts in the deblurring stage. To tackle this problem, we propose a cascaded framework consisting of a denoiser subnetwork and a deblurring subnetwork. In contrast to previous me…
Adaptive Enhancement and Dual-Pooling Sequential Attention for Lightweight Underwater Object Detection with YOLOv10
1,2Department of Computer Science；Engineering, Dhaka University of Engineering \& Technology Gazipur, 1707, Bangladesh 3Department of Computer Science；Engineering, BRAC University Dhaka, Bangladesh
Seoul National University（首尔大学）；Inha University（仁荷大学）
Removing camera motion blur from a single light field is a challenging task since it is highly ill-posed inverse problem. The problem becomes even worse when blur kernel varies spatially due to scene depth variation and high-order camera motion. In this paper, we propose a novel algorithm to estimate all blur model variables jointly, including latent sub-aperture image, camera motion, and scene depth from the blurred 4D light field. Exploiting multi-view nature of a light field relieves the inverse property of the optimization by utilizing strong depth cues and multi-view blur observation. The…
University of Oulu（奥卢大学）；Aalto University（阿尔托大学）；Czech Technical University in Prague（捷克技术大学布拉格分校）；Department of Computer Science（计算机科学系）；Center for Machine Perception（机器感知中心）；Faculty of Electrical Engineering（电气工程学院）；Department of Electrical Engineering and Automation（电气工程与自动化系）
The paper addresses the problem of acquiring high-quality photographs with handheld smartphone cameras in low-light imaging conditions. We propose an approach based on capturing pairs of short and long exposure images in rapid succession and fusing them into a single high-quality photograph. Unlike existing methods, we take advantage of both images simultaneously and perform a joint denoising and deblurring using a convolutional neural network. A novel approach is introduced to generate realistic short-long exposure image pairs. The method produces good images in extremely challenging conditio…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
In this paper, we propose a novel joint deblurring and multi-frame interpolation (DeMFI) framework, called DeMFI-Net, which accurately converts blurry videos of lower-frame-rate to sharp videos at higher-frame-rate based on flow-guided attentive-correlation-based feature bolstering (FAC-FB) module and recursive boosting (RB), in terms of multi-frame interpolation (MFI). The DeMFI-Net jointly performs deblurring and MFI where its baseline version performs feature-flow-based warping with FAC-FB module to obtain a sharp-interpolated frame as well to deblur two center-input frames. Moreover, its e…
The conventional methods for estimating camera poses and scene structures from severely blurry or low resolution images often result in failure. The off-the-shelf deblurring or super-resolution methods may show visually pleasing results. However, applying each technique independently before matching is generally unprofitable because this naive series of procedures ignores the consistency between images. In this paper, we propose a pioneering unified framework that solves four problems simultaneously, namely, dense depth reconstruction, camera pose estimation, super-resolution, and deblurring. …
University of Florida（佛罗里达大学）；University of Minnesota, Twin Cities（明尼苏达大学双城分校）
This paper presents a holistic approach to saliency-guided visual attention modeling (SVAM) for use by autonomous underwater robots. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective salient object detection (SOD) in natural underwater images. The SVAM-Net architecture is configured in a unique way to jointly accommodate bottom-up and top-down learning within two separate branches of the network while sharing the same encoding layers. We design dedicated spatial attention modules (SAMs) along these learning pathways to exploit the…
Jointly RS Image Deblurring and Super-Resolution with Adjustable-Kernel and Multi-Domain Attention
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences（中国科学院重庆绿色智能技术研究院）
Sensetime Research（商汤科技研究院）；Tencent（腾讯）；South China University of Technology（华南理工大学）；University of Science and Technology of China（中国科学技术大学）；University of California at Merced（加州大学默塞德分校）；Tencent AI Lab（腾讯人工智能实验室）；Nanjing University of Science and Technology（南京理工大学）
We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse f…
Australian National University（澳大利亚国立大学）；Northwestern Polytechnical University（西北工业大学）
Stereo videos for the dynamic scenes often show unpleasant blurred effects due to the camera motion and the multiple moving objects with large depth variations. Given consecutive blurred stereo video frames, we aim to recover the latent clean images, estimate the 3D scene flow and segment the multiple moving objects. These three tasks have been previously addressed separately, which fail to exploit the internal connections among these tasks and cannot achieve optimality. In this paper, we propose to jointly solve these three tasks in a unified framework by exploiting their intrinsic connection…
Ambient pressure compensation and robust position control of oil-filled electric joint systems for underwater manipulators
J.K. Institute of Science（J.K.科学学院）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院）
Night photography typically suffers from both low light and blurring issues due to the dim environment and the common use of long exposure. While existing light enhancement and deblurring methods could deal with each problem individually, a cascade of such methods cannot work harmoniously to cope well with joint degradation of visibility and textures. Training an end-to-end network is also infeasible as no paired data is available to characterize the coexistence of low light and blurs. We address the problem by introducing a novel data synthesis pipeline that models realistic low-light blurrin…
VJT: A Video Transformer on Joint Tasks of Deblurring, Low-light Enhancement and Denoising
The Chinese University of Hong Kong（香港中文大学）；National University of Defense Technology（国防科技大学）；City University of Hong Kong（香港城市大学）；Nanjing University of Science and Technology（南京理工大学）
Harbin Institute of Technology（哈尔滨工业大学）；City University of Hong Kong（香港城市大学）；Peng Cheng Laboratory（鹏城实验室）
Natural videos captured by consumer cameras often suffer from low framerate and motion blur due to the combination of dynamic scene complexity, lens and sensor imperfection, and less than ideal exposure setting. As a result, computational methods that jointly perform video frame interpolation and deblurring begin to emerge with the unrealistic assumption that the exposure time is known and fixed. In this work, we aim ambitiously for a more realistic and challenging task - joint video multi-frame interpolation and deblurring under unknown exposure time. Toward this goal, we first adopt a varian…
Facial image super-resolution (SR) is an important preprocessing for facial image analysis, face recognition, and image-based 3D face reconstruction. Recent convolutional neural network (CNN) based method has shown excellent performance by learning mapping relation using pairs of low-resolution (LR) and high-resolution (HR) facial images. However, since the HR facial image reconstruction using CNN is conventionally aimed to increase the PSNR and SSIM metrics, the reconstructed HR image might not be realistic even with high scores. An adversarial framework is proposed in this study to reconstru…
Xi’an Jiaotong University（西安交通大学）；Hikvision Research（海康威视研究院）；University of California, Merced（加州大学默塞德分校）；Google Cloud（谷歌云）
Single-image super-resolution is a fundamental task for vision applications to enhance the image quality with respect to spatial resolution. If the input image contains degraded pixels, the artifacts caused by the degradation could be amplified by super-resolution methods. Image blur is a common degradation source. Images captured by moving or still cameras are inevitably affected by motion blur due to relative movements between sensors and objects. In this work, we focus on the super-resolution task with the presence of motion blur. We propose a deep gated fusion convolution neural network to…
JDPNet: A Network Based on Joint Degradation Processing for Underwater Image Enhancement
School of Mechanical and Electrical Engineering, China University of Mining and Technology（中国矿业大学（北京）机械与电子工程学院）；China Shipbuilding Science Research Center（中国船舶科学研究院）；School of Physics and Optoelectronic Engineering, Foshan University（佛山大学物理与电子工程学院）
Joint Transmission and Deblurring: A Semantic Communication Approach Using Events
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；School of Electronic Information, Wuhan University（武汉大学电子信息学院）
Severe color casts, low contrast and blurriness of underwater images caused by light absorption and scattering result in a difficult task for exploring underwater environments. Different from most of previous underwater image enhancement methods that compute light attenuation along object-camera path through hazy image formation model, we propose a novel jointly wavelength compensation and dehazing network (JWCDN) that takes into account the wavelength attenuation along surface-object path and the scattering along object-camera path simultaneously. By embedding a simplified underwater formatio…
EndoCaver: Handling Fog, Blur and Glare in Endoscopic Images via Joint Deblurring-Segmentation
1 CyPhi ( ) AI Research Lab, School of IT, Monash University, Malaysia Campus；2 Department of Electronic \& Computer Engineering, The Hong Kong University of Science；3 Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
JADE-GS: Joint Allocation of Deblurring Evidence for Event-Assisted 3D Gaussian Splatting
School of Mechatronic Engineering and Automation, Shanghai University（上海大学机电工程与自动化学院）；Shanghai Baoshan Shangda General Intelligent Robotics Research Institute（上海宝山尚大通用智能机器人研究院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）
Australian National University（澳大利亚国立大学）；Tencent AI Laboratory（腾讯人工智能实验室）
Camera shake or target movement often leads to undesired blur effects in videos captured by a hand-held camera. Despite significant efforts having been devoted to video-deblur research, two major challenges remain: 1) how to model the spatio-temporal characteristics across both the spatial domain (i.e., image plane) and temporal domain (i.e., neighboring frames), and 2) how to restore sharp image details w.r.t. the conventionally adopted metric of pixel-wise errors. In this paper, to address the first challenge, we propose a DeBLuRring Network (DBLRNet) for spatial-temporal learning by applyin…
Non-uniform image deblurring is a challenging task due to the lack of temporal and textural information in the blurry image itself. Complementary information from auxiliary sensors such event sensors are being explored to address these limitations. The latter can record changes in a logarithmic intensity asynchronously, called events, with high temporal resolution and high dynamic range. Current event-based deblurring methods combine the blurry image with events to jointly estimate per-pixel motion and the deblur operator. In this paper, we argue that a divide-and-conquer approach is more suit…
Real-world dynamic scene deblurring has long been a challenging task since paired blurry-sharp training data is unavailable. Conventional Maximum A Posteriori estimation and deep learning-based deblurring methods are restricted by handcrafted priors and synthetic blurry-sharp training pairs respectively, thereby failing to generalize to real dynamic blurriness. To this end, we propose a Neural Maximum A Posteriori (NeurMAP) estimation framework for training neural networks to recover blind motion information and sharp content from unpaired data. The proposed NeruMAP consists of a motion estima…
City University of Hong Kong（香港城市大学）；SenseTime Research（商汤科技研究院）；Shanghai Jiao Tong University（上海交通大学）
Night imaging with modern smartphone cameras is troublesome due to low photon count and unavoidable noise in the imaging system. Directly adjusting exposure time and ISO ratings cannot obtain sharp and noise-free images at the same time in low-light conditions. Though many methods have been proposed to enhance noisy or blurry night images, their performances on real-world night photos are still unsatisfactory due to two main reasons: 1) Limited information in a single image and 2) Domain gap between synthetic training images and real-world photos (e.g., differences in blur area and resolution)…
Nanjing University of Science and Technology（南京理工大学）；CETC（中国电子科技集团公司）
We present an effective and efficient method that explores the properties of Transformers in the frequency domain for high-quality image deblurring. Our method is motivated by the convolution theorem that the correlation or convolution of two signals in the spatial domain is equivalent to an element-wise product of them in the frequency domain. This inspires us to develop an efficient frequency domain-based self-attention solver (FSAS) to estimate the scaled dot-product attention by an element-wise product operation instead of the matrix multiplication in the spatial domain. In addition, we no…
A Physical Model-Guided Framework for Underwater Image Enhancement and Depth Estimation
Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
ETH Zurich（苏黎世联邦理工学院）；University of Amsterdam（阿姆斯特丹大学）；Google Research（谷歌研究院）
We address the novel task of jointly reconstructing the 3D shape, texture, and motion of an object from a single motion-blurred image. While previous approaches address the deblurring problem only in the 2D image domain, our proposed rigorous modeling of all object properties in the 3D domain enables the correct description of arbitrary object motion. This leads to significantly better image decomposition and sharper deblurring results. We model the observed appearance of a motion-blurred object as a combination of the background and a 3D object with constant translation and rotation. Our meth…
UD-SfPNet: An Underwater Descattering Shape-from-Polarization Network for 3D Normal Reconstruction
School of Mechanical Engineering and Automation, Fuzhou University（福州大学机械与自动化学院）；The Research Institute of Highway, Ministry of Transport（交通部高速公路研究 institute）；The State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instruments, Tsinghua University（清华大学精密测量技术与仪器国家重点实验室）
SDT-DCSCN for Simultaneous Super-Resolution and Deblurring of Text Images
University of Gabes（加贝斯大学）；University of Sfax（斯法克斯大学）；University of Zaragoza（萨拉戈萨大学）；University of Johannesburg（约翰内斯堡大学）；Taif University（塔伊夫大学）；King Saud University（沙特国王大学）；University of Monastir（莫纳斯提尔大学）
DeepRFTv2: Kernel-level Learning for Image Deblurring
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University（上海多维信息处理重点实验室，华东师范大学）；State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Sciences（红外物理国家重点实验室，上海技术物理研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
Physen-Noise2Noise: Physics-Guided Self-Supervised Defocus Deblurring with Bias Correction under Low-Light Conditions
School of Mathematics and Statistics, Central South University（数学与统计学学院，中南大学）；Key Laboratory for Micro/Nano Optoelectronic Devices of Ministry of Education, Hunan Provincial Key Laboratory of Low-Dimensional Structural Physics and Devices, School of Physics and Electronics, Hunan University（教育部微/纳米光电器件重点实验室，湖南省低维结构物理与器件重点实验室，物理与电子学院，湖南大学）
Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）
When taking photos under an environment with insufficient light, the exposure time and the sensor gain usually require to be carefully chosen to obtain images with satisfying visual quality. For example, the images with high ISO usually have inescapable noise, while the long-exposure ones may be blurry due to camera shake or object motion. Existing solutions generally suggest to seek a balance between noise and blur, and learn denoising or deblurring models under either full- or self-supervision. However, the real-world training pairs are difficult to collect, and the self-supervised methods m…
In this paper, we introduce and tackle the simultaneous enhancement and super-resolution (SESR) problem for underwater robot vision and provide an efficient solution for near real-time applications. We present Deep SESR, a residual-in-residual network-based generative model that can learn to restore perceptual image qualities at 2x, 3x, or 4x higher spatial resolution. We supervise its training by formulating a multi-modal objective function that addresses the chrominance-specific underwater color degradation, lack of image sharpness, and loss in high-level feature representation. It is also s…
