Low-light image enhancement is a promising solution to tackle the problem of insufficient sensitivity of human vision system (HVS) to perceive information in low light environments. Previous Retinex-based works always accomplish enhancement task by estimating light intensity. Unfortunately, single light intensity modelling is hard to accurately simulate visual perception information, leading to the problems of imbalanced visual photosensitivity and weak adaptivity. To solve these problems, we explore the precise relationship between light source and visual perception and then propose the visual perception (VP) model to acquire a precise mathematical description of visual perception. The core of VP model is to decompose the light source into light intensity and light spatial distribution to describe the perception process of HVS, offering refinement estimation of illumination and reflectance. To reduce complexity of the estimation process, we introduce the rapid and adaptive $\mathbfβ$ and $\mathbfγ$ functions to build an illumination and reflectance estimation scheme. Finally, we present a optimal determination strategy, consisting of a \emph{cycle operation} and a \emph{comparator}. Specifically, the \emph{comparator} is responsible for determining the optimal enhancement results from multiple enhanced results through implementing the \emph{cycle operation}. By coordinating the proposed VP model, illumination and reflectance estimation scheme, and the optimal determination strategy, we propose a rapid and adaptive framework for low-light image enhancement. Extensive experiment results demenstrate that the proposed method achieves better performance in terms of visual comparison, quantitative assessment, and computational efficiency, compared with the currently state-of-the-arts.
University of Science and Technology of China（中国科学技术大学）；Shandong University（山东大学）；The University of Sydney（悉尼大学）
Contrast enhancement and noise removal are coupled problems for low-light image enhancement. The existing Retinex based methods do not take the coupling relation into consideration, resulting in under or over-smoothing of the enhanced images. To address this issue, this paper presents a novel progressive Retinex framework, in which illumination and noise of low-light image are perceived in a mutually reinforced manner, leading to noise reduction low-light enhancement results. Specifically, two fully pointwise convolutional neural networks are devised to model the statistical regularities of am…
University of Florida（佛罗里达大学）；SenseBrain Technology（感脑科技）
While humans can effortlessly transform complex visual scenes into simple words and the other way around by leveraging their high-level understanding of the content, conventional or the more recent learned image compression codecs do not seem to utilize the semantic meanings of visual content to their full potential. Moreover, they focus mostly on rate-distortion and tend to underperform in perception quality especially in low bitrate regime, and often disregard the performance of downstream computer vision algorithms, which is a fast-growing consumer group of compressed images in addition to …
Ocean University of China（中国海洋大学）；Qingdao University（青岛大学）；China University of Geosciences（中国地质大学）
Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement net…
CARNet: Collaborative Adversarial Resilience for Robust Underwater Image Enhancement and Perception
School of Software Technology, Dalian University of Technology（大连理工大学软件学院）；College of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）；School of Mechanical Engineering, Dalian University of Technology（大连理工大学机械工程学院）
In this paper, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP, a large-scale dataset of a paired and unpaired collection of underwater images (of `poor' and `good' quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experime…
In this paper, we introduce and tackle the simultaneous enhancement and super-resolution (SESR) problem for underwater robot vision and provide an efficient solution for near real-time applications. We present Deep SESR, a residual-in-residual network-based generative model that can learn to restore perceptual image qualities at 2x, 3x, or 4x higher spatial resolution. We supervise its training by formulating a multi-modal objective function that addresses the chrominance-specific underwater color degradation, lack of image sharpness, and loss in high-level feature representation. It is also s…
Underwater Variable Zoom: Depth-Guided Perception Network for Underwater Image Enhancement
School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；University of New South Wales（新南威尔士大学）；School of Computer Science and Technology, Shandong Technology and Business University（山东工商学院计算机科学与技术学院）；School of Information and Communication Engineering, Dalian Minzu University（大连民族大学信息与通信工程学院）；School of Innovation Experiment, Dalian University of Technology（大连理工大学创新实验学院）
Underwater visual perception is essentially important for underwater exploration, archeology, ecosystem and so on. The low illumination, light reflections, scattering, absorption and suspended particles inevitably lead to the critically degraded underwater image quality, which causes great challenges on recognizing the objects from the underwater images. The existing underwater enhancement methods that aim to promote the underwater visibility, heavily suffer from the poor image restoration performance and generalization ability. To reduce the difficulty of underwater image enhancement, we intr…
University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；City University of New York / CSI（纽约市立大学/斯塔滕岛学院）
In this paper a quaternion approach of enhancement method is proposed in which color in the image is considered as a single entity. This new method is referred as the alpha-rooting method of color image enhancement by the two-dimensional quaternion discrete Fourier transform (2-D QDFT) followed by a spatial transformation. The results of the proposed color image enhancement method are compared with its counterpart channel-by-channel enhancement algorithm by the 2-D DFT. The image enhancements are quantified to the enhancement measure that is based on visual perception referred as the color enh…
Infrared and Visible Image Fusion with Hierarchical Human Perception
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications（重庆邮电大学图像认知重点实验室）
Unified Restoration-Perception Learning: Maritime Infrared-Visible Image Fusion and Segmentation
Laboratory of Underwater Acoustic Communication and Marine Information Technology, Ministry of Education, Xiamen University（厦门大学水声通信与海洋信息技术教育部重点实验室）；PAMI Research Group, Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系PAMI研究组）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）
Image enhancement using the visible (V) and near-infrared (NIR) usually enhances useful image details. The enhanced images are evaluated by observers perception, instead of quantitative feature evaluation. Thus, can we say that these enhanced images using NIR information has better features in comparison to the computed features in the Red, Green, and Blue color channels directly? In this work, we present a new method to enhance the visible images using NIR information via edge-preserving filters, and also investigate which method performs best from a image features standpoint. We then show th…
Low-light images challenge both human perceptions and computer vision algorithms. It is crucial to make algorithms robust to enlighten low-light images for computational photography and computer vision applications such as real-time detection and segmentation. This paper proposes a semantic-guided zero-shot low-light enhancement network (SGZ) which is trained in the absence of paired images, unpaired datasets, and segmentation annotation. Firstly, we design an enhancement factor extraction network using depthwise separable convolution for an efficient estimate of the pixel-wise light deficienc…
A Non-Uniform Low-Light Image Enhancement Method with Multi-Scale Attention Transformer and Luminance Consistency Loss
Beijing University of Posts and Telecommunications（北京邮电大学）；China Electric Power Research Institute Company Limited（中国电力科学研究院有限公司）；Tianjin University（天津大学）；State Grid Shanxi Marketing Service Center（国网山西省电力公司营销服务中心）
ALEN: A Dual-Approach for Uniform and Non-Uniform Low-Light Image Enhancement
School of Computer Science and Technology, Zhejiang Gongshang University（计算机科学与技术学院，浙江工商大学）；Depto. de Ingeniería Electro-fotónica, Universidad de Guadalajara（电子光子工程系，瓜达拉哈拉大学）
Convolutional neural networks rely on image texture and structure to serve as discriminative features to classify the image content. Image enhancement techniques can be used as preprocessing steps to help improve the overall image quality and in turn improve the overall effectiveness of a CNN. Existing image enhancement methods, however, are designed to improve the perceptual quality of an image for a human observer. In this paper, we are interested in learning CNNs that can emulate image enhancement and restoration, but with the overall goal to improve image classification and not necessarily…
Downstream Task Inspired Underwater Image Enhancement: A Perception-Aware Study from Dataset Construction to Network Design
State Key Laboratory of Physical Oceanography, Ocean University of China（中国海洋大学物理海洋国家重点实验室）；Department of Electrical and Computer Engineering, Mississippi State University（密苏里州立大学电子与计算机工程系）
The limited dynamic range of commercial compact camera sensors results in an inaccurate representation of scenes with varying illumination conditions, adversely affecting image quality and subsequently limiting the performance of underlying image processing algorithms. Current state-of-the-art (SoTA) convolutional neural networks (CNN) are developed as post-processing techniques to independently recover under-/over-exposed images. However, when applied to images containing real-world degradations such as glare, high-beam, color bleeding with varying noise intensity, these algorithms amplify th…
Rapid Bone Scintigraphy Enhancement via Semantic Prior Distillation from Segment Anything Model
Department of Nuclear Medicine, Shanghai Children’s Hospital, School of Medicine, Shanghai Jiao Tong University（核医学科，上海儿童医学中心，上海交通大学医学院）；School of Microelectronics, Shanghai University（微电子学院，上海大学）；Department of Ophthalmology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（眼科，瑞金医院，上海交通大学医学院）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；School of Software Engineering, Sun Yat-sen University（软件工程学院，中山大学）；College of Computer Science and Electronic Engineering, Hunan University（计算机科学与电子工程学院，湖南大学）；School of Computer Science and Technology, Henan Polytechnic University（计算机科学与技术学院，河南理工大学）
本文首次将 SAM 语义先验引入医学图像恢复，通过级联网络和三个语义模块增强儿科快速骨闪烁显像，并构建首个专用数据集 RBS，在多项指标上优于现有方法。
Columbia University（哥伦比亚大学）；Columbia University Medical Center（哥伦比亚大学医学中心）
Recent advances in the interdisciplinary scientific field of machine perception, computer vision, and biomedical engineering underpin a collection of machine learning algorithms with a remarkable ability to decipher the contents of microscope and nanoscope images. Machine learning algorithms are transforming the interpretation and analysis of microscope and nanoscope imaging data through use in conjunction with biological imaging modalities. These advances are enabling researchers to carry out real-time experiments that were previously thought to be computationally impossible. Here we adapt th…
This paper introduces a divide-and-conquer inspired adversarial learning (DACAL) approach for photo enhancement. The key idea is to decompose the photo enhancement process into hierarchically multiple sub-problems, which can be better conquered from bottom to up. On the top level, we propose a perception-based division to learn additive and multiplicative components, required to translate a low-quality image or video into its high-quality counterpart. On the intermediate level, we use a frequency-based division with generative adversarial network (GAN) to weakly supervise the photo enhancement…
Underwater robots play an important role in oceanic geological exploration, resource exploitation, ecological research, and other fields. However, the visual perception of underwater robots is affected by various environmental factors. The main challenge now is that images captured by underwater robots are color-distorted. The hue of underwater images tends to be close to green and blue. In addition, the contrast is low and the details are fuzzy. In this paper, a new underwater image enhancement algorithm based on deep learning and image formation model is proposed. Experimental results show t…
Speech enhancement can potentially benefit from the visual information from the target speaker, such as lip movement and facial expressions, because the visual aspect of speech is essentially unaffected by acoustic environment. In this paper, we address the problem of enhancing corrupted speech signal from videos by using audio-visual (AV) neural processing. Most of recent AV speech enhancement approaches separately process the acoustic and visual features and fuse them via a simple concatenation operation. Although this strategy is convenient and easy to implement, it comes with two major dra…
Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization im…
