Neural architecture search (NAS) has recently reshaped our understanding on various vision tasks. Similar to the success of NAS in high-level vision tasks, it is possible to find a memory and computationally efficient solution via NAS with highly competent denoising performance. However, the optimization gap between the super-network and the sub-architectures has remained an open issue in both low-level and high-level vision. In this paper, we present a novel approach to filling in this gap by connecting model-guided design with NAS (MoD-NAS) and demonstrate its application into image denoising. Specifically, we propose to construct a new search space under model-guided framework and develop more stable and efficient differential search strategies. MoD-NAS employs a highly reusable width search strategy and a densely connected search block to automatically select the operations of each layer as well as network width and depth via gradient descent. During the search process, the proposed MoG-NAS is capable of avoiding mode collapse due to the smoother search space designed under the model-guided framework. Experimental results on several popular datasets show that our MoD-NAS has achieved even better PSNR performance than current state-of-the-art methods with fewer parameters, lower number of flops, and less amount of testing time.
Nanjing University of Science and Technology（南京理工大学）；University of Macau（澳门大学）；Griffith University（格里菲斯大学）；Zhejiang University（浙江大学）
Deep learning (DL) based hyperspectral images (HSIs) denoising approaches directly learn the nonlinear mapping between observed noisy images and underlying clean images. They normally do not consider the physical characteristics of HSIs, therefore making them lack of interpretability that is key to understand their denoising mechanism.. In order to tackle this problem, we introduce a novel model guided interpretable network for HSI denoising. Specifically, fully considering the spatial redundancy, spectral low-rankness and spectral-spatial properties of HSIs, we first establish a subspace base…
Fully supervised deep-learning based denoisers are currently the most performing image denoising solutions. However, they require clean reference images. When the target noise is complex, e.g. composed of an unknown mixture of primary noises with unknown intensity, fully supervised solutions are limited by the difficulty to build a suited training set for the problem. This paper proposes a gradual denoising strategy that iteratively detects the dominating noise in an image, and removes it using a tailored denoiser. The method is shown to keep up with state of the art blind denoisers on mixture…
Patch-based denoising algorithms like BM3D have achieved outstanding performance. An important idea for the success of these methods is to exploit the recurrence of similar patches in an input image to estimate the underlying image structures. However, in these algorithms, the similar patches used for denoising are obtained via Nearest Neighbour Search (NNS) and are sometimes not optimal. First, due to the existence of noise, NNS can select similar patches with similar noise patterns to the reference patch. Second, the unreliable noisy pixels in digital images can bring a bias to the patch sea…
Pan-denoising: Guided Hyperspectral Image Denoising via Weighted Represent Coefficient Total Variation
School of Mathematics and Statistics, Northwestern Polytechnical University（西北工业大学数学与统计学院）；Research and Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）
School of Statistics and Data Science, Nankai University（南开大学统计与数据科学学院）；The First Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第一附属医院）；Shanxi Engineering Research Center of Computational Imaging and Medical Intelligence（山西省计算成像与医学智能工程研究中心）
The Hong Kong Polytechnic University（香港理工大学）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
Most of existing image denoising methods learn image priors from either external data or the noisy image itself to remove noise. However, priors learned from external data may not be adaptive to the image to be denoised, while priors learned from the given noisy image may not be accurate due to the interference of corrupted noise. Meanwhile, the noise in real-world noisy images is very complex, which is hard to be described by simple distributions such as Gaussian distribution, making real-world noisy image denoising a very challenging problem. We propose to exploit the information in both ext…
SGDFormer: One-stage Transformer-based Architecture for Cross-Spectral Stereo Image Guided Denoising
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与工程学院）；Ningbo Research Institute, Zhejiang University（浙江大学宁波研究院）；Zhejiang GongShang University（浙江工商大学）
Hamamatsu Photonics K. K.（滨松光子学株式会社）；Hamamatsu University School of Medicine（滨松医科大学）
Although supervised convolutional neural networks (CNNs) often outperform conventional alternatives for denoising positron emission tomography (PET) images, they require many low- and high-quality reference PET image pairs. Herein, we propose an unsupervised 3D PET image denoising method based on an anatomical information-guided attention mechanism. The proposed magnetic resonance-guided deep decoder (MR-GDD) utilizes the spatial details and semantic features of MR-guidance image more effectively by introducing encoder-decoder and deep decoder subnetworks. Moreover, the specific shapes and pat…
Heterogeneous window transformer for image denoising
Northwestern Polytechnical University（西北工业大学）；National Tsing Hua University（国立清华大学）；Xidian University（西安电子科技大学）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）
School of Mathematics and Statistics, Wuhan University（武汉大学数学与统计学学院）；Department of Computer and Information Science, Faculty of Science and Technology, University of Macau（澳门大学计算机与信息科学系）；Hubei Center for Applied Mathematics, Wuhan University（湖北应用数学中心，武汉大学）；Hubei Key Laboratory of Computational Science, Wuhan University（湖北计算科学重点实验室，武汉大学）
Xiamen University（厦门大学）；The University of Hong Kong（香港大学）；Deepwise（深睿医疗）；Columbia University（哥伦比亚大学）
The effectiveness of existing denoising algorithms typically relies on accurate pre-defined noise statistics or plenty of paired data, which limits their practicality. In this work, we focus on denoising in the more common case where noise statistics and paired data are unavailable. Considering that denoising CNNs require supervision, we develop a new \textbf{adaptive noise imitation (ADANI)} algorithm that can synthesize noisy data from naturally noisy images. To produce realistic noise, a noise generator takes unpaired noisy/clean images as input, where the noisy image is a guide for noise g…
Hipandas: Hyperspectral Image Joint Denoising and Super-Resolution by Image Fusion with the Panchromatic Image
Northwestern Polytechnical University（西北工业大学）；Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；ETH Zürich（苏黎世联邦理工学院）；Xi’an Jiaotong University（西安交通大学）
提出零样本 Hipandas 框架，通过三网络耦合、两类低秩先验和两阶段训练，联合利用 PAN 图像完成高光谱去噪与超分辨率，性能优于现有方法。
Taishan University（泰山学院）；Nankai University（南开大学）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Guangxi Normal University（广西师范大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Non-local self similarity (NSS) is a powerful prior of natural images for image denoising. Most of existing denoising methods employ similar patches, which is a patch-level NSS prior. In this paper, we take one step forward by introducing a pixel-level NSS prior, i.e., searching similar pixels across a non-local region. This is motivated by the fact that finding closely similar pixels is more feasible than similar patches in natural images, which can be used to enhance image denoising performance. With the introduced pixel-level NSS prior, we propose an accurate noise level estimation method, …
Image denoising methods must effectively model, implicitly or explicitly, the vast diversity of patterns and textures that occur in natural images. This is challenging, even for modern methods that leverage deep neural networks trained to regress to clean images from noisy inputs. One recourse is to rely on "internal" image statistics, by searching for similar patterns within the input image itself. In this work, we propose a new method for natural image denoising that trains a deep neural network to determine whether patches in a noisy image input share common underlying patterns. Given a pai…
Hangzhou Dianzi University（杭州电子科技大学）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）
The depth images denoising are increasingly becoming the hot research topic nowadays because they reflect the three-dimensional (3D) scene and can be applied in various fields of computer vision. But the depth images obtained from depth camera usually contain stains such as noise, which greatly impairs the performance of depth related applications. In this paper, considering that group-based image restoration methods are more effective in gathering the similarity among patches, a group based nuclear norm and learning graph (GNNLG) model was proposed. For each patch, we find and group the most …
Xidian University（西安电子科技大学）；West Virginia University（西弗吉尼亚大学）
Deep neural networks (DNNs) based methods have achieved great success in single image super-resolution (SISR). However, existing state-of-the-art SISR techniques are designed like black boxes lacking transparency and interpretability. Moreover, the improvement in visual quality is often at the price of increased model complexity due to black-box design. In this paper, we present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN). Targeting at breaking the coherence barrier, we opt to work with a well-established image prior named nonlocal auto-…
McGill University（麦吉尔大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）
In recent years, self-supervised denoising methods have shown impressive performance, which circumvent painstaking collection procedure of noisy-clean image pairs in supervised denoising methods and boost denoising applicability in real world. One of well-known self-supervised denoising strategies is the blind-spot training scheme. However, a few works attempt to improve blind-spot based self-denoiser in the aspect of network architecture. In this paper, we take an intuitive view of blind-spot strategy and consider its process of using neighbor pixels to predict manipulated pixels as an inpain…
ETH Zürich（苏黎世联邦理工学院）；University of Wurzburg（维尔茨堡大学）；KU Leuven（鲁汶大学）
Video denoising aims at removing noise from videos to recover clean ones. Some existing works show that optical flow can help the denoising by exploiting the additional spatial-temporal clues from nearby frames. However, the flow estimation itself is also sensitive to noise, and can be unusable under large noise levels. To this end, we propose a new multi-scale refined optical flow-guided video denoising method, which is more robust to different noise levels. Our method mainly consists of a denoising-oriented flow refinement (DFR) module and a flow-guided mutual denoising propagation (FMDP) mo…
MDAA-Diff: CT-Guided Multi-Dose Adaptive Attention Diffusion Model for PET Denoising
School of Biomedical Engineering, Southern Medical University（南方医科大学生物医学工程学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Nanfang PET Center, Nanfang Hospital Southern Medical University（南方医科大学南芳医院PET中心）
Depth images captured by off-the-shelf RGB-D cameras suffer from much stronger noise than color images. In this paper, we propose a method to denoise the depth images in RGB-D images by color-guided graph filtering. Our iterative method contains two components: color-guided similarity graph construction, and graph filtering on the depth signal. Implemented in graph vertex domain, filtering is accelerated as computation only occurs among neighboring vertices. Experimental results show that our method outperforms state-of-art depth image denoising methods significantly both on quality and effici…
Cycle-guided Denoising Diffusion Probability Model for 3D Cross-modality MRI Synthesis
Emory University（埃默里大学）；Winship Cancer Institute（温希普癌症研究所）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Graz University of Technology（格拉茨工业大学）；GE Healthcare（GE医疗）
Purpose: To investigate feasibility of accelerating prostate diffusion-weighted imaging (DWI) by reducing the number of acquired averages and denoising the resulting image using a proposed guided denoising convolutional neural network (DnCNN). Materials and Methods: Raw data from the prostate DWI scans were retrospectively gathered (between July 2018 and July 2019) from six single-vendor MRI scanners. 118 data sets were used for training and validation (age: 64.3 +- 8 years) and 37 - for testing (age: 65.1 +- 7.3 years). High b-value diffusion-weighted (hb-DW) data were reconstructed into nois…
Low-dose CT Denoising with Language-engaged Dual-space Alignment
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学脑智能科学与技术研究院）；Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute（伦斯勒理工学院生物技术与跨学科研究中心）
Zhejiang University-University of Illinois Urbana-Champaign Institute（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院）；Zhejiang University（浙江大学）
Reducing the radiation dose in computed tomography (CT) is important to mitigate radiation-induced risks. One option is to employ a well-trained model to compensate for incomplete information and map sparse-view measurements to the CT reconstruction. However, reconstruction from sparsely sampled measurements is insufficient to uniquely characterize an object in CT, and a learned prior model may be inadequate for unencountered cases. Medical modal translation from magnetic resonance imaging (MRI) to CT is an alternative but may introduce incorrect information into the synthesized CT images in a…
Thresholding of Curvelet Coefficients, for image denoising, drains out subtle signal component in noise subspace. This produces ringing artifacts near edges and granular effect in the denoised image. We found the noise sensitivity of Curvelet phases (in contrast to their magnitude) reduces with higher noise level. Thus, we preserved the phase of the coefficients below threshold at coarser scale and estimated their magnitude by Joint Bilateral Filtering (JBF) technique from the thresholded and noisy coefficients. In the finest scale, we apply Bilateral Filter (BF) to keep edge information. Furt…
We propose a novel direction to improve the denoising quality of filtering-based denoising algorithms in real time by predicting the best filter parameter value using a Convolutional Neural Network (CNN). We take the use case of BM3D, the state-of-the-art filtering-based denoising algorithm, to demonstrate and validate our approach. We propose and train a simple, shallow CNN to predict in real time, the optimum filter parameter value, given the input noisy image. Each training example consists of a noisy input image (training data) and the filter parameter value that produces the best output (…
