Ultra Low-Parameter Denoising: Trainable Bilateral Filter Layers in Computed Tomography
Friedrich-Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Fraunhofer Institute for Ceramic Technologies and Systems IKTS（弗劳恩霍夫陶瓷技术与系统研究所（IKTS））；Leibniz Institute for Analytical Sciences ISAS（莱布尼茨分析科学研究所（ISAS））；University Hospital Erlangen（埃尔朗根大学医院）；Institute for Nanotechnology and Correlative Microscopy e.V. INAM（纳米技术与关联显微镜协会研究所（INAM））
Fabian Wagner, Mareike Thies, Mingxuan Gu, Yixing Huang, Sabrina Pechmann, Mayank Patwari, Stefan Ploner, Oliver Aust, Stefan Uderhardt, Georg Schett, Silke Chr… 展开作者
Fabian Wagner, Mareike Thies, Mingxuan Gu, Yixing Huang, Sabrina Pechmann, Mayank Patwari, Stefan Ploner, Oliver Aust, Stefan Uderhardt, Georg Schett, Silke Christiansen, Andreas Maier
Computed tomography is widely used as an imaging tool to visualize three-dimensional structures with expressive bone-soft tissue contrast. However, CT resolution and radiation dose are tightly entangled, highlighting the importance of low-dose CT combined with sophisticated denoising algorithms. Most data-driven denoising techniques are based on deep neural networks and, therefore, contain hundreds of thousands of trainable parameters, making them incomprehensible and prone to prediction failures. Developing understandable and robust denoising algorithms achieving state-of-the-art performance helps to minimize radiation dose while maintaining data integrity. This work presents an open-source CT denoising framework based on the idea of bilateral filtering. We propose a bilateral filter that can be incorporated into a deep learning pipeline and optimized in a purely data-driven way by calculating the gradient flow toward its hyperparameters and its input. Denoising in pure image-to-image pipelines and across different domains such as raw detector data and reconstructed volume, using a differentiable backprojection layer, is demonstrated. Although only using three spatial parameters and one range parameter per filter layer, the proposed denoising pipelines can compete with deep state-of-the-art denoising architectures with several hundred thousand parameters. Competitive denoising performance is achieved on x-ray microscope bone data (0.7053 and 33.10) and the 2016 Low Dose CT Grand Challenge dataset (0.9674 and 43.07) in terms of SSIM and PSNR. Due to the extremely low number of trainable parameters with well-defined effect, prediction reliance and data integrity is guaranteed at any time in the proposed pipelines, in contrast to most other deep learning-based denoising architectures.
Low-dose computed tomography (CT) denoising algorithms aim to enable reduced patient dose in routine CT acquisitions while maintaining high image quality. Recently, deep learning~(DL)-based methods were introduced, outperforming conventional denoising algorithms on this task due to their high model capacity. However, for the transition of DL-based denoising to clinical practice, these data-driven approaches must generalize robustly beyond the seen training data. We, therefore, propose a hybrid denoising approach consisting of a set of trainable joint bilateral filters (JBFs) combined with a co…
Friedrich-Alexander Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Siemens Healthcare GmbH（西门子医疗有限公司）
Deep neural networks have shown great success in low dose CT denoising. However, most of these deep neural networks have several hundred thousand trainable parameters. This, combined with the inherent non-linearity of the neural network, makes the deep neural network diffcult to understand with low accountability. In this study we introduce JBFnet, a neural network for low dose CT denoising. The architecture of JBFnet implements iterative bilateral filtering. The filter functions of the Joint Bilateral Filter (JBF) are learned via shallow convolutional networks. The guidance image is estimated…
In the past few decades, to reduce the risk of X-ray in computed tomography (CT), low-dose CT image denoising has attracted extensive attention from researchers, which has become an important research issue in the field of medical images. In recent years, with the rapid development of deep learning technology, many algorithms have emerged to apply convolutional neural networks to this task, achieving promising results. However, there are still some problems such as low denoising efficiency, over-smoothed result, etc. In this paper, we propose the Edge enhancement based Densely connected Convol…
Thresholding of Curvelet Coefficients, for image denoising, drains out subtle signal component in noise subspace. This produces ringing artifacts near edges and granular effect in the denoised image. We found the noise sensitivity of Curvelet phases (in contrast to their magnitude) reduces with higher noise level. Thus, we preserved the phase of the coefficients below threshold at coarser scale and estimated their magnitude by Joint Bilateral Filtering (JBF) technique from the thresholded and noisy coefficients. In the finest scale, we apply Bilateral Filter (BF) to keep edge information. Furt…
Friedrich-Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Siemens Healthineers AG（西门子医疗集团）
The use of deep learning has successfully solved several problems in the field of medical imaging. Deep learning has been applied to the CT denoising problem successfully. However, the use of deep learning requires large amounts of data to train deep convolutional networks (CNNs). Moreover, due to large parameter count, such deep CNNs may cause unexpected results. In this study, we introduce a novel CT denoising framework, which has interpretable behaviour, and provides useful results with limited data. We employ bilateral filtering in both the projection and volume domains to remove noise. To…
In this letter, we propose a novel low-rank quaternion approximation (LRQA) model by directly constraining the quaternion rank prior for effectively removing the noise in color images. The LRQA model treats the color image holistically rather than independently for the color space components, thus it can fully utilize the high correlation among RGB channels. We design an iterative algorithm by using quaternion bilateral random projections (Q-BRP) to efficiently optimize the proposed model. The main advantage of Q-BRP is that the approximation of the low-rank quaternion matrix can be obtained q…
Friedrich-Alexander University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；Siemens Healthcare GmbH（西门子医疗有限公司）
Denoising of clinical CT images is an active area for deep learning research. Current clinically approved methods use iterative reconstruction methods to reduce the noise in CT images. Iterative reconstruction techniques require multiple forward and backward projections, which are time-consuming and computationally expensive. Recently, deep learning methods have been successfully used to denoise CT images. However, conventional deep learning methods suffer from the 'black box' problem. They have low accountability, which is necessary for use in clinical imaging situations. In this paper, we us…
In this paper, we propose a regularization technique for noisy-image super-resolution and image denoising. Total variation (TV) regularization is adopted in many image processing applications to preserve the local smoothness. However, TV prior is prone to oversmoothness, staircasing effect, and contrast losses. Nonlocal TV (NLTV) mitigates the contrast losses by adaptively weighting the smoothness based on the similarity measure of image patches. Although it suppresses the noise effectively in the flat regions, it might leave residual noise surrounding the edges especially when the image is no…
In this paper, we investigate whether is it possible to train a neural network directly from user inputs. We consider this approach to be highly relevant for applications in which the point of optimality is not well-defined and user-dependent. Our application is medical image denoising which is essential in fluoroscopy imaging. In this field every user, i.e. physician, has a different flavor and image quality needs to be tailored towards each individual. To address this important problem, we propose to construct a loss function derived from a forced-choice experiment. In order to make the lear…
Deep Unfolded BM3D: Unrolling Non-local Collaborative Filtering into a Trainable Neural Network
Electronics and Communication Engineering Department, Istanbul Technical University, Istanbul, Turkey（伊斯坦布尔技术大学电子与通信工程系）；Electrical and Electronics Engineering Department, Istanbul University, Istanbul, Turkey（伊斯坦布尔大学电气与电子工程系）
We set out to simulate four reduced dose-levels (60%-dose, 40%-dose, 20%-dose, and 10%-dose) of standard CT imaging using Beer-Lambert's law across 49 patients infected with COVID-19. Then, three denoising filters, namely Gaussian, Bilateral, and Median, were applied to the different low-dose CT images, the quality of which was assessed prior to and after the application of the various filters via calculation of peak signal-to-noise ratio (PSNR), root mean square error (RMSE), structural similarity index measure (SSIM), and relative CT-value bias, separately for the lung tissue and whole-body.…
Low-light imaging on mobile devices is typically challenging due to insufficient incident light coming through the relatively small aperture, resulting in a low signal-to-noise ratio. Most of the previous works on low-light image processing focus either only on a single task such as illumination adjustment, color enhancement, or noise removal; or on a joint illumination adjustment and denoising task that heavily relies on short-long exposure image pairs collected from specific camera models, and thus these approaches are less practical and generalizable in real-world settings where camera-spec…
Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspectral signals acquired on the spot makes restoration tasks challenging. In particular, training deep neural networks for restoration is difficult, in contrast to traditional RGB imaging problems where d…
Ryerson University（瑞尔森大学）；University of Saskatoon Health Region（萨斯卡通健康区域大学）
Low-dose CT denoising is a challenging task that has been studied by many researchers. Some studies have used deep neural networks to improve the quality of low-dose CT images and achieved fruitful results. In this paper, we propose a deep neural network that uses dilated convolutions with different dilation rates instead of standard convolution helping to capture more contextual information in fewer layers. Also, we have employed residual learning by creating shortcut connections to transmit image information from the early layers to later ones. To further improve the performance of the netwo…
King Fahd University of Petroleum and Minerals（法赫德国王石油与矿产大学）
The recent application of deep learning (DL) to various tasks has seen the performance of classical techniques surpassed by their DL-based counterparts. As a result, DL has equally seen application in the removal of noise from images. In particular, the use of deep feed-forward convolutional neural networks (DnCNNs) has been investigated for denoising. It utilizes advances in DL techniques such as deep architecture, residual learning, and batch normalization to achieve better denoising performance when compared with the other classical state-of-the-art denoising algorithms. However, its deep a…
During the acquisition of an image from its source, noise always becomes an integral part of it. Various algorithms have been used in past to denoise the images. Image denoising still has scope for improvement. Visual information transmitted in the form of digital images has become a considerable method of communication in the modern age, but the image obtained after the transmission is often corrupted due to noise. In this paper, we review the existing denoising algorithms such as filtering approach and wavelets based approach and then perform their comparative study with bilateral filters. W…
Filter2Noise: A Framework for Interpretable and Zero-Shot Low-Dose CT Image Denoising
Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg（模式识别实验室，弗里德里希-亚历山大大学埃尔朗根-纽伦堡）；Siemens Healthineers AG（西门子医疗影像有限公司）；Department of Radiology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（放射科，复杂严重罕见疾病国家重点实验室，北京协和医院，中国医学科学院及北京协和医学院）；Theranostics and Translational Research Center, National Infrastructures for Translational Medicine, Institute of Clinical Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（诊疗与转化研究中心，国家转化医学基础设施，临床医学研究院，北京协和医院，中国医学科学院及北京协和医学院）
Northeastern University（东北大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Due to the widespread use of positron emission tomography (PET) in clinical practice, the potential risk of PET-associated radiation dose to patients needs to be minimized. However, with the reduction in the radiation dose, the resultant images may suffer from noise and artifacts that compromise diagnostic performance. In this paper, we propose a parameter-transferred Wasserstein generative adversarial network (PT-WGAN) for low-dose PET image denoising. The contributions of this paper are twofold: i) a PT-WGAN framework is designed to denoise low-dose PET images without compromising structural…
Convolution has been the core operation of modern deep neural networks. It is well-known that convolutions can be implemented in the Fourier Transform domain. In this paper, we propose to use binary block Walsh-Hadamard transform (WHT) instead of the Fourier transform. We use WHT-based binary layers to replace some of the regular convolution layers in deep neural networks. We utilize both one-dimensional (1-D) and two-dimensional (2-D) binary WHTs in this paper. In both 1-D and 2-D layers, we compute the binary WHT of the input feature map and denoise the WHT domain coefficients using a nonlin…
