Ministry of Industry and Information Technology Key Lab of Micro-Nano Optoelectronic Information System（工业和信息化部微纳光电信息系统重点实验室）；Guangdong Provincial Key Laboratory of Semiconductor Optoelectronic Materials and Intelligent Photonic System（广东省半导体光电材料与智能光子系统重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；School of Electronics and Information Engineering（电子与信息工程学院）；Zhejiang Provincial Key Laboratory of Intelligent Vehicle Electronics Research（浙江省智能车辆电子研究所）；Hangzhou Dianzi University（杭州电子科技大学）；School of Science and Engineering（科学与工程学院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Quantum Science Center of Guangdong-Hong Kong-Macao Greater Bay Area（粤港澳大湾区量子科学中心）；Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education（教育部光电一体化感知与通信技术重点实验室）；Guangdong University of Technology（广东工业大学）
Xudong Lv, Yuxiang Sun, Shuo Wang, Nanxing Chen, Jun Guan, Jingtian Hu
Optical neural networks are emerging as powerful machine learning and information processing tools because of their potential advantages in speed and energy efficiency. The training methods of these physical models, however, remain underexplored compared to their digital counterparts and are leading to suboptimal performance. This paper reports a pre-training-driven approach that leads to snapshot image denoising with substantially improved quality. We demonstrated effective free-space optical denoising by a diffractive network optimized by a two-step process including (1) pre-training using a massive dataset of 3.45 million diverse but simple images and (2) fine-tuning with the corresponding task-specific datasets. Compared to conventional Fourier-domain filtering and directly trained diffractive networks, such a transfer learning process exhibited prominent advantages for denoising images degraded by severe noise, peak signal-to-noise ratio (PSNR) below 8 dB, while preserving fine image features and improving the PSNR to above 18 dB. Importantly, the same pre-trained optical network could be consistently fine-tuned to process degraded images from highly diverse styles ranging from handwritten digits (MNIST) and chest X-rays (ChestMNIST) to CIFAR-10 images and human faces (CelebA). We further demonstrated the critical role of our optical denoisers in vision-based applications, including face detection, plate recognition, and localization of UAVs in noisy conditions.
Beijing University of Posts and Telecommunications（北京邮电大学）；GuangDong TUS-TuWei Technology Co.,Ltd（广东拓思途维科技有限公司）
While deep Convolutional Neural Networks (CNNs) have shown extraordinary capability of modelling specific noise and denoising, they still perform poorly on real-world noisy images. The main reason is that the real-world noise is more sophisticated and diverse. To tackle the issue of blind denoising, in this paper, we propose a novel pyramid real image denoising network (PRIDNet), which contains three stages. First, the noise estimation stage uses channel attention mechanism to recalibrate the channel importance of input noise. Second, at the multi-scale denoising stage, pyramid pooling is util…
Instituto de Telecomunicações（葡萄牙电信研究所）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等理工学院）
In this paper, we address the problem of denoising images degraded by Poisson noise. We propose a new patch-based approach based on best linear prediction to estimate the underlying clean image. A simplified prediction formula is derived for Poisson observations, which requires the covariance matrix of the underlying clean patch. We use the assumption that similar patches in a neighborhood share the same covariance matrix, and we use off-the-shelf Poisson denoising methods in order to obtain an initial estimate of the covariance matrices. Our method can be seen as a post-processing step for Po…
The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）
The lack of large-scale noisy-clean image pairs restricts supervised denoising methods' deployment in actual applications. While existing unsupervised methods are able to learn image denoising without ground-truth clean images, they either show poor performance or work under impractical settings (e.g., paired noisy images). In this paper, we present a practical unsupervised image denoising method to achieve state-of-the-art denoising performance. Our method only requires single noisy images and a noise model, which is easily accessible in practical raw image denoising. It performs two steps it…
A wide variety of image denoising methods are available now. However, the performance of a denoising algorithm often depends on individual input noisy images as well as its parameter setting. In this paper, we present a no-reference image denoising quality assessment method that can be used to select for an input noisy image the right denoising algorithm with the optimal parameter setting. This is a challenging task as no ground truth is available. This paper presents a data-driven approach to learn to predict image denoising quality. Our method is based on the observation that while individua…
Self-supervised frameworks that learn denoising models with merely individual noisy images have shown strong capability and promising performance in various image denoising tasks. Existing self-supervised denoising frameworks are mostly built upon the same theoretical foundation, where the denoising models are required to be J-invariant. However, our analyses indicate that the current theory and the J-invariance may lead to denoising models with reduced performance. In this work, we introduce Noise2Same, a novel self-supervised denoising framework. In Noise2Same, a new self-supervised loss is …
Recently, Self-supervised learning methods able to perform image denoising without ground truth labels have been proposed. These methods create low-quality images by adding random or Gaussian noise to images and then train a model for denoising. Ideally, it would be beneficial if one can generate high-quality CT images with only a few training samples via self-supervision. However, the performance of CT denoising is generally limited due to the complexity of CT noise. To address this problem, we propose a novel self-supervised learning-based CT denoising method. In particular, we train pre-tra…
Xiamen University（厦门大学）；University of Science and Technology of China（中国科学技术大学）；Columbia University（哥伦比亚大学）
We propose a new framework called Noise2Blur (N2B) for training robust image denoising models without pre-collected paired noisy/clean images. The training of the model requires only some (or even one) noisy images, some random unpaired clean images, and noise-free but blurred labels obtained by predefined filtering of the noisy images. The N2B model consists of two parts: a denoising network and a noise extraction network. First, the noise extraction network learns to output a noise map using the noise information from the denoising network under the guidence of the blurred labels. Then, the …
Heterogeneous window transformer for image denoising
Northwestern Polytechnical University（西北工业大学）；National Tsing Hua University（国立清华大学）；Xidian University（西安电子科技大学）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）
An image processing unit (IPU), or image signal processor (ISP) for high dynamic range (HDR) imaging usually consists of demosaicing, white balancing, lens shading correction, color correction, denoising, and tone-mapping. Besides noise from the imaging sensors, almost every step in the ISP introduces or amplifies noise in different ways, and denoising operators are designed to reduce the noise from these sources. Designed for dynamic range compressing, tone-mapping operators in an ISP can significantly amplify the noise level, especially for images captured in low-light conditions, making den…
Image denoising is a classical signal processing problem that has received significant interest within the image processing community during the past two decades. Most of the algorithms for image denoising has focused on the paradigm of non-local similarity, where image blocks in the neighborhood that are similar, are collected to build a basis for reconstruction. Through rigorous experimentation, this paper reviews multiple aspects of image denoising algorithm development based on non-local similarity. Firstly, the concept of non-local similarity as a foundational quality that exists in natur…
King Fahd University of Petroleum and Minerals（法赫德国王石油与矿业大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
In spite of the improvements achieved by the several denoising algorithms over the years, many of them still fail at preserving the fine details of the image after denoising. This is as a result of the smooth-out effect they have on the images. Most neural network-based algorithms have achieved better quantitative performance than the classical denoising algorithms. However, they also suffer from qualitative (visual) performance as a result of the smooth-out effect. In this paper, we propose an algorithm to address this shortcoming. We propose a deep collaborative filtering-based (Deep-CoFiB) …
In this letter we present an analytical theory of Extraordinary optical transmission (EOT) through semi-transparent screens, such as thin metallic plates or high permittivity dielectric slabs. Using this theory we show that EOT appears not only for screens perforated by holes or slits, but also for screens loaded by any defects, including opaque defects. These results widen the scope of EOT concept, opening up the way to the study of new physical effects.
Extraordinary optical transmission and good focusing properties of a two-dimensional scattering structure is presented. The structure is made of Fresnel zone plates periodically arranged along two orthogonal directions. Each plate consists of two ring-shaped waveguides supporting modes that match the symmetry of a circularly polarized incident plane wave. High field concentration at the focal plane is obtained with short transverse and long longitudinal foci diameters. Optical vortex excitation in a paraxial region of the transmitted field is also observed and analysed in terms of cross-polari…
We report on the feedback-optimized extraordinary optical transmission of continuous-variable entangled states through a hexagonal metal-hole array. The continuous-variable entanglements from a nondegenerate optical parametric amplifier are first demonstrated to survive after a photon-plasmon-photon conversion process. By controlling the reflected light from the metal-hole array, a significant enhancement of quantum correlations has then been experimentally achieved comparing to the case of without such coherent feedback control. This result presents a useful technique to efficiently recover t…
Optical lenses are pervasive in various areas of sciences and technologies. It is well-known that the resolving power of a lens and thus optical systems is limited by the diffraction of light. Recently, various plasmonics and metamaterials based superlenses have been emerging to achieve super resolution. Here, we show that the phase compensated negative refraction lenses perform as "Janus Lenses", i.e. either converging lenses or diverging lenses depending on the illumination directions. Extraordinary imaging equations and properties that are different from those of all the existing optical le…
This paper has presented inversely determining the resonant configuration of the bilateral nanostructures for periodic metallic slits with extraordinary optical transmission performance. The topology optimization approach is utilized to implement the inverse design procedure. Several geometrical configurations of the bilateral nanostructures are derived for periodic metallic slits. The resonant performance of the derived nanostructures are demonstrated by the transmission spectra, where the transmission peak is presented at the specified wavelength in the inverse design procedure. This provide…
Plasmonic metasurfaces based on the extraordinary optical transmission effect (EOT) can be designed to efficiently transmit specific spectral bands from the visible to the far-infrared regimes, offering numerous applications in im-portant technological fields such as compact multispectral imaging, biological and chemical sensing, or color displays. However, due to their subwavelength nature, EOT metasurfaces are nowadays fabricated with nano- and micro-lithographic techniques, requiring many processing steps and carried out in expensive cleanroom environments. In this work, we propose and expe…
We provide a vivid demonstration of the mechanical effect of transverse spin momentum in an optical beam in free space. This component of the Poynting momentum was previously thought to be virtual, and unmeasurable. Here, its effect is revealed in the inertial motion of a probe particle in a circularly polarized Gaussian trap, in vacuum. Transverse spin forces combine with thermal fluctuations to induce a striking range of non-equilibrium phenomena. With increasing beam power we observe (i) growing departures from energy equipartition, (ii) the formation of coherent, thermally excited orbits a…
Known since Kepler's observation that a comet's tail is oriented away from the sun, radiation pressure stimulated remarkable discoveries in electromagnetism, quantum physics and relativity [1,2]. This phenomenon plays a crucial role in a variety of systems, from atomic [3-5] to astronomical [6] scales. The pressure of light is associated with the momentum of photons, and it is usually assumed that both the optical momentum and the radiation-pressure force are naturally aligned with the propagation of light, i.e., its wavevector. Here we report the direct observation of an extraordinary optical…
In modern terahertz (THz) sensing and imaging spectroscopy, water is considered a nemesis to be avoided due to strong absorption in the THz frequency range. Here we report the first experimental demonstration and theoretical implications of using femtosecond laser pulses to generate intense broadband THz emission from water vapor. When we focused an intense laser pulse in water vapor contained in a gas cell or injected from a gas jet nozzle, an extraordinarily strong THz field from optically excited water vapor is observed. Water vapor has more than 50% greater THz generation efficiency than d…
