De-Noising of Photoacoustic Microscopy Images by Deep Learning
University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University（上海交通大学密西根学院）；Biomedical Engineering Department, Peking University（北京大学生物医学工程系）
Da He, Jiasheng Zhou, Xiaoyu Shang, Jiajia Luo, Sung-Liang Chen
As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is a post-processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on mathematical priors as well as manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove complex noise from PAM images without mathematical priors and manual selection of settings for different input images. An attention enhanced generative adversarial network is used to extract image features and remove various noises. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising speed of 0.016 s is achieved for an image with $256\times256$ pixels. Our approach is effective and practical for the de-noising of PAM images.
As an important branch of photoacoustic microscopy, optical-resolution photoacoustic microscopy suffers from limited depth of field due to the strongly focused laser beam. In this work, a 3D information fusion algorithm based on 3D stationary wavelet transform and joint weighted evaluation optimization is proposed to fuse multi-focus photoacoustic data to achieve large-volumetric and high-resolution 3D imaging. First, a three-dimensional stationary wavelet transform was performed on the multi-focus data to obtain eight wavelet coefficients. Differential evolution algorithm based on joint weigh…
One primary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed. In this study, we propose a novel application of deep learning principles to reconstruct undersampled PAM images and transcend the trade-off between spatial resolution and imaging speed. We compared various convolutional neural network (CNN) architectures, and selected a fully dense U-net (FD U-net) model that produced the best results. To mimic various undersampling conditions in practice, we artificially downsampled fully-sampled PAM images of mouse brai…
We have developed a multimodal photoacoustic remote sensing (PARS) microscope combined with swept source optical coherence tomography for in vivo, non-contact retinal imaging. Building on the proven strength of multiwavelength PARS imaging, the system is applied for estimating retinal oxygen saturation in the rat retina. The capability of the technology is demonstrated by imaging both microanatomy and the microvasculature of the retina in vivo. To our knowledge this is the first time a non-contact photoacoustic imaging technique is employed for in vivo oxygen saturation measurement in the reti…
Photoacoustic imaging is a new imaging technology in recent years, which combines the advantages of high resolution and rich contrast of optical imaging with the advantages of high penetration depth of acoustic imaging. Photoacoustic imaging has been widely used in biomedical fields, such as brain imaging, tumor detection and so on. The signal-to-noise ratio (SNR) of image signals in photoacoustic imaging is generally low due to the limitation of laser pulse energy, electromagnetic interference in the external environment and system noise. In order to solve the problem of low SNR of photoacous…
Mohs micrographic surgery (MMS) is a precise oncological technique where layers of tissue are resected and examined with intraoperative histopathology to minimize the removal of normal tissue while completely excising the cancer. To achieve intraoperative pathology, the tissue is frozen, sectioned and stained over a 20- to 60-minute period, then analyzed by the MMS surgeon. Surgery is continued one layer at a time until no cancerous cells remain, meaning MMS can take several hours to complete. Ideally, it would be desirable to circumvent or augment frozen sectioning methods and directly visual…
Histological images are critical in the diagnosis and treatment of cancers. Unfortunately, the current method for capturing these microscopy images require resource intensive tissue preparation that delays diagnosis for many days to a few weeks. To streamline this process, clinicians are limited to assessing small macroscopically representative subsets of tissues. Here, we present a combined photoacoustic remote sensing (PARS) microscope and swept source optical coherence tomography (SS-OCT) system designed to circumvent these diagnostic limitations. The proposed multimodal microscope provides…
Duke University（杜克大学）；Tsinghua University（清华大学）；Brigham and Women’s Hospital（布列根和妇女医院）；Harvard Medical School（哈佛医学院）；University of Southern California（南加州大学）
Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited by the laser's repetition rate, state-of-the-art high-speed PAM technology often sacrifices spatial sampling density (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have recently been used to improve sparsely sampled PAM images; however, these methods often require time-consuming pre-training and large training dataset with ground truth. Here, we propose the use of deep image prior (DIP) to improve the image quality of undersampled PA…
University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University（上海交通大学密西根学院）；Peking University（北京大学）
Photoacoustic microscopy (PAM) has been a promising biomedical imaging technology in recent years. However, the point-by-point scanning mechanism results in low-speed imaging, which limits the application of PAM. Reducing sampling density can naturally shorten image acquisition time, which is at the cost of image quality. In this work, we propose a method using convolutional neural networks (CNNs) to improve the quality of sparse PAM images, thereby speeding up image acquisition while keeping good image quality. The CNN model utilizes both squeeze-and-excitation blocks and residual blocks to a…
Dc-EEMF: Pushing depth-of-field limit of photoacoustic microscopy via decision-level constrained learning
Center for Biomedical-photonics and Molecular Imaging（生物医学光子学与分子成像中心）；Xi’an Key Laboratory of Intelligent Sensing and Regulation of trans-Scale Life Information（西安智能感知与跨尺度生命信息调节重点实验室）；School of Life Science and Technology（生命科学与技术学院）；School of Computer Science and Engineering（计算机科学与工程学院）；Faculty of Health Sciences（健康科学学院）；Laboratory of Carcinogenesis and Translational Research (Ministry of Education/Beijing)（致癌研究与转化医学实验室（教育部/北京））
Malignant brain tumors are among the deadliest neoplasms with the lowest survival rates of any cancer type. In considering surgical tumor resection, suboptimal extent of resection is linked to poor clinical outcomes and lower overall survival rates. Currently available tools for intraoperative histopathological assessment require an average of 20 minutes processing and are of limited diagnostic quality for guiding surgeries. Consequently, there is an unaddressed need for a rapid imaging technique to guide maximal resection of brain tumors. Working towards this goal, presented here is an all op…
We developed a Bessel-beam photoacoustic microscopical simulation platform by using the k-Wave: MATLAB toolbox. The simulation platform uses the ring slit method to generate Bessel beam. By controlling the inner and outer radius of the ring slit, the depth-of-field (DoF) of Bessel beam can be controlled. And the large volumetric image is obtained by point scanning. The simulation experiments on blood vessels was carried out to demonstrate the feasibility of the simulation plat-form. This simulation work can be used as an auxiliary tool for the research of Bessel-beam photoacoustic microscopy.
Volume Tells: Dual Cycle-Consistent Diffusion for 3D Fluorescence Microscopy De-noising and Super-Resolution
Department of Electrical Engineering, City University of Hong Kong（香港城市大学电气工程系）；Centre for Intelligent Multidimensional Data Analysis, Hong Kong Science Park（香港科学园智能多维数据分析中心）；Department of Biology, Hong Kong Baptist University（香港浸会大学生物学系）
Acoustic-resolution photoacoustic microscopy (AR-PAM) is a promising tool for microvascular imaging. In the focal region, resolution of AR-PAM is determined by the ultrasound transducer and ultimately limited by acoustic diffraction. In the out-of-focus region, resolution deteriorates with increasing distance from the focal plane, which restricts depth of focus (DOF). Besides, a trade-off exists between resolution and DOF. Previously, synthetic aperture focusing technique (SAFT) and/or deconvolution methods have been demonstrated to enhance AR-PAM images. However, they suffer from issues in lo…
Histological visualizations are critical to clinical disease management and are fundamental to biological understanding. However, current approaches that rely on bright-field microscopy require extensive tissue preparation prior to imaging. These processes are labor intensive and contribute to delays in clinical feedback that can extend to two to three weeks for standard paraffin-embedded tissue preparation and interpretation. Here, we present a label-free reflection-mode imaging modality that reveals cellular-scale morphology by detecting intrinsic endogenous contrast. We accomplish this with…
Significance: Histopathological analysis of tissues is an essential tool for grading, staging, diagnosing and resecting cancers and other malignancies. Current histopathological techniques require substantial sample processing prior to staining with hematoxylin and eosin (H&E) dyes, to highlight nuclear and cellular morphology. Sample preparation and staining is resource-intensive and introduces potential for variability during sample preparation. Aim: We present a novel method for direct label-free histopathological assessment of formalin fixed paraffin embedded tissue blocks and thin tissue …
Photoacoustic microscopy is becoming an important tool for the biomedical research. It has been widely used in biological researches, such as structural imaging of vasculature, brain structural and functional imaging, and tumor detection. The conventional optical-resolution photoacoustic microscopy (OR- PAM) employs focused gaussian beam to achieve high lateral resolution by a microscope objective with high numerical apertures. Since the focused gaussian beam only has narrow depth range in focus, little detail in depth direction can be revealed. Here, we developed a synthetic multi-focus optic…
Photoacoustic microscopy with large depth of focus is significant to the biomedical research. The conventional optical-resolution photoacoustic microscope (OR-PAM) suffers from limited depth of field (DoF) since the employed focused Gaussian beam only has a narrow depth range in focus, little details in depth direction can be revealed. Here, we developed a computed extended depth of field method for photoacoustic microscope by using wavelet transform image fusion rules. Wavelet transform is performed on the max amplitude projection (MAP) images acquired at different axial positions by OR-PAM t…
