Robustness of deep learning methods for limited angle tomography is challenged by two major factors: a) due to insufficient training data the network may not generalize well to unseen data; b) deep learning methods are sensitive to noise. Thus, generating reconstructed images directly from a neural network appears inadequate. We propose to constrain the reconstructed images to be consistent with the measured projection data, while the unmeasured information is complemented by learning based methods. For this purpose, a data consistent artifact reduction (DCAR) method is introduced: First, a prior image is generated from an initial limited angle reconstruction via deep learning as a substitute for missing information. Afterwards, a conventional iterative reconstruction algorithm is applied, integrating the data consistency in the measured angular range and the prior information in the missing angular range. This ensures data integrity in the measured area, while inaccuracies incorporated by the deep learning prior lie only in areas where no information is acquired. The proposed DCAR method achieves significant image quality improvement: for 120-degree cone-beam limited angle tomography more than 10% RMSE reduction in noise-free case and more than 24% RMSE reduction in noisy case compared with a state-of-the-art U-Net based method.
Z 2 Sky Technologies Inc.（Z 2 Sky科技公司）；Adobe（奥多比公司）；University of Rochester（罗切斯特大学）；Beijing United Family Hospital（北京和睦家医院）；Peking Union Medical College Hospital（北京协和医院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）
Metal artifact reduction (MAR) in computed tomography (CT) is a notoriously challenging task because the artifacts are structured and non-local in the image domain. However, they are inherently local in the sinogram domain. Thus, one possible approach to MAR is to exploit the latter characteristic by learning to reduce artifacts in the sinogram. However, if we directly treat the metal-affected regions in sinogram as missing and replace them with the surrogate data generated by a neural network, the artifact-reduced CT images tend to be over-smoothed and distorted since fine-grained details wit…
Wellman Center for Photomedicine, Massachusetts General Hospital（马萨诸塞州总医院韦尔曼光医学中心）；Harvard Medical School（哈佛医学院）；Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）；Erasmus Medical Center（伊拉斯谟医学中心）
Intravascular ultrasound and optical coherence tomography are widely available for characterizing coronary stenoses and provide critical vessel parameters to optimize percutaneous intervention. Intravascular polarization-sensitive optical coherence tomography (PS-OCT) simultaneously provides high-resolution cross-sectional images of vascular structures while also revealing preponderant tissue components such as collagen and smooth muscle and thereby enhances plaque characterization. Automated interpretation of these features promises to facilitate the objective clinical investigation of the na…
Optical Projection Tomography (OPT) is a powerful tool for 3D imaging of mesoscopic samples, thus of great importance to image whole organs for the study of various disease models in life sciences. OPT is able to achieve resolution at a few tens of microns over a large sample volume of several cubic centimeters. However, the reconstructed OPT images often suffer from artifacts caused by different kinds of physical miscalibration. This work focuses on the refractive index (RI) mismatch between the rotating object and the surrounding medium. We derive a 3D cone beam forward model to approximate …
George Mason University（乔治梅森大学）；The MITRE Corporation（MITRE公司）
In photoacoustic tomography (PAT), the acoustic pressure waves produced by optical excitation are measured by an array of detectors and used to reconstruct an image. Sparse spatial sampling and limited-view detection are two common challenges faced in PAT. Reconstructing from incomplete data using standard methods results in severe streaking artifacts and blurring. We propose a modified convolutional neural network (CNN) architecture termed Dense Dilation UNet (DD-UNet) for correcting artifacts in 3D PAT. The DD-Net leverages the benefits of dense connectivity and dilated convolutions to impro…
George Mason University（乔治梅森大学）；The MITRE Corporation（MITRE公司）
Photoacoustic tomography (PAT) is a nonionizing imaging modality capable of acquiring high contrast and resolution images of optical absorption at depths greater than traditional optical imaging techniques. Practical considerations with instrumentation and geometry limit the number of available acoustic sensors and their view of the imaging target, which result in significant image reconstruction artifacts degrading image quality. Iterative reconstruction methods can be used to reduce artifacts but are computationally expensive. In this work, we propose a novel deep learning approach termed pi…
Computed Tomography (CT) is a non-invasive imaging modality with applications ranging from healthcare to security. It reconstructs cross-sectional images of an object using a collection of projection data collected at different angles. Conventional methods, such as FBP, require that the projection data be uniformly acquired over the complete angular range. In some applications, it is not possible to acquire such data. Security is one such domain where non-rotational scanning configurations are being developed which violate the complete data assumption. Conventional methods produce images from …
National Taiwan University（台湾大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Chang Gung University（长庚大学）
Automated vascular segmentation on optical coherence tomography angiography (OCTA) is important for the quantitative analyses of retinal microvasculature in neuroretinal and systemic diseases. Despite recent improvements, artifacts continue to pose challenges in segmentation. Our study focused on removing the speckle noise artifact from OCTA images when performing segmentation. Speckle noise is common in OCTA and is particularly prominent over large non-perfusion areas. It may interfere with the proper assessment of retinal vasculature. In this study, we proposed a novel Supervision Vessel Seg…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Optical diffraction tomography (ODT) produces three dimensional distribution of refractive index (RI) by measuring scattering fields at various angles. Although the distribution of RI index is highly informative, due to the missing cone problem stemming from the limited-angle acquisition of holograms, reconstructions have very poor resolution along axial direction compared to the horizontal imaging plane. To solve this issue, here we present a novel unsupervised deep learning framework, which learns the probability distribution of missing projection views through optimal transport driven cycle…
Shenzhen University（深圳大学）；University of Delaware（特拉华大学）；Chongqing University（重庆大学）；Chongqing University of Science and Technology（重庆科技大学）
In this paper, we first propose a variational model for the limited-angle computed tomography (CT) image reconstruction and then convert the model into an end-to-end deep network.We use the penalty method to solve the model and divide it into three iterative subproblems, where the first subproblem completes the sinograms by utilizing the prior information of sinograms in the frequency domain and the second refines the CT images by using the prior information of CT images in the spatial domain, and the last merges the outputs of the first two subproblems. In each iteration, we use the convoluti…
Argonne National Laboratory（阿贡国家实验室）；Technical University of Denmark（丹麦技术大学）
Resolution level and reconstruction quality in nano-computed tomography (nano-CT) are in part limited by the stability of microscopes, because the magnitude of mechanical vibrations during scanning becomes comparable to the imaging resolution, and the ability of the samples to resist beam damage during data acquisition. In such cases, there is no incentive in recovering the sample state at different time steps like in time-resolved reconstruction methods, but instead the goal is to retrieve a single reconstruction at the highest possible spatial resolution and without any imaging artifacts. He…
Computed Tomography (CT) is an imaging technique where information about an object are collected at different angles (called projections or scans). Then the cross-sectional image showing the internal structure of the slice is produced by solving an inverse problem. Limited by certain factors such as radiation dosage, projection angles, the produced images can be noisy or contain artifacts. Inspired by the success of transformer for natural language processing, the core idea of this preliminary study is to consider a projection of tomography as a word token, and the whole scan of the cross-sect…
Michigan State University（密歇根州立大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）
Object density reconstruction from projections containing scattered radiation and noise is of critical importance in many applications. Existing scatter correction and density reconstruction methods may not provide the high accuracy needed in many applications and can break down in the presence of unmodeled or anomalous scatter and other experimental artifacts. Incorporating machine-learned models could prove beneficial for accurate density reconstruction particularly in dynamic imaging, where the time-evolution of the density fields could be captured by partial differential equations or by le…
Kyoto University（京都大学）；e-Growth Co., Ltd.（e-Growth 有限公司）；Nara Medical University（奈良县立医科大学）；Rakuwakai Otowa Hospital（洛和会音羽医院）
The reduction of metal artifacts in computed tomography (CT) images, specifically for strong artifacts generated from multiple metal objects, is a challenging issue in medical imaging research. Although there have been some studies on supervised metal artifact reduction through the learning of synthesized artifacts, it is difficult for simulated artifacts to cover the complexity of the real physical phenomena that may be observed in X-ray propagation. In this paper, we introduce metal artifact reduction methods based on an unsupervised volume-to-volume translation learned from clinical CT imag…
Limited-angle tomography is a highly ill-posed linear inverse problem. It arises in many applications, such as digital breast tomosynthesis. Reconstructions from limited-angle data typically suffer from severe stretching of features along the central direction of projections, leading to poor separation between slices perpendicular to the central direction. A new method is introduced, based on machine learning and geometry, producing an estimate for interfaces between regions of different X-ray attenuation. The estimate can be presented on top of the reconstruction, indicating more reliably the…
Friedrich-Alexander Universität Erlangen-Nürnberg（弗里德里希-亚历山大大学埃尔朗根-纽伦堡分校）；Erlangen Graduate School in Advanced Optical Technologies (SAOT)（埃尔兰根先进光学技术研究生院（SAOT））
In computer tomography, due to the presence of metal implants in the patient body, reconstructed images will suffer from metal artifacts. In order to reduce metal artifacts, metals are typically removed in projection images. Therefore, the metal corrupted projection areas need to be inpainted. For deep learning inpainting methods, convolutional neural networks (CNNs) are widely used, for example, the U-Net. However, such CNNs use convolutional filter responses on both valid and corrupted pixel values, resulting in unsatisfactory image quality. In this work, partial convolution is applied for p…
Massachusetts Institute of Technology（麻省理工学院）；Sandia National Laboratory（桑迪亚国家实验室）
Limited-angle X-ray tomography reconstruction is an ill-conditioned inverse problem in general. Especially when the projection angles are limited and the measurements are taken in a photon-limited condition, reconstructions from classical algorithms such as filtered backprojection may lose fidelity and acquire artifacts due to the missing-cone problem. To obtain satisfactory reconstruction results, prior assumptions, such as total variation minimization and nonlocal image similarity, are usually incorporated within the reconstruction algorithm. In this work, we introduce deep neural networks t…
Northwestern University（西北大学）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
Computed tomography is widely used to examine internal structures in a non-destructive manner. To obtain high-quality reconstructions, one typically has to acquire a densely sampled trajectory to avoid angular undersampling. However, many scenarios require a sparse-view measurement leading to streak-artifacts if unaccounted for. Current methods do not make full use of the domain-specific information, and hence fail to provide reliable reconstructions for highly undersampled data. We present a novel framework for sparse-view tomography by decoupling the reconstruction into two steps: First, we …
University of Rochester（罗切斯特大学）；University of Maryland, College Park（马里兰大学帕克分校）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Current deep neural network based approaches to computed tomography (CT) metal artifact reduction (MAR) are supervised methods which rely heavily on synthesized data for training. However, as synthesized data may not perfectly simulate the underlying physical mechanisms of CT imaging, the supervised methods often generalize poorly to clinical applications. To address this problem, we propose, to the best of our knowledge, the first unsupervised learning approach to MAR. Specifically, we introduce a novel artifact disentanglement network that enables different forms of generations and regulariz…
Electron tomography, as an important 3D imaging method, offers a powerful method to probe the 3D structure of materials from the nano- to the atomic-scale. However, as a grant challenge, radiation intolerance of the nanoscale samples and the missing-wedge-induced information loss and artifacts greatly hindered us from obtaining 3D atomic structures with high fidelity. Here, for the first time, by combining generative adversarial models with state-of-the-art network architectures, we demonstrate the resolution of electron tomography can be improved to 0.71 angstrom which is the highest three-di…
University of Helsinki（赫尔辛基大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
We propose a novel convolutional neural network (CNN), called $Ψ$DONet, designed for learning pseudodifferential operators ($Ψ$DOs) in the context of linear inverse problems. Our starting point is the Iterative Soft Thresholding Algorithm (ISTA), a well-known algorithm to solve sparsity-promoting minimization problems. We show that, under rather general assumptions on the forward operator, the unfolded iterations of ISTA can be interpreted as the successive layers of a CNN, which in turn provides fairly general network architectures that, for a specific choice of the parameters involved, allow…
Deep learning classification models for medical image analysis often perform well on data from scanners that were used during training. However, when these models are applied to data from different vendors, their performance tends to drop substantially. Artifacts that only occur within scans from specific scanners are major causes of this poor generalizability. We aimed to improve the reliability of deep learning classification models by proposing Uncertainty-Based Instance eXclusion (UBIX). This technique, based on multiple-instance learning, reduces the effect of corrupted instances on the b…
We present a Machine Learning-based method for tomographic reconstruction of dense layered objects, with range of projection angles limited to $\pm $10$^\circ$. Whereas previous approaches to phase tomography generally require two steps, first to retrieve phase projections from intensity projections and then perform tomographic reconstruction on the retrieved phase projections, in our work a physics-informed pre-processor followed by a Deep Neural Network (DNN) conduct the three-dimensional reconstruction directly from the intensity projections. We demonstrate this single-step method experimen…
Tencent Jarvis Lab（腾讯天衍实验室）；Xi’an Jiaotong University（西安交通大学）；Peng Cheng Laboratory（鹏城实验室）；Macau University of Science and Technology（澳门科技大学）
During X-ray computed tomography (CT) scanning, metallic implants carrying with patients often lead to adverse artifacts in the captured CT images and then impair the clinical treatment. Against this metal artifact reduction (MAR) task, the existing deep-learning-based methods have gained promising reconstruction performance. Nevertheless, there is still some room for further improvement of MAR performance and generalization ability, since some important prior knowledge underlying this specific task has not been fully exploited. Hereby, in this paper, we carefully analyze the characteristics o…
University of Rochester（罗切斯特大学）；University of Maryland, College Park（马里兰大学帕克分校）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）
Current deep neural network based approaches to computed tomography (CT) metal artifact reduction (MAR) are supervised methods that rely on synthesized metal artifacts for training. However, as synthesized data may not accurately simulate the underlying physical mechanisms of CT imaging, the supervised methods often generalize poorly to clinical applications. To address this problem, we propose, to the best of our knowledge, the first unsupervised learning approach to MAR. Specifically, we introduce a novel artifact disentanglement network that disentangles the metal artifacts from CT images i…
National Tsing Hua University（清华大学）；University of Notre Dame（圣母大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Acadamic of Medical Science（广东医学科学院）
Medical images may contain various types of artifacts with different patterns and mixtures, which depend on many factors such as scan setting, machine condition, patients' characteristics, surrounding environment, etc. However, existing deep-learning-based artifact reduction methods are restricted by their training set with specific predetermined artifact types and patterns. As such, they have limited clinical adoption. In this paper, we introduce One-Shot medical image Artifact Reduction (OSAR), which exploits the power of deep learning but without using pre-trained general networks. Specific…
The electrical property (EP) of human tissues is a quantitative biomarker that facilitates early diagnosis of cancerous tissues. Magnetic resonance electrical properties tomography (MREPT) is an imaging modality that reconstructs EPs by the radio-frequency field in an MRI system. MREPT reconstructs EPs by solving analytic models numerically based on Maxwell's equations. Most MREPT methods suffer from artifacts caused by inaccuracy of the hypotheses behind the models, and/or numerical errors. These artifacts can be mitigated by adding coefficients to stabilize the models, however, the selection…
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Spallation Neutron Source Science Center（散裂中子源科学中心）；Institute of High Energy Physics, Chinese Academy of Sciences（中国科学院高能物理研究所）；University of Science and Technology of China（中国科学技术大学）
In transmission X-ray microscopy (TXM) systems, the rotation of a scanned sample might be restricted to a limited angular range to avoid collision to other system parts or high attenuation at certain tilting angles. Image reconstruction from such limited angle data suffers from artifacts due to missing data. In this work, deep learning is applied to limited angle reconstruction in TXMs for the first time. With the challenge to obtain sufficient real data for training, training a deep neural network from synthetic data is investigated. Particularly, the U-Net, the state-of-the-art neural networ…
An algorithm is presented for constructing high-order signed distance fields for two phase materials imaged with computed tomography. The signed distance field is high-order in that it is free of the quantization artifact associated with the distance transform of sampled signals. The narrowband is solved using a closest point algorithm extended for implicit embeddings that are not a signed distance field. The high-order fast sweeping algorithm is used to extend the narrowband to the remainder of the domain. The order of accuracy of the narrowband and extension methods are verified on ideal imp…
Dispersion is a common phenomenon in optics due to the frequency dependence of the refractive index in polychromatic light. This issue, if left untreated in optical coherence tomography (OCT) imaging, leads to signal broadening of the coherence length and deterioration of the axial resolution. We report a new numeric method for the systematic dispersion compensation in a spectral-domain (SD) OCT for imaging the iridocorneal angle of human cadaver eyes. The dispersion compensation for our OCT system is calculated by an automated iterative process that minimizes the wavenumber-dependent variance…
Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time
Argonne National Laboratory（阿贡国家实验室）；Massachusetts Institute of Technology（麻省理工学院）；Singapore-MIT Alliance for Research and Technology (SMART) Centre（新加坡-麻省理工研究与技术联盟（SMART）中心）；University of California, Berkeley（加州大学伯克利分校）
The sparse-views x-ray computed tomography (CT) is essential for medical diagnosis and industrial nondestructive testing. However, in particular, the reconstructed image usually suffers from complex artifacts and noise, when the sampling is insufficient. In order to deal such issue, a full-automatic reconstruction (FAR) net is proposed for sparse-views CT reconstruction via deep learning technique. Different with the usual network in deep learning reconstruction, the proposed neural network is an End-to-End network by which the image is predicted directly from projection data. The main challen…
