Deep learning based approaches have been used to improve image quality in cone-beam computed tomography (CBCT), a medical imaging technique often used in applications such as image-guided radiation therapy, implant dentistry or orthopaedics. In particular, while deep learning methods have been applied to reduce various types of CBCT image artifacts arising from motion, metal objects, or low-dose acquisition, a comprehensive review summarizing the successes and shortcomings of these approaches, with a primary focus on the type of artifacts rather than the architecture of neural networks, is lacking in the literature. In this review, the data generation and simulation pipelines, and artifact reduction techniques are specifically investigated for each type of artifact. We provide an overview of deep learning techniques that have successfully been shown to reduce artifacts in 3D, as well as in time-resolved (4D) CBCT through the use of projection- and/or volume-domain optimizations, or by introducing neural networks directly within the CBCT reconstruction algorithms. Research gaps are identified to suggest avenues for future exploration. One of the key findings of this work is an observed trend towards the use of generative models including GANs and score-based or diffusion models, accompanied with the need for more diverse and open training datasets and simulations.
Optical coherence tomography angiography (OCTA) is an important imaging modality in many bioengineering tasks. The image quality of OCTA, however, is often degraded by Bulk Motion Artifacts (BMA), which are due to micromotion of subjects and typically appear as bright stripes surrounded by blurred areas. State-of-the-art methods usually treat BMA removal as a learning-based image inpainting problem, but require numerous training samples with nontrivial annotation. In addition, these methods discard the rich structural and appearance information carried in the BMA stripe region. To address thes…
The Batch Artifact Scanning Protocol: A new method using computed tomography (CT) to rapidly create three-dimensional models of objects from large collections en masse
University of Minnesota（明尼苏达大学）；School of Mathematics, University of Minnesota（明尼苏达大学数学学院）；Department of Anthropology, University of Minnesota（明尼苏达大学人类学系）
Z 2 Sky Technologies Inc.（Z 2 Sky科技公司）；Adobe（奥多比公司）；University of Rochester（罗切斯特大学）；Beijing United Family Hospital（北京和睦家医院）；Peking Union Medical College Hospital（北京协和医院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）
Metal artifact reduction (MAR) in computed tomography (CT) is a notoriously challenging task because the artifacts are structured and non-local in the image domain. However, they are inherently local in the sinogram domain. Thus, one possible approach to MAR is to exploit the latter characteristic by learning to reduce artifacts in the sinogram. However, if we directly treat the metal-affected regions in sinogram as missing and replace them with the surrogate data generated by a neural network, the artifact-reduced CT images tend to be over-smoothed and distorted since fine-grained details wit…
Wellman Center for Photomedicine, Massachusetts General Hospital（马萨诸塞州总医院韦尔曼光医学中心）；Harvard Medical School（哈佛医学院）；Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）；Erasmus Medical Center（伊拉斯谟医学中心）
Intravascular ultrasound and optical coherence tomography are widely available for characterizing coronary stenoses and provide critical vessel parameters to optimize percutaneous intervention. Intravascular polarization-sensitive optical coherence tomography (PS-OCT) simultaneously provides high-resolution cross-sectional images of vascular structures while also revealing preponderant tissue components such as collagen and smooth muscle and thereby enhances plaque characterization. Automated interpretation of these features promises to facilitate the objective clinical investigation of the na…
National Institute for Mathematical Sciences（国家数学科学研究所）；Yonsei University（延世大学）
This paper describes the mathematical structure of the ill-posed nonlinear inverse problem of low-dose dental cone-beam computed tomography (CBCT) and explains the advantages of a deep learning-based approach to the reconstruction of computed tomography images over conventional regularization methods. This paper explains the underlying reasons why dental CBCT is more ill-posed than standard computed tomography. Despite this severe ill-posedness, the demand for dental CBCT systems is rapidly growing because of their cost competitiveness and low radiation dose. We then describe the limitations o…
In this paper, the Primal-Dual UNet for sparse view CT reconstruction is modified to be applicable to cone beam projections and perform reconstructions of entire volumes instead of slices. Experiments show that the PSNR of the proposed method is increased by 10dB compared to the direct FDK reconstruction and almost 3dB compared to the modified original Primal-Dual Network when using only 23 projections. The presented network is not optimized wrt. memory consumption or hyperparameters but merely serves as a proof of concept and is limited to low resolution projections and volumes.
Leibniz University Hanover（汉诺威莱布尼茨大学）；Hannover Medical School（汉诺威医学院）
Cochlear implants (CIs) are implantable medical devices that can restore the hearing sense of people suffering from profound hearing loss. The CI uses a set of electrode contacts placed inside the cochlea to stimulate the auditory nerve with current pulses. The exact location of these electrodes may be an important parameter to improve and predict the performance with these devices. Currently the methods used in clinics to characterize the geometry of the cochlea as well as to estimate the electrode positions are manual, error-prone and time consuming. We propose a Markov random field (MRF) mo…
Computed Tomography (CT) is a non-invasive imaging modality with applications ranging from healthcare to security. It reconstructs cross-sectional images of an object using a collection of projection data collected at different angles. Conventional methods, such as FBP, require that the projection data be uniformly acquired over the complete angular range. In some applications, it is not possible to acquire such data. Security is one such domain where non-rotational scanning configurations are being developed which violate the complete data assumption. Conventional methods produce images from …
MInDI-3D: Iterative Deep Learning in 3D for Sparse-view Cone Beam Computed Tomography
Centre for Artificial Intelligence (CAI), Zurich University of Applied Sciences (ZHAW)（人工智能中心（CAI）、应用科学大学（ZHAW））；Institute of Applied Mathematics and Physics (IAMP), Zurich University of Applied Sciences (ZHAW)（应用数学与物理研究所（IAMP）、应用科学大学（ZHAW））；Varian Medical Systems Imaging Lab（Varian医疗系统影像实验室）；Biomedical Image Analysis and Machine Learning, University of Zurich（生物医学影像分析与机器学习、苏黎世大学）
George Mason University（乔治梅森大学）；The MITRE Corporation（MITRE公司）
Photoacoustic imaging is an emerging imaging modality that is based upon the photoacoustic effect. In photoacoustic tomography (PAT), the induced acoustic pressure waves are measured by an array of detectors and used to reconstruct an image of the initial pressure distribution. A common challenge faced in PAT is that the measured acoustic waves can only be sparsely sampled. Reconstructing sparsely sampled data using standard methods results in severe artifacts that obscure information within the image. We propose a modified convolutional neural network (CNN) architecture termed Fully Dense UNe…
National Tsing Hua University（清华大学）；University of Notre Dame（圣母大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Acadamic of Medical Science（广东医学科学院）
Medical images may contain various types of artifacts with different patterns and mixtures, which depend on many factors such as scan setting, machine condition, patients' characteristics, surrounding environment, etc. However, existing deep-learning-based artifact reduction methods are restricted by their training set with specific predetermined artifact types and patterns. As such, they have limited clinical adoption. In this paper, we introduce One-Shot medical image Artifact Reduction (OSAR), which exploits the power of deep learning but without using pre-trained general networks. Specific…
An algorithm is presented for constructing high-order signed distance fields for two phase materials imaged with computed tomography. The signed distance field is high-order in that it is free of the quantization artifact associated with the distance transform of sampled signals. The narrowband is solved using a closest point algorithm extended for implicit embeddings that are not a signed distance field. The high-order fast sweeping algorithm is used to extend the narrowband to the remainder of the domain. The order of accuracy of the narrowband and extension methods are verified on ideal imp…
Kyoto University（京都大学）；e-Growth Co., Ltd.（e-Growth 有限公司）；Nara Medical University（奈良县立医科大学）；Rakuwakai Otowa Hospital（洛和会音羽医院）
The reduction of metal artifacts in computed tomography (CT) images, specifically for strong artifacts generated from multiple metal objects, is a challenging issue in medical imaging research. Although there have been some studies on supervised metal artifact reduction through the learning of synthesized artifacts, it is difficult for simulated artifacts to cover the complexity of the real physical phenomena that may be observed in X-ray propagation. In this paper, we introduce metal artifact reduction methods based on an unsupervised volume-to-volume translation learned from clinical CT imag…
Computed Tomography (CT) is an imaging technique where information about an object are collected at different angles (called projections or scans). Then the cross-sectional image showing the internal structure of the slice is produced by solving an inverse problem. Limited by certain factors such as radiation dosage, projection angles, the produced images can be noisy or contain artifacts. Inspired by the success of transformer for natural language processing, the core idea of this preliminary study is to consider a projection of tomography as a word token, and the whole scan of the cross-sect…
Image Prediction for Limited-angle Tomography via Deep Learning with Convolutional Neural Network
National Digital Switching System Engineering and Technological Research Center（国家数字交换系统工程技术研究中心）；Tsinghua University（清华大学）；Key Laboratory of Particle and Radiation Imaging (Tsinghua University), Ministry of Education（粒子与辐射成像教育部重点实验室（清华大学））
Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Conebeam CT using a circular trajectory is quite often used for various applications due to its relative simple geometry. For conebeam geometry, Feldkamp, Davis and Kress algorithm is regarded as the standard reconstruction method, but this algorithm suffers from so-called conebeam artifacts as the cone angle increases. Various model-based iterative reconstruction methods have been developed to reduce the cone-beam artifacts, but these algorithms usually require multiple applications of computational expensive forward and backprojections. In this paper, we develop a novel deep learning approac…
We present an effective post-processing method to reduce the artifacts from sparsely reconstructed cone-beam CT (CBCT) images. The proposed method is based on the state-of-the-art, image-to-image generative models with a perceptual loss as regulation. Unlike the traditional CT artifact-reduction approaches, our method is trained in an adversarial fashion that yields more perceptually realistic outputs while preserving the anatomical structures. To address the streak artifacts that are inherently local and appear across various scales, we further propose a novel discriminator architecture based…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Optical diffraction tomography (ODT) produces three dimensional distribution of refractive index (RI) by measuring scattering fields at various angles. Although the distribution of RI index is highly informative, due to the missing cone problem stemming from the limited-angle acquisition of holograms, reconstructions have very poor resolution along axial direction compared to the horizontal imaging plane. To solve this issue, here we present a novel unsupervised deep learning framework, which learns the probability distribution of missing projection views through optimal transport driven cycle…
Texture-preserving implicit neural representation for Cone beam CT truncated reconstruction
National Key Research and Development Program of China（中华人民共和国国家重点研发计划）；National Natural Science Foundation of China（中华人民共和国国家自然科学基金）；Fundamental Research Funds for the Central Universities（中央高校基本科研业务费）
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Siemens Healthcare GmbH（西门子医疗有限公司）；Shanghai Jiao Tong University（上海交通大学）；Erlangen Graduate School in Advanced Optical Technologies (SAOT)（埃尔朗根先进光学技术研究生院（SAOT））
Robustness of deep learning methods for limited angle tomography is challenged by two major factors: a) due to insufficient training data the network may not generalize well to unseen data; b) deep learning methods are sensitive to noise. Thus, generating reconstructed images directly from a neural network appears inadequate. We propose to constrain the reconstructed images to be consistent with the measured projection data, while the unmeasured information is complemented by learning based methods. For this purpose, a data consistent artifact reduction (DCAR) method is introduced: First, a pr…
Deep Residual Network for Off-Resonance Artifact Correction with Application to Pediatric Body Magnetic Resonance Angiography with 3D Cones
Stanford University（斯坦福大学）
目的：通过深度学习进行共振伪影校正，以便于使用扫描时间高效的3D锥体轨迹进行快速儿科身体成像。研究方法：通过对30例儿科磁共振血管造影检查的前瞻性研究，对用于校正非共振伪影（Off-ResNet）的残差卷积神经网络进行了训练。每次检查在3 T下采集短读出扫描（1.18 ms +- 0.38）和长读出扫描（3.35 ms +- 0.74）。具有较长扫描时间但可忽略的非共振模糊的短读出扫描被用作参考图像，并使用额外的非共振来增强监督训练样本。长读出扫描，具有更大的非共振伪影，但扫描时间更短，通过自动聚焦和Off-ResNet进行校正，并通过归一化均方根误差（NRMSE），结构相似性指数（SSIM）和峰值信噪比（PSNR）与短读出扫描进行比较。两名放射科医生对8个解剖特征进行评分，使用方差分析和事后Tukey检验，对扫描进行比较。通过组内相关性确定阅片者一致性。结果：长读出扫描比短读出扫描平均短59.3%。在+-1kHz非共振范围内，Off-ResNet图像的NRMSE、SSIM和PSNR均优于未校正图像（P<0.01）。在-677Hz ~+1kHz范围内，该方法的归一化均方根误差（NRMSE）优于自聚焦，在+-1kHz范围内，该方法的信噪比（SSIM）和峰值信噪比（PSNR）优于自聚焦（P<0.01）。放射学评分显示，Off-ResNet校正的长读出扫描不劣于短读出扫描（P<0.01）…
Seoul National University（首尔国立大学）；Soong-sil University（崇实大学）
Individual tooth segmentation from cone beam computed tomography (CBCT) images is an essential prerequisite for an anatomical understanding of orthodontic structures in several applications, such as tooth reformation planning and implant guide simulations. However, the presence of severe metal artifacts in CBCT images hinders the accurate segmentation of each individual tooth. In this study, we propose a neural network for pixel-wise labeling to exploit an instance segmentation framework that is robust to metal artifacts. Our method comprises of three steps: 1) image cropping and realignment b…
Queen Mary University of London（伦敦玛丽女王大学）；Communication University of Zhejiang（浙江传媒学院）；University of Leicester（莱斯特大学）；University of Electronic Science and Technology of China（电子科技大学）；Hangzhou Dianzi University（杭州电子科技大学）
3D tooth segmentation is a prerequisite for computer-aided dental diagnosis and treatment. However, segmenting all tooth regions manually is subjective and time-consuming. Recently, deep learning-based segmentation methods produce convincing results and reduce manual annotation efforts, but it requires a large quantity of ground truth for training. To our knowledge, there are few tooth data available for the 3D segmentation study. In this paper, we establish a fully annotated cone beam computed tomography dataset CTooth with tooth gold standard. This dataset contains 22 volumes (7363 slices) w…
Artifact Reduction in Undersampled 3D Cone-Beam CTs using a Hybrid 2D-3D CNN Framework
Chair of Biomedical Physics, Department of Physics, School of Natural Sciences（生物医学物理教授会，物理系，自然科学学院）；Munich Institute of Biomedical Engineering（慕尼黑生物医学工程研究所）；Institute for Diagnostic and Interventional Radiology, School of Medicine and Health, TUM Klinikum（诊断与介入放射学研究所，医学与健康学院，TUM医院）
George Mason University（乔治梅森大学）；The MITRE Corporation（MITRE公司）
Photoacoustic tomography (PAT) is a nonionizing imaging modality capable of acquiring high contrast and resolution images of optical absorption at depths greater than traditional optical imaging techniques. Practical considerations with instrumentation and geometry limit the number of available acoustic sensors and their view of the imaging target, which result in significant image reconstruction artifacts degrading image quality. Iterative reconstruction methods can be used to reduce artifacts but are computationally expensive. In this work, we propose a novel deep learning approach termed pi…
National Taiwan University（台湾大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Chang Gung University（长庚大学）
Automated vascular segmentation on optical coherence tomography angiography (OCTA) is important for the quantitative analyses of retinal microvasculature in neuroretinal and systemic diseases. Despite recent improvements, artifacts continue to pose challenges in segmentation. Our study focused on removing the speckle noise artifact from OCTA images when performing segmentation. Speckle noise is common in OCTA and is particularly prominent over large non-perfusion areas. It may interfere with the proper assessment of retinal vasculature. In this study, we proposed a novel Supervision Vessel Seg…
Cone-beam computed tomography (CBCT) is a valuable imaging method in dental diagnostics that provides information not available in traditional 2D imaging. However, interpretation of CBCT images is a time-consuming process that requires a physician to work with complicated software. In this work we propose an automated pipeline composed of several deep convolutional neural networks and algorithmic heuristics. Our task is two-fold: a) find locations of each present tooth inside a 3D image volume, and b) detect several common tooth conditions in each tooth. The proposed system achieves 96.3\% acc…
Queen Mary University of London（伦敦玛丽女王大学）；Communication University of Zhejiang（浙江传媒学院）；University of Electronic Science and Technology of China（电子科技大学）；West China Hospital of Stomatology, Sichuan University（四川大学华西口腔医院）；University of Leicester（莱斯特大学）
Accurate tooth volume segmentation is a prerequisite for computer-aided dental analysis. Deep learning-based tooth segmentation methods have achieved satisfying performances but require a large quantity of tooth data with ground truth. The dental data publicly available is limited meaning the existing methods can not be reproduced, evaluated and applied in clinical practice. In this paper, we establish a 3D dental CBCT dataset CTooth+, with 22 fully annotated volumes and 146 unlabeled volumes. We further evaluate several state-of-the-art tooth volume segmentation strategies based on fully-supe…
During spinal fusion surgery, screws are placed close to critical nerves suggesting the need for highly accurate screw placement. Verifying screw placement on high-quality tomographic imaging is essential. C-arm Cone-beam CT (CBCT) provides intraoperative 3D tomographic imaging which would allow for immediate verification and, if needed, revision. However, the reconstruction quality attainable with commercial CBCT devices is insufficient, predominantly due to severe metal artifacts in the presence of pedicle screws. These artifacts arise from a mismatch between the true physics of image format…
George Mason University（乔治梅森大学）；The MITRE Corporation（MITRE公司）
In photoacoustic tomography (PAT), the acoustic pressure waves produced by optical excitation are measured by an array of detectors and used to reconstruct an image. Sparse spatial sampling and limited-view detection are two common challenges faced in PAT. Reconstructing from incomplete data using standard methods results in severe streaking artifacts and blurring. We propose a modified convolutional neural network (CNN) architecture termed Dense Dilation UNet (DD-UNet) for correcting artifacts in 3D PAT. The DD-Net leverages the benefits of dense connectivity and dilated convolutions to impro…
Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Optical diffraction tomography (ODT) is an emerging 3D imaging technique that is used for the 3D reconstruction of the refractive index (RI) for semi-transparent samples. Various inverse models have been proposed to reconstruct the 3D RI based on the holographic detection of different samples such as the Born and the Rytov approximations. However, such approximations usually suffer from the so-called missing-cone problem that results in an elongation of the final reconstruction along the optical axis. Different iterative schemes have been proposed to solve the missing cone problem relying on p…
Cornell University（康奈尔大学）；University of Michigan（密歇根大学）
Electron tomography (ET) has become a standard technique for 3D characterization of materials at the nano-scale. Traditional reconstruction algorithms such as weighted back projection suffer from disruptive artifacts with insufficient projections. Popularized by compressed sensing, sparsity-exploiting algorithms have been applied to experimental ET data and show promise for improving reconstruction quality or reducing the total beam dose applied to a specimen. Nevertheless, theoretical bounds for these methods have been less explored in the context of ET applications. Here, we perform numerica…
Anatomy-preserving unpaired cone-beam CT refinement for image-guided radiotherapy using pseudo-label guided diffusion
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
