A robust single-shot 3D shape reconstruction technique integrating the fringe projection profilometry (FPP) technique with the deep convolutional neural networks (CNNs) is proposed in this letter. The input of the proposed technique is a single FPP image, and the training and validation data sets are prepared by using the conventional multi-frequency FPP technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D images to its corresponding 3D shape. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.
Deep Learning-based models such as Convolutional Neural Networks, have led to significant advancements in several areas of computing applications. Seismogram quality assurance is a relevant Geophysics task, since in the early stages of seismic processing, we are required to identify and fix noisy sail lines. In this work, we introduce a real-world seismogram quality classification dataset based on 6,613 examples, manually labeled by human experts as good, bad or ugly, according to their noise intensity. This dataset is used to train a CNN classifier for seismic shot-gathers quality prediction.…
Bilkent University（比尔肯特大学）；National Magnetic Resonance Research Center, Bilkent University（比尔肯特大学国家磁共振研究中心）
Supervised reconstruction models are characteristically trained on matched pairs of undersampled and fully-sampled data to capture an MRI prior, along with supervision regarding the imaging operator to enforce data consistency. To reduce supervision requirements, the recent deep image prior framework instead conjoins untrained MRI priors with the imaging operator during inference. Yet, canonical convolutional architectures are suboptimal in capturing long-range relationships, and priors based on randomly initialized networks may yield suboptimal performance. To address these limitations, here …
Tada-DIP: Input-adaptive Deep Image Prior for One-shot 3D Image Reconstruction
1 Dept. of Computational Mathematics, Science \& Engineering, Michigan State University, East Lansing, MI, USA 2 Dept. of Electrical Engineering；Computer Science, University of Michigan, Ann Arbor, MI, USA 3 Dept. of Biomedical Engineering, Michigan State University, East Lansing, MI, USA
Queen Mary University of London（伦敦玛丽女王大学）；Barts Health NHS Trust（巴特国民保健信托基金会）
X-ray coronary angiography (XCA) is used to assess coronary artery disease and provides valuable information on lesion morphology and severity. However, XCA images are 2D and therefore limit visualisation of the vessel. 3D reconstruction of coronary vessels is possible using multiple views, however lumen border detection in current software is performed manually resulting in limited reproducibility and slow processing time. In this study we propose 3DAngioNet, a novel deep learning (DL) system that enables rapid 3D vessel mesh reconstruction using 2D XCA images from two views. Our approach lea…
With the effective application of deep learning in computer vision, breakthroughs have been made in the research of super-resolution images reconstruction. However, many researches have pointed out that the insufficiency of the neural network extraction on image features may bring the deteriorating of newly reconstructed image. On the other hand, the generated pictures are sometimes too artificial because of over-smoothing. In order to solve the above problems, we propose a novel self-calibrated convolutional generative adversarial networks. The generator consists of feature extraction and ima…
University of Basel（巴塞尔大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Convolutional neural networks (CNNs) have been tremendously successful in solving imaging inverse problems. To understand their success, an effective strategy is to construct simpler and mathematically more tractable convolutional sparse coding (CSC) models that share essential ingredients with CNNs. Existing CSC methods, however, underperform leading CNNs in challenging inverse problems. We hypothesize that the performance gap may be attributed in part to how they process images at different spatial scales: While many CNNs use multiscale feature representations, existing CSC models mostly rel…
Improved Multi-Shot Diffusion-Weighted MRI with Zero-Shot Self-Supervised Learning Reconstruction
Athinoula A. Martinos Center for Biomedical Imaging（阿瑟诺拉·A·马蒂诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；University of São Paulo（圣保罗大学）；Technische Universität München（慕尼黑工业大学）；Harvard-MIT Health Sciences and Technology（哈佛-麻省理工健康科学与技术部）
University of Southern Denmark（南丹麦大学）；Newtec Engineering A/S（Newtec工程公司）；Mads Clausen Institute（马德斯·克劳森研究所）
A novel method, utilizing convolutional neural networks (CNNs), is proposed to reconstruct hyperspectral cubes from computed tomography imaging spectrometer (CTIS) images. Current reconstruction algorithms are usually subject to long reconstruction times and mediocre precision in cases of a large number of spectral channels. The constructed CNNs deliver higher precision and shorter reconstruction time than a sparse expectation maximization algorithm. In addition, the network can handle two different types of real-world images at the same time -- specifically ColorChecker and carrot spectral im…
Many real-world signal sources are complex-valued, having real and imaginary components. However, the vast majority of existing deep learning platforms and network architectures do not support the use of complex-valued data. MRI data is inherently complex-valued, so existing approaches discard the richer algebraic structure of the complex data. In this work, we investigate end-to-end complex-valued convolutional neural networks - specifically, for image reconstruction in lieu of two-channel real-valued networks. We apply this to magnetic resonance imaging reconstruction for the purpose of acce…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California Nano Systems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（大卫·格芬医学院）
Deep learning-based methods in computational microscopy have been shown to be powerful but in general face some challenges due to limited generalization to new types of samples and requirements for large and diverse training data. Here, we demonstrate a few-shot transfer learning method that helps a holographic image reconstruction deep neural network rapidly generalize to new types of samples using small datasets. We pre-trained a convolutional recurrent neural network on a large dataset with diverse types of samples, which serves as the backbone model. By fixing the recurrent blocks and tran…
Shape reconstruction of deformable organs from two-dimensional X-ray images is a key technology for image-guided intervention. In this paper, we propose an image-to-graph convolutional network (IGCN) for deformable shape reconstruction from a single-viewpoint projection image. The IGCN learns relationship between shape/deformation variability and the deep image features based on a deformation mapping scheme. In experiments targeted to the respiratory motion of abdominal organs, we confirmed the proposed framework with a regularized loss function can reconstruct liver shapes from a single digit…
Zhejiang University（浙江大学）；University of Florida（佛罗里达大学）
Dynamic positron emission tomography (dPET) image reconstruction is extremely challenging due to the limited counts received in individual frame. In this paper, we propose a spatial-temporal convolutional primal dual network (STPDnet) for dynamic PET image reconstruction. Both spatial and temporal correlations are encoded by 3D convolution operators. The physical projection of PET is embedded in the iterative learning process of the network, which provides the physical constraints and enhances interpretability. The experiments of real rat scan data have shown that the proposed method can achie…
Technische Universität München（慕尼黑工业大学）；Harvard University（哈佛大学）；Massachusetts General Hospital（麻省总医院）
Undersampling the k-space data is widely adopted for acceleration of Magnetic Resonance Imaging (MRI). Current deep learning based approaches for supervised learning of MRI image reconstruction employ real-valued operations and representations by treating complex valued k-space/spatial-space as real values. In this paper, we propose complex dense fully convolutional neural network ($\mathbb{C}$DFNet) for learning to de-alias the reconstruction artifacts within undersampled MRI images. We fashioned a densely-connected fully convolutional block tailored for complex-valued inputs by introducing d…
United Imaging Intelligence（联影智能）；Johns Hopkins University（约翰斯·霍普金斯大学）
Fast and accurate MRI image reconstruction from undersampled data is crucial in clinical practice. Deep learning based reconstruction methods have shown promising advances in recent years. However, recovering fine details from undersampled data is still challenging. In this paper, we introduce a novel deep learning based method, Pyramid Convolutional RNN (PC-RNN), to reconstruct images from multiple scales. Based on the formulation of MRI reconstruction as an inverse problem, we design the PC-RNN model with three convolutional RNN (ConvRNN) modules to iteratively learn the features in multiple…
Indian Institute of Technology Madras (IITM)（印度理工学院马德拉斯分校）；Healthcare Technology Innovation Centre (HTIC), IITM（印度理工学院马德拉斯分校医疗技术创新中心）
Several variants of Convolutional Neural Networks (CNN) have been developed for Magnetic Resonance (MR) image reconstruction. Among them, U-Net has shown to be the baseline architecture for MR image reconstruction. However, sub-sampling is performed by its pooling layers, causing information loss which in turn leads to blur and missing fine details in the reconstructed image. We propose a modification to the U-Net architecture to recover fine structures. The proposed network is a wavelet packet transform based encoder-decoder CNN with residual learning called CNN. The proposed WCNN has discret…
Purpose: To introduce a fast and improved direct reconstruction method for multi-shot diffusion weighted (msDW) scans for high-resolution studies. Methods:Multi-shot EPI methods can enable higher spatial resolution for diffusion MRI studies. Traditionally, such acquisitions required specialized reconstructions involving phase compensation to correct for inter-shot motion artifacts. The recently proposed MUSSELS reconstruction belongs to a new class of parallel imaging-based methods that recover artifact-free DWIs from msDW data without needing phase compensation. However, computational demands…
Reconstructing magnetic resonance (MR) images from undersampled data is a challenging problem due to various artifacts introduced by the under-sampling operation. Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture which captures low-level features at the initial layers and high-level features at the deeper layers. Such networks focus much on global features which may not be optimal to reconstruct the fully-sampled image. In this paper, we propose an Over-and-Under Complete Convolutional Recurrent Neural Network (OUCR), which cons…
Harbin Institute of Technology（哈尔滨工业大学）；The Second Affiliated Hospital of Harbin Medical University（哈尔滨医科大学附属第二医院）
Low-dose computed tomography (CT) allows the reduction of radiation risk in clinical applications at the expense of image quality, which deteriorates the diagnosis accuracy of radiologists. In this work, we present a High-Quality Imaging network (HQINet) for the CT image reconstruction from Low-dose computed tomography (CT) acquisitions. HQINet was a convolutional encoder-decoder architecture, where the encoder was used to extract spatial and temporal information from three contiguous slices while the decoder was used to recover the spacial information of the middle slice. We provide experimen…
RSIP Vision（RSIP Vision 公司）；Weizmann Institute of Science（魏茨曼科学研究所）
We present an end-to-end Convolutional Neural Network (CNN) approach for 3D reconstruction of knee bones directly from two bi-planar X-ray images. Clinically, capturing the 3D models of the bones is crucial for surgical planning, implant fitting, and postoperative evaluation. X-ray imaging significantly reduces the exposure of patients to ionizing radiation compared to Computer Tomography (CT) imaging, and is much more common and inexpensive compared to Magnetic Resonance Imaging (MRI) scanners. However, retrieving 3D models from such 2D scans is extremely challenging. In contrast to the commo…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Zhejiang University（浙江大学）
Positron emission tomography (PET) is widely used for clinical diagnosis. As PET suffers from low resolution and high noise, numerous efforts try to incorporate anatomical priors into PET image reconstruction, especially with the development of hybrid PET/CT and PET/MRI systems. In this work, we proposed a novel 3D structural convolutional sparse coding (CSC) concept for penalized-likelihood PET image reconstruction, named 3D PET-CSC. The proposed 3D PET-CSC takes advantage of the convolutional operation and manages to incorporate anatomical priors without the need of registration or supervise…
Northwestern University（西北大学）；Wormpex AI Research（万沛克斯人工智能研究院）
HDR reconstruction is an important task in computer vision with many industrial needs. The traditional approaches merge multiple exposure shots to generate HDRs that correspond to the physical quantity of illuminance of the scene. However, the tedious capturing process makes such multi-shot approaches inconvenient in practice. In contrast, recent single-shot methods predict a visually appealing HDR from a single LDR image through deep learning. But it is not clear whether the previously mentioned physical properties would still hold, without training the network to explicitly model them. In th…
With the increasing availability of optical and synthetic aperture radar (SAR) images thanks to the Sentinel constellation, and the explosion of deep learning, new methods have emerged in recent years to tackle the reconstruction of optical images that are impacted by clouds. In this paper, we focus on the evaluation of convolutional neural networks that use jointly SAR and optical images to retrieve the missing contents in one single polluted optical image. We propose a simple framework that ease the creation of datasets for the training of deep nets targeting optical image reconstruction, an…
United Imaging Intelligence（联影智能）；UIH America, Inc.（联影美国公司）
Real-time cardiac cine MRI does not require ECG gating in the data acquisition and is more useful for patients who can not hold their breaths or have abnormal heart rhythms. However, to achieve fast image acquisition, real-time cine commonly acquires highly undersampled data, which imposes a significant challenge for MRI image reconstruction. We propose a residual convolutional RNN for real-time cardiac cine reconstruction. To the best of our knowledge, this is the first work applying deep learning approach to Cartesian real-time cardiac cine reconstruction. Based on the evaluation from radiol…
Zhejiang University（浙江大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；First Hospital of Shanxi Medical University（山西医科大学第一医院）
Patlak model is widely used in 18F-FDG dynamic positron emission tomography (PET) imaging, where the estimated parametric images reveal important biochemical and physiology information. Because of better noise modeling and more information extracted from raw sinogram, direct Patlak reconstruction gains its popularity over the indirect approach which utilizes reconstructed dynamic PET images alone. As the prerequisite of direct Patlak methods, raw data from dynamic PET are rarely stored in clinics and difficult to obtain. In addition, the direct reconstruction is time-consuming due to the bottl…
Fast spin-echo (FSE) pulse sequences for Magnetic Resonance Imaging (MRI) offer important imaging contrast in clinically feasible scan times. T2-shuffling is widely used to resolve temporal signal dynamics in FSE acquisitions by exploiting temporal correlations via linear latent space and a predefined regularizer. However, predefined regularizers fail to exploit the incoherence especially for 2D acquisitions.Recent self-supervised learning methods achieve high-fidelity reconstructions by learning a regularizer from undersampled data without a standard supervised training data set. In this work…
University of Dundee（邓迪大学）；University Hospital Poitiers（普瓦捷大学医院）
Objective. Dual-energy computed tomography (DECT) has the potential to improve contrast, reduce artifacts and the ability to perform material decomposition in advanced imaging applications. The increased number or measurements results with a higher radiation dose and it is therefore essential to reduce either number of projections per energy or the source X-ray intensity, but this makes tomographic reconstruction more ill-posed. Approach. We developed the multi-channel convolutional analysis operator learning (MCAOL) method to exploit common spatial features within attenuation images at differ…
Athinoula A. Martinos Center for Biomedical Imaging（阿西努拉·A·马蒂诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Harvard-MIT Health Sciences and Technology（哈佛-麻省理工健康科学与技术项目）；MIT（麻省理工学院）；National Nanotechnology Center, National Science and Technology Development Agency（国家科学技术发展局国家纳米技术中心）
Purpose: To introduce a combined machine learning (ML) and physics-based image reconstruction framework that enables navigator-free, highly accelerated multishot echo planar imaging (msEPI), and demonstrate its application in high-resolution structural and diffusion imaging. Methods: Singleshot EPI is an efficient encoding technique, but does not lend itself well to high-resolution imaging due to severe distortion artifacts and blurring. While msEPI can mitigate these artifacts, high-quality msEPI has been elusive because of phase mismatch arising from shot-to-shot variations which preclude th…
Purpose: To develop a single-shot multi-slice T1 mapping method by combing simultaneous multi-slice (SMS) excitations, single-shot inversion-recovery (IR) radial fast low-angle shot (FLASH) and a nonlinear model-based reconstruction method. Methods: SMS excitations are combined with a single-shot IR radial FLASH sequence for data acquisition. A previously developed single-slice calibrationless model-based reconstruction is extended to SMS, formulating the estimation of parameter maps and coil sensitivities from all slices as a single nonlinear inverse problem. Joint-sparsity constraints are fu…
In recent years, diverging-wave (DW) ultrasound imaging has become a very promising methodology for cardiovascular imaging due to its high temporal resolution. However, if they are limited in number, DW transmits provide lower image quality compared with classical focused schemes. A conventional reconstruction approach consists in summing series of ultrasound signals coherently, at the expense of the frame rate. To deal with this limitation, we propose a convolutional neural networks (CNN) architecture for high-quality reconstruction of DW ultrasound images using a small number of transmission…
Ultrafast ultrasound imaging remains an active area of interest in the ultrasound community due to its ultra-high frame rates. Recently, a wide variety of studies based on deep learning have sought to improve ultrafast ultrasound imaging. Most of these approaches have been performed on radio frequency (RF) signals. However, inphase/quadrature (I/Q) digital beamformers are now widely used as low-cost strategies. In this work, we used complex convolutional neural networks for reconstruction of ultrasound images from I/Q signals. We recently described a convolutional neural network architecture c…
Purpose: To develop new encoding and reconstruction techniques for fast multi-contrast quantitative imaging. Methods: The recently proposed Echo Planar Time-resolved Imaging (EPTI) technique can achieve fast distortion- and blurring-free multi-contrast quantitative imaging. In this work, a subspace reconstruction framework is developed to improve the reconstruction accuracy of EPTI at high encoding accelerations. The number of unknowns in the reconstruction is significantly reduced by modeling the temporal signal evolutions using low-rank subspace. As part of the proposed reconstruction approa…
Multidimensional imaging, capturing image data in more than two dimensions, has been an emerging field with diverse applications. Due to the limitation of two-dimensional detectors in obtaining the high-dimensional image data, computational imaging approaches have been developed to pass on some of the burden to a reconstruction algorithm. In various image reconstruction problems in multidimensional imaging, the measurements are in the form of superimposed convolutions. In this paper, we introduce a general framework for the solution of these problems, called here convolutional inverse problems…
Multi-shot echo planar imaging (msEPI) is a promising approach to achieve high in-plane resolution with high sampling efficiency and low T2* blurring. However, due to the geometric distortion, shot-to-shot phase variations and potential subject motion, msEPI continues to be a challenge in MRI. In this work, we introduce acquisition and reconstruction strategies for robust, high-quality msEPI without phase navigators. We propose Blip Up-Down Acquisition (BUDA) using interleaved blip-up and -down phase encoding, and incorporate B0 forward-modeling into Hankel structured low-rank model to enable …
