The Magnetic Resonance Imaging (MRI) processing chain starts with a critical acquisition stage that provides raw data for reconstruction of images for medical diagnosis. This flow usually includes a near-lossless data compression stage that enables digital storage and/or transmission in binary formats. In this work we propose a framework for joint optimization of the MRI reconstruction and lossy compression, producing compressed representations of medical images that achieve improved trade-offs between quality and bit-rate. Moreover, we demonstrate that lossy compression can even improve the reconstruction quality compared to settings based on lossless compression. Our method has a modular optimization structure, implemented using the alternating direction method of multipliers (ADMM) technique and the state-of-the-art image compression technique (BPG) as a black-box module iteratively applied. This establishes a medical data compression approach compatible with a lossy compression standard of choice. A main novelty of the proposed algorithm is in the total-variation regularization added to the modular compression process, leading to decompressed images of higher quality without any additional processing at/after the decompression stage. Our experiments show that our regularization-based approach for joint MRI reconstruction and compression often achieves significant PSNR gains between 4 to 9 dB at high bit-rates compared to non-regularized solutions of the joint task. Compared to regularization-based solutions, our optimization method provides PSNR gains between 0.5 to 1 dB at high bit-rates, which is the range of interest for medical image compression.
Path and Bone-Contour Regularized Unpaired MRI-to-CT Translation
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China（计算机科学与技术学院，广东技术大学，广州，中国）；Neurological Institute, Cleveland Clinic, OH, USA（神经医学研究院，克利夫兰医学中心，俄亥俄州，美国）；Department of Neurosurgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China（神经外科，中山大学附属第一医院，广州，中国）；Brigham and Women’s Hospital, Harvard Medical School, MA, USA（布里奇沃特医院，哈佛医学院，马萨诸塞州，美国）；Department of Radiology, Medical Imaging Center, University Medical Center Groningen, University of Groningen, 9712 CP Groningen, The Netherlands（放射科，医学影像中心，格罗宁根大学医学中心，格罗宁根大学，荷兰）
Computational Radiology Laboratory, Boston Children’s Hospital（波士顿儿童医院计算放射实验室）；Harvard Medical School（哈佛医学院）
Fast data acquisition in Magnetic Resonance Imaging (MRI) is vastly in demand and scan time directly depends on the number of acquired k-space samples. Recently, the deep learning-based MRI reconstruction techniques were suggested to accelerate MR image acquisition. The most common issues in any deep learning-based MRI reconstruction approaches are generalizability and transferability. For different MRI scanner configurations using these approaches, the network must be trained from scratch every time with new training dataset, acquired under new configurations, to be able to provide good recon…
Cornell University（康奈尔大学）；Massachusetts Institute of Technology（麻省理工学院）；Massachusetts General Hospital（麻省总医院）
Compressed Sensing MRI (CS-MRI) has shown promise in reconstructing under-sampled MR images, offering the potential to reduce scan times. Classical techniques minimize a regularized least-squares cost function using an expensive iterative optimization procedure. Recently, deep learning models have been developed that model the iterative nature of classical techniques by unrolling iterations in a neural network. While exhibiting superior performance, these methods require large quantities of ground-truth images and have shown to be non-robust to unseen data. In this paper, we explore a novel st…
Cornell University（康奈尔大学）；Massachusetts Institute of Technology（麻省理工学院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Reconstructing under-sampled k-space measurements in Compressed Sensing MRI (CS-MRI) is classically solved with regularized least-squares. Recently, deep learning has been used to amortize this optimization by training reconstruction networks on a dataset of under-sampled measurements. Here, a crucial design choice is the regularization function(s) and corresponding weight(s). In this paper, we explore a novel strategy of using a hypernetwork to generate the parameters of a separate reconstruction network as a function of the regularization weight(s), resulting in a regularization-agnostic rec…
Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑生物医学成像机器学习研究所）；Helmholtz AI, Helmholtz Munich（亥姆霍兹慕尼黑人工智能中心）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；School of Biomedical Engineering and Imaging Sciences, King's College London（伦敦国王学院生物医学工程与影像科学学院）
The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；INRIA Sophia Antipolis（法国国立信息学及自动化研究院索菲亚·安蒂波利斯分院）
Compressed Sensing (CS) takes advantage of signal sparsity or compressibility and allows superb signal reconstruction from relatively few measurements. Based on CS theory, a suitable dictionary for sparse representation of the signal is required. In diffusion MRI (dMRI), CS methods were proposed to reconstruct diffusion-weighted signal and the Ensemble Average Propagator (EAP), and there are two kinds of Dictionary Learning (DL) methods: 1) Discrete Representation DL (DR-DL), and 2) Continuous Representation DL (CR-DL). DR-DL is susceptible to numerical inaccuracy owing to interpolation and re…
Interpretable Representation Learning of Cardiac MRI via Attribute Regularization
Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑生物医学成像机器学习研究所）；Helmholtz AI, Helmholtz Munich（亥姆霍兹慕尼黑亥姆霍兹人工智能研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）
ALMA: a mathematics-driven approach for determining tuning parameters in generalized LASSO problems, with applications to MRI
University of Bologna, Department of Mathematics, Bologna, Italy（博洛尼亚大学数学系）；Lausanne University Hospital and University of Lausanne（洛桑大学医院和洛桑大学）；Department of Diagnostic and Interventional Radiology, Lausanne, Switzerland（诊断与介入放射学部，洛桑，瑞士）
University College London（伦敦大学学院）；Technion -- Israel Institute of Technology（以色列理工学院）；Heriot-Watt University（赫瑞-瓦特大学）
Medical imaging tasks often involve multiple contrasts, such as T1- and T2-weighted magnetic resonance imaging (MRI) data. These contrasts capture information associated with the same underlying anatomy and thus exhibit similarities. In this paper, we propose a Coupled Dictionary Learning based multi-contrast MRI reconstruction (CDLMRI) approach to leverage an available guidance contrast to restore the target contrast. Our approach consists of three stages: coupled dictionary learning, coupled sparse denoising, and $k$-space consistency enforcing. The first stage learns a group of dictionaries…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Maryland School of Dentistry（马里兰大学牙医学院）
Cycle reconstruction regularized adversarial training -- e.g., CycleGAN, DiscoGAN, and DualGAN -- has been widely used for image style transfer with unpaired training data. Several recent works, however, have shown that local distortions are frequent, and structural consistency cannot be guaranteed. Targeting this issue, prior works usually relied on additional segmentation or consistent feature extraction steps that are task-specific. To counter this, this work aims to learn a general add-on structural feature extractor, by explicitly enforcing the structural alignment between an input and it…
Magnetic field inhomogeneity estimation is important in some types of magnetic resonance imaging (MRI), including field-corrected reconstruction for fast MRI with long readout times, and chemical shift based water-fat imaging. Regularized field map estimation methods that account for phase wrapping and noise involve nonconvex cost functions that require iterative algorithms. Most existing minimization techniques were computationally or memory intensive for 3D datasets, and are designed for single-coil MRI. This paper considers 3D MRI with optional consideration of coil sensitivity, and address…
University of Minnesota（明尼苏达大学）；Center for Magnetic Resonance Research, University of Minnesota（明尼苏达大学磁共振研究中心）
Deep learning (DL) has emerged as a tool for improving accelerated MRI reconstruction. A common strategy among DL methods is the physics-based approach, where a regularized iterative algorithm alternating between data consistency and a regularizer is unrolled for a finite number of iterations. This unrolled network is then trained end-to-end in a supervised manner, using fully-sampled data as ground truth for the network output. However, in a number of scenarios, it is difficult to obtain fully-sampled datasets, due to physiological constraints such as organ motion or physical constraints such…
Flow-Guided Implicit Neural Representation for Motion-Aware Dynamic MRI Reconstruction
School of Automation, Guangdong University of Technology（广东工业大学自动化学院）；Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所医学人工智能研究中心）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）
University of Toulouse（图卢兹大学）；CEA-NeuroSpin（法国原子能和替代能源委员会-NeuroSpin）；University Paris-Est（巴黎东大学）
Parallel MRI is a fast imaging technique that enables the acquisition of highly resolved images in space or/and in time. The performance of parallel imaging strongly depends on the reconstruction algorithm, which can proceed either in the original k-space (GRAPPA, SMASH) or in the image domain (SENSE-like methods). To improve the performance of the widely used SENSE algorithm, 2D- or slice-specific regularization in the wavelet domain has been deeply investigated. In this paper, we extend this approach using 3D-wavelet representations in order to handle all slices together and address reconstr…
Sparse coding has been popularly used as an effective data representation method in various applications, such as computer vision, medical imaging and bioinformatics, etc. However, the conventional sparse coding algorithms and its manifold regularized variants (graph sparse coding and Laplacian sparse coding), learn the codebook and codes in a unsupervised manner and neglect the class information available in the training set. To address this problem, in this paper we propose a novel discriminative sparse coding method based on multi-manifold, by learning discriminative class-conditional codeb…
Inverse problems for accelerated MRI typically incorporate domain-specific knowledge about the forward encoding operator in a regularized reconstruction framework. Recently physics-driven deep learning (DL) methods have been proposed to use neural networks for data-driven regularization. These methods unroll iterative optimization algorithms to solve the inverse problem objective function, by alternating between domain-specific data consistency and data-driven regularization via neural networks. The whole unrolled network is then trained end-to-end to learn the parameters of the network. Due t…
Zhejiang University（浙江大学）；Shandong Medical Imaging Research Institute（山东医学影像研究所）；Shandong University（山东大学）；Lanzhou University（兰州大学）；Siemens Healthineers Ltd.（西门子医疗有限公司）；Beijing Institute of Technology（北京理工大学）
Diffusion magnetic resonance imaging (dMRI) is an important tool in characterizing tissue microstructure based on biophysical models, which are complex and highly non-linear. Resolving microstructures with optimization techniques is prone to estimation errors and requires dense sampling in the q-space. Deep learning based approaches have been proposed to overcome these limitations. Motivated by the superior performance of the Transformer, in this work, we present a learning-based framework based on Transformer, namely, a Microstructure Estimation Transformer with Sparse Coding (METSC) for dMRI…
The University of Sheffield（谢菲尔德大学）；University of East Anglia（东英吉利大学）
Magnetic Resonance Imaging (MRI) offers high-resolution \emph{in vivo} imaging and rich functional and anatomical multimodality tissue contrast. In practice, however, there are challenges associated with considerations of scanning costs, patient comfort, and scanning time that constrain how much data can be acquired in clinical or research studies. In this paper, we explore the possibility of generating high-resolution and multimodal images from low-resolution single-modality imagery. We propose the weakly-supervised joint convolutional sparse coding to simultaneously solve the problems of sup…
BioFace3D: A fully automatic pipeline for facial biomarkers extraction of 3D face reconstructions segmented from MRI
La Salle - Universitat Ramon Llull（拉蒙·柳利大学拉萨尔学院）；Universitat de Barcelona（巴塞罗那大学）；FIDMAG, Sisters Hospitallers Research Foundation（仁爱修女会研究基金会FIDMAG研究所）；CIBERSAM（精神健康生物医学研究网络）；Hospital de Sant Pau i la Santa Creu（圣保禄圣十字医院）
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Mannheim University of Applied Science（曼海姆应用科学大学）
Cardiac magnetic resonance imaging improves on diagnosis of cardiovascular diseases by providing images at high spatiotemporal resolution. Manual evaluation of these time-series, however, is expensive and prone to biased and non-reproducible outcomes. In this paper, we present a method that addresses named limitations by integrating segmentation and disease classification into a fully automatic processing pipeline. We use an ensemble of UNet inspired architectures for segmentation of cardiac structures such as the left and right ventricular cavity (LVC, RVC) and the left ventricular myocardium…
Univ Lyon（里昂大学）；INSA‐Lyon（里昂国立应用科学学院）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安让·莫奈大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；CREATIS UMR 5220（CREATIS联合研究单位5220）；U1206（U1206研究单位）
Computer aided diagnosis (CAD) systems are designed to assist clinicians in various tasks, including highlighting abnormal regions in a medical image. A common approach consists in training a voxel-level binary classifier on a set of feature vectors extracted from normal and pathological areas in patients' scans. However, many pathologies (such as epilepsy) are characterized by lesions that may be located anywhere in the brain, have various shapes, sizes and texture. An adequate representation of such a heterogeneity requires a significant amount of annotated data which is a major issue in the…
A Total Variation Regularized Framework for Epilepsy-Related MRI Image Segmentation
Department of Computer Engineering, Modeling, Electronics and Systems (DIMES), University of Calabria, Rende 87036, Italy（计算机工程、建模、电子与系统系（DIMES），卡拉布里亚大学，Rende 87036，意大利）
Institute of Mechatronic Systems, Leibniz Universität Hannover（汉诺威莱布尼茨大学机电系统研究所）
We present deformable unsupervised medical image registration using a randomly-initialized deep convolutional neural network (CNN) as regularization prior. Conventional registration methods predict a transformation by minimizing dissimilarities between an image pair. The minimization is usually regularized with manually engineered priors, which limits the potential of the registration. By learning transformation priors from a large dataset, CNNs have achieved great success in deformable registration. However, learned methods are restricted to domain-specific data and the required amounts of me…
University of Cambridge（剑桥大学）；Shanghai General Hospital（上海市第一人民医院）；Shanghai Jiao Tong University（上海交通大学）；Southern University of Science and Technology（南方科技大学）；University of Bath（巴斯大学）
We present an Expectation-Maximization (EM) Regularized Deep Learning (EMReDL) model for weakly supervised tumor segmentation. The proposed framework is tailored to glioblastoma, a type of malignant tumor characterized by its diffuse infiltration into the surrounding brain tissue, which poses significant challenge to treatment target and tumor burden estimation using conventional structural MRI. Although physiological MRI provides more specific information regarding tumor infiltration, the relatively low resolution hinders a precise full annotation. This has motivated us to develop a weakly su…
Kavli Institute for Systems Neuroscience, NTNU（挪威科技大学卡弗里系统神经科学研究所）
Early diagnosis and accurate segmentation of brain tumors are imperative for successful treatment. Unfortunately, manual segmentation is time consuming, costly and despite extensive human expertise often inaccurate. Here, we present an MRI-based tumor segmentation framework using an autoencoder-regularized 3D-convolutional neural network. We trained the model on manually segmented structural T1, T1ce, T2, and Flair MRI images of 335 patients with tumors of variable severity, size and location. We then tested the model using independent data of 125 patients and successfully segmented brain tumo…
We evaluate a new approach for achieving diffusion MRI data with high spatial resolution, large volume coverage, and fast acquisition speed. A recent method called gSlider-SMS enables whole-brain sub-millimeter diffusion MRI with high signal-to-noise ratio (SNR) efficiency. However, despite the efficient acquisition, the resulting images can still suffer from low SNR due to the small size of the imaging voxels. This work proposes to mitigate the SNR problem by combining gSlider-SMS with a regularized SNR-enhancing reconstruction approach. Illustrative results show that, from gSlider-SMS data a…
In this work, we implement a fully convolutional segmenter featuring both a learned group structure and a regularized weight-pruner to reduce the high computational cost in volumetric image segmentation. We validated our framework on the ACDC dataset featuring one healthy and four pathology groups imaged throughout the cardiac cycle. Our technique achieved Dice scores of 96.8% (LV blood-pool), 93.3% (RV blood-pool) and 90.0% (LV Myocardium) with five-fold cross-validation and yielded similar clinical parameters as those estimated from the ground truth segmentation data. Based on these results,…
Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）
Multimodal medical image fusion helps in combining contrasting features from two or more input imaging modalities to represent fused information in a single image. One of the pivotal clinical applications of medical image fusion is the merging of anatomical and functional modalities for fast diagnosis of malignant tissues. In this paper, we present a novel end-to-end unsupervised learning-based Convolutional Neural Network (CNN) for fusing the high and low frequency components of MRI-PET grayscale image pairs, publicly available at ADNI, by exploiting Structural Similarity Index (SSIM) as the …
Arizona State University（亚利桑那州立大学）；Banner Alzheimer’s Institute（班纳阿尔茨海默病研究所）；Southeast University（东南大学）；Children’s Hospital Los Angeles（洛杉矶儿童医院）；Mayo Clinic Arizona（梅奥诊所亚利桑那分院）；University of Southern California（南加州大学）
Biomarker-assisted diagnosis and intervention in Alzheimer's disease (AD) may be the key to prevention breakthroughs. One of the hallmarks of AD is the accumulation of tau plaques in the human brain. However, current methods to detect tau pathology are either invasive (lumbar puncture) or quite costly and not widely available (Tau PET). In our previous work, structural MRI-based hippocampal multivariate morphometry statistics (MMS) showed superior performance as an effective neurodegenerative biomarker for preclinical AD and Patch Analysis-based Surface Correntropy-induced Sparse coding and ma…
This paper reports the development of a Python Non-Uniform Fast Fourier Transform (PyNUFFT) package, which accelerates non-Cartesian image reconstruction on heterogeneous platforms. Scientific computing with Python encompasses a mature and integrated environment. The NUFFT algorithm has been extensively used for non-Cartesian image reconstruction but previously there was no native Python NUFFT library. The current PyNUFFT software enables multi-dimensional NUFFT on heterogeneous platforms. The PyNUFFT also provides several solvers, including the conjugate gradient method, $\ell$1 total-variati…
Reversible visible watermarking (RVW) is an active copyright protection mechanism. It not only transparently superimposes copyright patterns on specific positions of digital images or video frames to declare the copyright ownership information, but also completely erases the visible watermark image and thus enables restoring the original host image without any distortion. However, existing RVW algorithms mostly construct the reversible mapping mechanism for a specific visible watermarking scheme, which is not versatile. Hence, we propose a generic RVW framework to accommodate various visible w…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Chung-Ang University Hospital（中央大学医院）；Chung-Ang University College of Medicine（中央大学医学院）
Patient scans from MRI often suffer from noise, which hampers the diagnostic capability of such images. As a method to mitigate such artifact, denoising is largely studied both within the medical imaging community and beyond the community as a general subject. However, recent deep neural network-based approaches mostly rely on the minimum mean squared error (MMSE) estimates, which tend to produce a blurred output. Moreover, such models suffer when deployed in real-world sitautions: out-of-distribution data, and complex noise distributions that deviate from the usual parametric noise models. In…
We propose Deep Lossless Image Coding (DLIC), a full resolution learned lossless image compression algorithm. Our algorithm is based on a neural network combined with an entropy encoder. The neural network performs a density estimation on each pixel of the source image. The density estimation is then used to code the target pixel, beating FLIF in terms of compression rate. Similar approaches have been attempted. However, long run times make them unfeasible for real world applications. We introduce a parallelized GPU based implementation, allowing for encoding and decoding of grayscale, 8-bit i…
Center for Magnetic Resonance Research (CMRR)（磁共振研究中心）；University of Minnesota Twin Cities（明尼苏达大学双城分校）
k-space undersampling is a standard technique to accelerate MR image acquisitions. Reconstruction techniques including GeneRalized Autocalibrating Partial Parallel Acquisition(GRAPPA) and its variants are utilized extensively in clinical and research settings. A reconstruction model combining GRAPPA with a conditional generative adversarial network (GAN) was developed and tested on multi-coil human brain images from the fastMRI dataset. For various acceleration rates, GAN and GRAPPA reconstructions were compared in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Fo…
Pansharpening is a fundamental issue in remote sensing field. This paper proposes a side information partially guided convolutional sparse coding (SCSC) model for pansharpening. The key idea is to split the low resolution multispectral image into a panchromatic image related feature map and a panchromatic image irrelated feature map, where the former one is regularized by the side information from panchromatic images. With the principle of algorithm unrolling techniques, the proposed model is generalized as a deep neural network, called as SCSC pansharpening neural network (SCSC-PNN). Compared…
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；University Hospital Magdeburg（马格德堡大学医院）；German Centre for Neurodegenerative Diseases（德国神经退行性疾病中心）；Center for Behavioral Brain Sciences（行为脑科学中心）；Leibniz Institute for Neurobiology（莱布尼茨神经生物学研究所）；MedDigit
Diffusion-weighted magnetic resonance imaging (DW-MRI) can be used to characterise the microstructure of the nervous tissue, e.g. to delineate brain white matter connections in a non-invasive manner via fibre tracking. Magnetic Resonance Imaging (MRI) in high spatial resolution would play an important role in visualising such fibre tracts in a superior manner. However, obtaining an image of such resolution comes at the expense of longer scan time. Longer scan time can be associated with the increase of motion artefacts, due to the patient's psychological and physical conditions. Single Image S…
While medical imaging typically provides massive amounts of data, the extraction of relevant information for predictive diagnosis remains a difficult challenge. Functional MRI (fMRI) data, that provide an indirect measure of task-related or spontaneous neuronal activity, are classically analyzed in a mass-univariate procedure yielding statistical parametric maps. This analysis framework disregards some important principles of brain organization: population coding, distributed and overlapping representations. Multivariate pattern analysis, i.e., the prediction of behavioural variables from brai…
University of Eastern Finland（东芬兰大学）；Tampere University（坦佩雷大学）
The Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog) is a neuropsychological tool that has been designed to assess the severity of cognitive symptoms of dementia. Personalized prediction of the changes in ADAS-Cog scores could help in timing therapeutic interventions in dementia and at-risk populations. In the present work, we compared single and multitask learning approaches to predict the changes in ADAS-Cog scores based on T1-weighted anatomical magnetic resonance imaging (MRI). In contrast to most machine learning-based prediction methods ADAS-Cog changes, we stratified t…
Regularized joint reconstruction and slab combination for accelerated three-dimensional multi-slab diffusion-weighted imaging using multi-scale energy models
University of Virginia（弗吉尼亚大学）；University of Iowa（爱荷华大学）
Hunan University of Science and Technology（湖南科技大学）；Nanjing First Hospital, Nanjing Medical University（南京医科大学附属南京第一医院）；University of Macau（澳门大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Compressive sensing (CS) based computed tomography (CT) image reconstruction aims at reducing the radiation risk through sparse-view projection data. It is usually challenging to achieve satisfying image quality from incomplete projections. Recently, the nonconvex ${L_ {1/2}} $-norm has achieved promising performance in sparse recovery, while the applications on imaging are unsatisfactory due to its nonconvexity. In this paper, we develop a ${L_ {1/2}} $-regularized nonlocal self-similarity (NSS) denoiser for CT reconstruction problem, which integrates low-rank approximation with group sparse …
