Fast and accurate MRI image reconstruction from undersampled data is crucial in clinical practice. Deep learning based reconstruction methods have shown promising advances in recent years. However, recovering fine details from undersampled data is still challenging. In this paper, we introduce a novel deep learning based method, Pyramid Convolutional RNN (PC-RNN), to reconstruct images from multiple scales. Based on the formulation of MRI reconstruction as an inverse problem, we design the PC-RNN model with three convolutional RNN (ConvRNN) modules to iteratively learn the features in multiple scales. Each ConvRNN module reconstructs images at different scales and the reconstructed images are combined by a final CNN module in a pyramid fashion. The multi-scale ConvRNN modules learn a coarse-to-fine image reconstruction. Unlike other common reconstruction methods for parallel imaging, PC-RNN does not employ coil sensitive maps for multi-coil data and directly model the multiple coils as multi-channel inputs. The coil compression technique is applied to standardize data with various coil numbers, leading to more efficient training. We evaluate our model on the fastMRI knee and brain datasets and the results show that the proposed model outperforms other methods and can recover more details. The proposed method is one of the winner solutions in the 2019 fastMRI competition.
Assessing the structure and function of the right ventricle (RV) is important in the diagnosis of several cardiac pathologies. However, it remains more challenging to segment the RV than the left ventricle (LV). In this paper, we focus on segmenting the RV in both short (SA) and long-axis (LA) cardiac MR images simultaneously. For this task, we propose a new multi-input/output architecture, hybrid 2D/3D geometric spatial TransformEr Multi-Pass fEature pyRAmid (Tempera). Our feature pyramid extends current designs by allowing not only a multi-scale feature output but multi-scale SA and LA input…
Northeastern University（东北大学）；Shengjing Hospital, China Medical University（中国医科大学附属盛京医院）
In this paper, we propose a dual aggregation network to adaptively aggregate different information in infant brain MRI segmentation. More precisely, we added two modules based on 3D-UNet to better model information at different levels and locations. The dilated convolution pyramid downsampling module is mainly to solve the problem of loss of spatial information on the downsampling process, and it can effectively save details while reducing the resolution. The self-attention module can integrate the remote dependence on the feature maps in two dimensions of spatial and channel, effectively impr…
Reconstructing magnetic resonance (MR) images from undersampled data is a challenging problem due to various artifacts introduced by the under-sampling operation. Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture which captures low-level features at the initial layers and high-level features at the deeper layers. Such networks focus much on global features which may not be optimal to reconstruct the fully-sampled image. In this paper, we propose an Over-and-Under Complete Convolutional Recurrent Neural Network (OUCR), which cons…
We propose and evaluate a new MRI reconstruction method named LORAKI that trains an autocalibrated scan-specific recurrent neural network (RNN) to recover missing k-space data. Methods like GRAPPA, SPIRiT, and AC-LORAKS assume that k-space data has shift-invariant autoregressive structure, and that the scan-specific autoregression relationships needed to recover missing samples can be learned from fully-sampled autocalibration (ACS) data. Recently, the structure of the linear GRAPPA method has been translated into a nonlinear deep learning method named RAKI. RAKI uses ACS data to train an arti…
TU Munich（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）；ETH Zurich（苏黎世联邦理工学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Brain pathologies can vary greatly in size and shape, ranging from few pixels (i.e. MS lesions) to large, space-occupying tumors. Recently proposed Autoencoder-based methods for unsupervised anomaly segmentation in brain MRI have shown promising performance, but face difficulties in modeling distributions with high fidelity, which is crucial for accurate delineation of particularly small lesions. Here, similar to these previous works, we model the distribution of healthy brain MRI to localize pathologies from erroneous reconstructions. However, to achieve improved reconstruction fidelity at hi…
University of Florida（佛罗里达大学）；Nvidia Corporation（英伟达公司）
Automatic pancreas segmentation in radiology images, eg., computed tomography (CT) and magnetic resonance imaging (MRI), is frequently required by computer-aided screening, diagnosis, and quantitative assessment. Yet pancreas is a challenging abdominal organ to segment due to the high inter-patient anatomical variability in both shape and volume metrics. Recently, convolutional neural networks (CNNs) have demonstrated promising performance on accurate segmentation of pancreas. However, the CNN-based method often suffers from segmentation discontinuity for reasons such as noisy image quality an…
EPRA U-Net: An Efficient Pyramid Residual Attention Framework for Accurate Infarct Segmentation in Diffusion-Weighted MRI
Yozgat Bozok University（亚兹加特博兹克大学）；Izmir Bakircay University（伊兹米尔巴克伊大学）；Queen Mary’s Digital Environment Research Institute (DERI)（皇后玛丽数字环境研究 institute (DERI)）；Firat University（费拉特大学）；Qatar University（卡塔尔大学）
Pyramid Attention Network for Medical Image Registration
School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；Medical UltraSound Image Computing (MUSIC) Lab（医学超声图像计算实验室）；Smart Medical Imaging, Learning and Engineering (SMILE) Lab（智能医学影像、学习与工程实验室）
Accelerating magnetic resonance image (MRI) reconstruction process is a challenging ill-posed inverse problem due to the excessive under-sampling operation in k-space. In this paper, we propose a recurrent transformer model, namely ReconFormer, for MRI reconstruction which can iteratively reconstruct high fertility magnetic resonance images from highly under-sampled k-space data. In particular, the proposed architecture is built upon Recurrent Pyramid Transformer Layers (RPTL), which jointly exploits intrinsic multi-scale information at every architecture unit as well as the dependencies of th…
Center for Biomedical Image Computing and Analytics（生物医学图像计算与分析中心）；Perelman School of Medicine（佩雷尔曼医学院）；University of Pennsylvania（宾夕法尼亚大学）
Decoding brain functional states underlying different cognitive processes using multivariate pattern recognition techniques has attracted increasing interests in brain imaging studies. Promising performance has been achieved using brain functional connectivity or brain activation signatures for a variety of brain decoding tasks. However, most of existing studies have built decoding models upon features extracted from imaging data at individual time points or temporal windows with a fixed interval, which might not be optimal across different cognitive processes due to varying temporal durations…
East China Normal University（华东师范大学）；Tongji University（同济大学）
The encoder-decoder networks are commonly used in medical image segmentation due to their remarkable performance in hierarchical feature fusion. However, the expanding path for feature decoding and spatial recovery does not consider the long-term dependency when fusing feature maps from different layers, and the universal encoder-decoder network does not make full use of the multi-modality information to improve the network robustness especially for segmenting medical MRI. In this paper, we propose a novel feature fusion unit called Recurrent Decoding Cell (RDC) which leverages convolutional R…
Biomediq A/S（Biomediq公司）；University of Copenhagen（哥本哈根大学）；University College London（伦敦大学学院）
Disease progression modeling (DPM) using longitudinal data is a challenging task in machine learning for healthcare that can provide clinicians with better tools for diagnosis and monitoring of disease. Existing DPM algorithms neglect temporal dependencies among measurements and make parametric assumptions about biomarker trajectories. In addition, they do not model multiple biomarkers jointly and need to align subjects' trajectories. In this paper, recurrent neural networks (RNNs) are utilized to address these issues. However, in many cases, longitudinal cohorts contain incomplete data, which…
Malong LLC（马龙有限责任公司）；Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；ARTORG Center for Biomedical Engineering Research, Univ. of Bern（伯尔尼大学ARTORG生物医学工程研究中心）
Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Multi-modal biological, imaging, and neuropsychological markers have demonstrated promising performance for distinguishing Alzheimer's disease (AD) patients from cognitively normal elders. However, it remains difficult to early predict when and which mild cognitive impairment (MCI) individuals will convert to AD dementia. Informed by pattern classification studies which have demonstrated that pattern classifiers built on longitudinal data could achieve better classification performance than those built on cross-sectional data, we develop a deep learning model based on recurrent neural networks…
Shandong Normal University（山东师范大学）；Western University（韦仕敦大学）
Simultaneous segmentation and detection of liver tumors (hemangioma and hepatocellular carcinoma (HCC)) by using multi-modality non-contrast magnetic resonance imaging (NCMRI) are crucial for the clinical diagnosis. However, it is still a challenging task due to: (1) the HCC information on NCMRI is invisible or insufficient makes extraction of liver tumors feature difficult; (2) diverse imaging characteristics in multi-modality NCMRI causes feature fusion and selection difficult; (3) no specific information between hemangioma and HCC on NCMRI cause liver tumors detection difficult. In this stu…
iPEAR: Iterative Pyramid Estimation with Attention and Residuals for Deformable Medical Image Registration
College of Software, Jilin University（吉林大学软件学院）；Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Nanyang Technological University（老年活跃生活卓越研究中⼼（NTU-UBC）, 新加坡国立大学）；College of Computer Science（计算机科学学院）；Technology, Zhejiang University（技术, 浙江大学）；Key Laboratory of Symbolic Computation（符号计算重点实验室）；Knowledge Engineering of Ministry of Education, College of Computer Science（教育部知识工程重点实验室, 计算机科学学院）；Technology, Jilin University（技术, 吉林大学）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
To deform or not: treatment-aware longitudinal registration for breast DCE-MRI during neoadjuvant chemotherapy via unsupervised keypoints detection
Radboud University Medical Centre（拉德堡德大学医学中心）；The Netherlands Cancer Institute（荷兰癌症研究所）；Macao Polytechnic University（澳门理工大学）；Maastricht University Medical Centre（马斯特里赫特大学医学中心）
Shanghai Jiao Tong University（上海交通大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Ruijin Hospital affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Mayo Clinic in Arizona（亚利桑那州妙佑医疗国际）；ShanghaiTech University（上海科技大学）
Interventional magnetic resonance imaging (i-MRI) for surgical guidance could help visualize the interventional process such as deep brain stimulation (DBS), improving the surgery performance and patient outcome. Different from retrospective reconstruction in conventional dynamic imaging, i-MRI for DBS has to acquire and reconstruct the interventional images sequentially online. Here we proposed a convolutional long short-term memory (Conv-LSTM) based recurrent neural network (RNN), or ConvLR, to reconstruct interventional images with golden-angle radial sampling. By using an initializer and C…
Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Dynamic functional connectivity analysis provides valuable information for understanding brain functional activity underlying different cognitive processes. Besides sliding window based approaches, a variety of methods have been developed to automatically split the entire functional MRI scan into segments by detecting change points of functional signals to facilitate better characterization of temporally dynamic functional connectivity patterns. However, these methods are based on certain assumptions for the functional signals, such as Gaussian distribution, which are not necessarily suitable …
Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) automatic 3-D registration is implemented and validated for small animal image volumes so that the high-resolution anatomical MRI information can be fused with the low spatial resolution of functional PET information for the localization of lesion that is currently in high demand in the study of tumor of cancer (oncology) and its corresponding preparation of pharmaceutical drugs. Though many registration algorithms are developed and applied on human brain volumes, these methods may not be as efficient on small animal datas…
Deep Learning-Based Diffusion MRI Tractography: Integrating Spatial and Anatomical Information
School of Biomedical Engineering, Southern Medical University（南方医科大学生物医学工程学院）；School of Computer Science and Technology, Nanjing University of Science and Technology（南京理工大学计算机科学与技术学院）；Shunde Hospital, Southern Medical University（南方医科大学顺德医院）
Speech Audio Synthesis from Tagged MRI and Non-Negative Matrix Factorization via Plastic Transformer
Massachusetts General Hospital（马萨诸塞州总医院）；Harvard Medical School（哈佛医学院）；University of Maryland（马里兰大学）；University of British Columbia（不列颠哥伦比亚大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Xiao Ying AI Lab（小鹰人工智能实验室）；School of Electrical and Information Engineering, Tianjin University（天津大学电气自动化与信息工程学院）；Peking Union Medical College Hospital（北京协和医院）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）
Magnetic resonance imaging plays an essential role in clinical diagnosis by acquiring the structural information of biological tissue. Recently, many multi-contrast MRI super-resolution networks achieve good effects. However, most studies ignore the impact of the inappropriate foreground scale and patch size of multi-contrast MRI, which probably leads to inappropriate feature alignment. To tackle this problem, we propose the Flexible Alignment Super-Resolution Network (FASR-Net) for multi-contrast MRI Super-Resolution. The Flexible Alignment module of FASR-Net consists of two modules for featu…
École de technologie supérieure ÉTS（高等技术学院）；Université de Nantes（南特大学）；Brigham and Women’s Hospital, Harvard Medical School（哈佛医学院布莱根妇女医院）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；National Research Council（国家研究委员会）
This work details a highly efficient implementation of the 3D scale-invariant feature transform (SIFT) algorithm, for the purpose of machine learning from large sets of volumetric medical image data. The primary operations of the 3D SIFT code are implemented on a graphics processing unit (GPU), including convolution, sub-sampling, and 4D peak detection from scale-space pyramids. The performance improvements are quantified in keypoint detection and image-to-image matching experiments, using 3D MRI human brain volumes of different people. Computationally efficient 3D keypoint descriptors are pro…
United Imaging Intelligence（联影智能）；UIH America, Inc.（联影美国公司）
Real-time cardiac cine MRI does not require ECG gating in the data acquisition and is more useful for patients who can not hold their breaths or have abnormal heart rhythms. However, to achieve fast image acquisition, real-time cine commonly acquires highly undersampled data, which imposes a significant challenge for MRI image reconstruction. We propose a residual convolutional RNN for real-time cardiac cine reconstruction. To the best of our knowledge, this is the first work applying deep learning approach to Cartesian real-time cardiac cine reconstruction. Based on the evaluation from radiol…
University of Waterloo（滑铁卢大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）
We present Recurrence with Correlation Network (RWCNet), a medical image registration network with multi-scale features and a cost volume layer. We demonstrate that these architectural features improve medical image registration accuracy in two image registration datasets prepared for the MICCAI 2022 Learn2Reg Workshop Challenge. On the large-displacement National Lung Screening Test (NLST) dataset, RWCNet is able to achieve a total registration error (TRE) of 2.11mm between corresponding keypoints without instance fine-tuning. On the OASIS brain MRI dataset, RWCNet is able to achieve an avera…
The Hong Kong University of Science and Technology（香港科技大学）
Registration of pre-operative and follow-up brain MRI scans is challenging due to the large variation of tissue appearance and missing correspondences in tumour recurrence regions caused by tumour mass effect. Although recent deep learning-based deformable registration methods have achieved remarkable success in various medical applications, most of them are not capable of registering images with pathologies. In this paper, we propose a 3-step registration pipeline for pre-operative and follow-up brain MRI scans that consists of 1) a multi-level affine registration, 2) a conditional deep Lapla…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Imperial College London（伦敦帝国学院）
Our main objective is to develop a novel deep learning-based algorithm for automatic segmentation of prostate zone and to evaluate the proposed algorithm on an additional independent testing data in comparison with inter-reader consistency between two experts. With IRB approval and HIPAA compliance, we designed a novel convolutional neural network (CNN) for automatic segmentation of the prostatic transition zone (TZ) and peripheral zone (PZ) on T2-weighted (T2w) MRI. The total study cohort included 359 patients from two sources; 313 from a deidentified publicly available dataset (SPIE-AAPM-NCI…
University of Leicester（莱斯特大学）；NIHR Leicester Biomedical Research Centre（NIHR莱斯特生物医学研究中心）；Glenfield Hospital（格伦菲尔德医院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学）；Saint-Petersburg State Electrotechnical University（圣彼得堡国立电工大学）
A "heart attack" or myocardial infarction (MI), occurs when an artery supplying blood to the heart is abruptly occluded. The "gold standard" method for imaging MI is Cardiovascular Magnetic Resonance Imaging (MRI), with intravenously administered gadolinium-based contrast (late gadolinium enhancement). However, no "gold standard" fully automated method for the quantification of MI exists. In this work, we propose an end-to-end fully automatic system (MyI-Net) for the detection and quantification of MI in MRI images. This has the potential to reduce the uncertainty due to the technical variabil…
University of British Columbia（不列颠哥伦比亚大学）；BC Cancer Research Institute（不列颠哥伦比亚癌症研究所）；Geneva University Hospital（日内瓦大学医院）
Radiomics features extract quantitative information from medical images, towards the derivation of biomarkers for clinical tasks, such as diagnosis, prognosis, or treatment response assessment. Different image discretization parameters (e.g. bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels can be used to generate radiomics features and ultimately signatures. Commonly, only one set of parameters is used; resulting in only one value or flavour for a given RF. We propose tensor radiomics (TR) where tensors of features calculated with multiple …
Self-Consistent Nested Diffusion Bridge for Accelerated MRI Reconstruction
School of Information Science and Technology, Fudan University, Shanghai, China（复旦大学信息科学与技术学院）；Department of Data Science & AI, Faculty of Information Technology, Monash University, Melbourne, Australia（墨尔本大学信息科技学院数据科学与人工智能系）；Nuffield Department of Clinical Neurosciences, University of Oxford, London, UK（牛津大学临床神经科学系）；School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China（电子科技大学机械与电子工程学院）；Department of Radiology, Sichuan Provincial People’s Hospital, Chengdu, China（四川省人民医院放射科）；Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing, China（东南大学医学院中大医院放射科）；Shanghai AI Lab, Shanghai, China（上海人工智能实验室）；Sensetime Research, Shanghai, China（商汤研究）
Boosting ViT-based MRI Reconstruction from the Perspectives of Frequency Modulation, Spatial Purification, and Scale Diversification
Fudan University（复旦大学）；Digital Medical Research Center（数字医学研究中心）；School of Basic Medical Science（基础医学院）；Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention（上海医学影像计算与计算机辅助干预重点实验室）；Academy for Engineering and Technology（工程与技术学院）
