Multi-task Swin Transformer for Motion Artifacts Classification and Cardiac Magnetic Resonance Image Segmentation
Sano Centre for Computational Medicine（萨诺计算医学中心）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Michal K. Grzeszczyk, Szymon Płotka, Arkadiusz Sitek
备注：
Accepted for Statistical Atlases and Computational Modeling of the Heart (STACOM) workshop, The Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion) challenge
Cardiac Magnetic Resonance Imaging is commonly used for the assessment of the cardiac anatomy and function. The delineations of left and right ventricle blood pools and left ventricular myocardium are important for the diagnosis of cardiac diseases. Unfortunately, the movement of a patient during the CMR acquisition procedure may result in motion artifacts appearing in the final image. Such artifacts decrease the diagnostic quality of CMR images and force redoing of the procedure. In this paper, we present a Multi-task Swin UNEt TRansformer network for simultaneous solving of two tasks in the CMRxMotion challenge: CMR segmentation and motion artifacts classification. We utilize both segmentation and classification as a multi-task learning approach which allows us to determine the diagnostic quality of CMR and generate masks at the same time. CMR images are classified into three diagnostic quality classes, whereas, all samples with non-severe motion artifacts are being segmented. Ensemble of five networks trained using 5-Fold Cross-validation achieves segmentation performance of DICE coefficient of 0.871 and classification accuracy of 0.595.
Technische Universität München（慕尼黑工业大学）；Fudan University（复旦大学）；Huawei Technologies（华为技术有限公司）
In the past few years, convolutional neural networks (CNNs) have achieved milestones in medical image analysis. Especially, the deep neural networks based on U-shaped architecture and skip-connections have been widely applied in a variety of medical image tasks. However, although CNN has achieved excellent performance, it cannot learn global and long-range semantic information interaction well due to the locality of the convolution operation. In this paper, we propose Swin-Unet, which is an Unet-like pure Transformer for medical image segmentation. The tokenized image patches are fed into the …
Broadly speaking, the objective in cardiac image segmentation is to delineate the outer and inner walls of the heart to segment out either the entire or parts of the organ boundaries. This paper will focus on MR images as they are the most widely used in cardiac segmentation -- as a result of the accurate morphological information and better soft tissue contrast they provide. This cardiac segmentation information is very useful as it eases physical measurements that provides useful metrics for cardiac diagnosis such as infracted volumes, ventricular volumes, ejection fraction, myocardial mass,…
Rutgers University（罗格斯大学）；Nvidia Corporation（英伟达公司）；New York University（纽约大学）
The 3D volumetric shape of the heart's left ventricle (LV) myocardium (MYO) wall provides important information for diagnosis of cardiac disease and invasive procedure navigation. Many cardiac image segmentation methods have relied on detection of region-of-interest as a pre-requisite for shape segmentation and modeling. With segmentation results, a 3D surface mesh and a corresponding point cloud of the segmented cardiac volume can be reconstructed for further analyses. Although state-of-the-art methods (e.g., U-Net) have achieved decent performance on cardiac image segmentation in terms of ac…
Dublin City University（都柏林城市大学）；University College Dublin（都柏林大学学院）
Methods that are resilient to artifacts in the cardiac magnetic resonance imaging (MRI) while performing ventricle segmentation, are crucial for ensuring quality in structural and functional analysis of those tissues. While there has been significant efforts on improving the quality of the algorithms, few works have tackled the harm that the artifacts generate in the predictions. In this work, we study fine tuning of pretrained networks to improve the resilience of previous methods to these artifacts. In our proposed method, we adopted the extensive usage of data augmentations that mimic those…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Cedars-Sinai Medical Center（西达赛奈医疗中心）；University of California, Los Angeles（加州大学洛杉矶分校）
Cine cardiac magnetic resonance imaging (MRI) is widely used for diagnosis of cardiac diseases thanks to its ability to present cardiovascular features in excellent contrast. As compared to computed tomography (CT), MRI, however, requires a long scan time, which inevitably induces motion artifacts and causes patients' discomfort. Thus, there has been a strong clinical motivation to develop techniques to reduce both the scan time and motion artifacts. Given its successful applications in other medical imaging tasks such as MRI super-resolution and CT metal artifact reduction, deep learning is a…
Purpose: To develop a method for banding-free bSSFP cardiac cine with substantially reduced flow artifacts. Methods: A projection-reconstruction (PR) trajectory is proposed for a frequency-modulated cine sequence, facilitating reconstruction of three phase cycles and a field-map time series from a short, breath-held scan. Data is also acquired during the gradient rewinders to enable generation of field maps using BMART, B$_0$ mapping using rewinding trajectories, where the rewind data forms the second TE image for calculating the field map. A field-map-based combination method is developed whi…
Reconstruction of Cardiac Cine MRI Using Motion-Guided Deformable Alignment and Multi-Resolution Fusion
School of Health Science and Engineering, University of Shanghai for Science and Technology（上海理工大学健康科学与工程学院）；ToolSensing Technologies Co., Ltd（拓深科技有限公司）；School of Medical Instruments, Shanghai University of Medicine and Health Sciences（上海健康医学院医疗器械学院）；Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication & Electronic Engineering, East China Normal University（华东师范大学通信与电子工程学院多维信息处理上海重点实验室）
University of Würzburg（维尔茨堡大学）；MegaStudyEdu（MegaStudy教育）
Compression plays an important role on the efficient transmission and storage of images and videos through band-limited systems such as streaming services, virtual reality or videogames. However, compression unavoidably leads to artifacts and the loss of the original information, which may severely degrade the visual quality. For these reasons, quality enhancement of compressed images has become a popular research topic. While most state-of-the-art image restoration methods are based on convolutional neural networks, other transformers-based methods such as SwinIR, show impressive performance …
Coronary Mask Guided Registration for Continuous Time 4D Cardiac CT Dataset Construction
Tsinghua University（清华大学）；The Ohio State University（俄亥俄州立大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College（中国医学科学院阜外医院 北京协和医学院）；Tianjin Medical University General Hospital（天津医科大学总医院）
University of Science and Technology of China（中国科学技术大学）
Compressed Image Super-resolution has achieved great attention in recent years, where images are degraded with compression artifacts and low-resolution artifacts. Since the complex hybrid distortions, it is hard to restore the distorted image with the simple cooperation of super-resolution and compression artifacts removing. In this paper, we take a step forward to propose the Hierarchical Swin Transformer (HST) network to restore the low-resolution compressed image, which jointly captures the hierarchical feature representations and enhances each-scale representation with Swin transformer, re…
Fudan University（复旦大学）；Imperial College London（帝国理工学院）；University of Oxford（牛津大学）；Zhongshan Hospital Affiliated to Fudan University（复旦大学附属中山医院）
The quality of cardiac magnetic resonance (CMR) imaging is susceptible to respiratory motion artifacts. The model robustness of automated segmentation techniques in face of real-world respiratory motion artifacts is unclear. This manuscript describes the design of extreme cardiac MRI analysis challenge under respiratory motion (CMRxMotion Challenge). The challenge aims to establish a public benchmark dataset to assess the effects of respiratory motion on image quality and examine the robustness of segmentation models. The challenge recruited 40 healthy volunteers to perform different breath-ho…
Sun Yat-sen University（中山大学）；China Three Gorges University（三峡大学）；The First People’s Hospital of Yichang（宜昌市第一人民医院）
Decreasing projection views to lower X-ray radiation dose usually leads to severe streak artifacts. To improve image quality from sparse-view data, a Multi-domain Integrative Swin Transformer network (MIST-net) was developed in this article. First, MIST-net incorporated lavish domain features from data, residual-data, image, and residual-image using flexible network architectures, where residual-data and residual-image sub-network was considered as data consistency module to eliminate interpolation and reconstruction errors. Second, a trainable edge enhancement filter was incorporated to detec…
Beijing University of Posts and Telecommunications（北京邮电大学）；Dalian University of Technology（大连理工大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）
The success of Transformer in computer vision has attracted increasing attention in the medical imaging community. Especially for medical image segmentation, many excellent hybrid architectures based on convolutional neural networks (CNNs) and Transformer have been presented and achieve impressive performance. However, most of these methods, which embed modular Transformer into CNNs, struggle to reach their full potential. In this paper, we propose a novel hybrid architecture for medical image segmentation called PHTrans, which parallelly hybridizes Transformer and CNN in main building blocks …
KTH Royal Institute of Technology（KTH皇家理工学院）；Science for Life Laboratory（生命科学实验室）
Artifacts on magnetic resonance scans are a serious challenge for both radiologists and computer-aided diagnosis systems. Most commonly, artifacts are caused by motion of the patients, but can also arise from device-specific abnormalities such as noise patterns. Irrespective of the source, artifacts can not only render a scan useless, but can potentially induce misdiagnoses if left unnoticed. For instance, an artifact may masquerade as a tumor or other abnormality. Retrospective artifact correction (RAC) is concerned with removing artifacts after the scan has already been taken. In this work, …
Respiratory motion can cause strong blurring artifacts in the reconstructed image during MR acquisition. These artifacts become more prominent when use in the presence of undersampled data. Recently, compressed sensing (CS) is developed as an MR reconstruction technique, to recover good quality images from the compressive k-space samples. To maximize the benefits of CS in free breathing data, it is understandable to use CS with the motion corrected images. In this paper, we have developed a new CS based motion corrected image reconstruction technique. In this two-stage technique, we use simila…
Image restoration is a long-standing low-level vision problem that aims to restore high-quality images from low-quality images (e.g., downscaled, noisy and compressed images). While state-of-the-art image restoration methods are based on convolutional neural networks, few attempts have been made with Transformers which show impressive performance on high-level vision tasks. In this paper, we propose a strong baseline model SwinIR for image restoration based on the Swin Transformer. SwinIR consists of three parts: shallow feature extraction, deep feature extraction and high-quality image recons…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Imperial College London（帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）
Temporal patterns of cardiac motion provide important information for cardiac disease diagnosis. This pattern could be obtained by three-directional CINE multi-slice left ventricular myocardial velocity mapping (3Dir MVM), which is a cardiac MR technique providing magnitude and phase information of the myocardial motion simultaneously. However, long acquisition time limits the usage of this technique by causing breathing artifacts, while shortening the time causes low temporal resolution and may provide an inaccurate assessment of cardiac motion. In this study, we proposed a frame synthesis al…
University of Science and Technology of China（中国科学技术大学）
Image compression has raised widespread interest recently due to its significant importance for multimedia storage and transmission. Meanwhile, a reliable image quality assessment (IQA) for compressed images can not only help to verify the performance of various compression algorithms but also help to guide the compression optimization in turn. In this paper, we design a full-reference image quality assessment metric SwinIQA to measure the perceptual quality of compressed images in a learned Swin distance space. It is known that the compression artifacts are usually non-uniformly distributed w…
Monash Biomedical Imaging, Monash University（莫纳什大学生物医学成像中心）；Department of Electrical and Computer Systems Engineering, Monash University（莫纳什大学电气与计算机系统工程系）；School of Psychological Sciences, Monash University（莫纳什大学心理科学学院）；Department of Data Science and AI, Monash University（莫纳什大学数据科学与人工智能系）
Global correlations are widely seen in human anatomical structures due to similarity across tissues and bones. These correlations are reflected in magnetic resonance imaging (MRI) scans as a result of close-range proton density and T1/T2 parameters. Furthermore, to achieve accelerated MRI, k-space data are undersampled which causes global aliasing artifacts. Convolutional neural network (CNN) models are widely utilized for accelerated MRI reconstruction, but those models are limited in capturing global correlations due to the intrinsic locality of the convolution operation. The self-attention-…
University of Virginia（弗吉尼亚大学）；University of Virginia Health System（弗吉尼亚大学卫生系统）
Automated identification of myocardial scar from late gadolinium enhancement cardiac magnetic resonance images (LGE-CMR) is limited by image noise and artifacts such as those related to motion and partial volume effect. This paper presents a novel joint deep learning (JDL) framework that improves such tasks by utilizing simultaneously learned myocardium segmentations to eliminate negative effects from non-region-of-interest areas. In contrast to previous approaches treating scar detection and myocardium segmentation as separate or parallel tasks, our proposed method introduces a message passin…
Deep Learning Assisted Outer Volume Removal for Highly-Accelerated Real-Time Dynamic MRI
Department of Electrical and Computer Engineering, University of Minnesota（电气与计算机工程系，明尼苏达大学）；Center for Magnetic Resonance Research, University of Minnesota（磁共振研究中心，明尼苏达大学）；Department of Imaging Physics, Delft University of Technology（成像物理系，代尔夫特理工大学）；HollandPTC, Delft, the Netherlands（HollandPTC，代尔夫特，荷兰）
DualSwinUnet++: An Enhanced Swin-Unet Architecture With Dual Decoders For PTMC Segmentation
Department of Mechanical and Mechatronics Engineering（机械与机电工程系）；University of Waterloo（滑铁卢大学）；Department of Radiology Withsim Clinic Seongnam, Korea（Seongnam诊所放射科）
Motion free reconstruction of compressively sampled cardiac perfusion MR images is a challenging problem. It is due to the aliasing artifacts and the rapid contrast changes in the reconstructed perfusion images. In addition to the reconstruction limitations, many registration algorithms under perform in the presence of the rapid intensity changes. In this paper, we propose a novel motion correction method that reconstructs the motion free image series from the undersampled cardiac perfusion MR data. The motion correction method uses the novel robust principal component analysis based reconstru…
Direct Cardiac Segmentation from Undersampled K-space Using Transformers
School of Computation, Information and Technology, Technical University Munich（计算、信息与技术学院，技术大学慕尼黑）；Klinikum Rechts der Isar, Technical University Munich（技术大学慕尼黑医院）；Munich Center for Machine Learning, Technical University Munich（慕尼黑机器学习中心，技术大学慕尼黑）；Department of Computing, Imperial College London（计算系，帝国理工学院伦敦）
Preserving Cardiac Integrity: A Topology-Infused Approach to Whole Heart Segmentation
Imperial College London（帝国理工学院）；Jilin university（吉林大学）；National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；School of Biomedical Engineering & Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）
In nuclear imaging, limited resolution causes partial volume effects (PVEs) that affect image sharpness and quantitative accuracy. Partial volume correction (PVC) methods incorporating high-resolution anatomical information from CT or MRI have been demonstrated to be effective. However, such anatomical-guided methods typically require tedious image registration and segmentation steps. Accurately segmented organ templates are also hard to obtain, particularly in cardiac SPECT imaging, due to the lack of hybrid SPECT/CT scanners with high-end CT and associated motion artifacts. Slight mis-regist…
Istanbul Technical University（伊斯坦布尔技术大学）；Mehmet Akif Ersoy Thoracic and Cardiovascular Surgery Training and Research Hospital（穆罕默德·阿基夫·埃尔索伊胸心血管外科培训研究医院）；Acibadem University（阿哲巴登大学）；King’s College London（伦敦国王学院）
To maintain a standard in a medical imaging study, images should have necessary image quality for potential diagnostic use. Although CNN-based approaches are used to assess the image quality, their performance can still be improved in terms of accuracy. In this work, we approach this problem by using Swin Transformer, which improves the poor-quality image classification performance that causes the degradation in medical image quality. We test our approach on Foreign Object Classification problem on Chest X-Rays (Object-CXR) and Left Ventricular Outflow Tract Classification problem on Cardiac M…
Extreme Cardiac MRI Analysis under Respiratory Motion: Results of the CMRxMotion Challenge
Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, Shanghai 200032, China；Shanghai Key Laboratory of MICCAI, Fudan University, Shanghai, Shanghai 200032, China；Department of Electrical；Electronic Engineering \& I-X, Imperial College London, London, London SW7 2AZ, United Kingdom；Department of Radiology, Zhongshan Hospital Affiliated to Fudan University, Shanghai, Shanghai 200032, China；Department of Computing, Imperial College London, London, London SW7 2AZ, United Kingdom；School of Computer Science, University of Sheffield, Sheffield, S1 4DP, United Kingdom；Department of Engineering Science, University of Oxford, Oxford, OX2 0ES, United Kingdom；Shanghai Pudong Hospital；Human Phenome Institute, Fudan University, Shanghai, 201203, China；School of Computer Science, University of Nottingham, Nottingham, NG8 1BB, United Kingdom；Diagnostic Imaging, University of Alberta, Edmonton, AB T6G 1K4, Canada；Department of Computer Science；Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong 000000, China；The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology；Reproduction, Maastricht University, 6220 MD Maastricht, The Netherlands；Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven 5612 AZ, The Netherlands；Department of Radiology, Northwestern University, 737 N. Michigan Ave, Suite 1600, Chicago 60611, United States；United Imaging Research, 393 Middle Huaxia Road, Pudong, Shanghai 201210, China；Division of Image Processing, Department of Radiology, Leiden University Medical Center, PO Box 9600, Leiden 2300 RC, The Netherlands；Université Bourgogne Europe, CNRS, ICMUB UMR 6302, 21000 Dijon, France；Istanbul Technical University, Maslak, 34467, İstanbul, Türkiye；Computer Science, Technical University of Darmstadt, Karolinenpl. 5, 64289 Darmstadt, Germany；Lung Institute, Faculty of Medicine, Imperial College London, Guy Scadding Building, Cale Street, London, SW3 6LY,United Kingdom；Hawkes Institute, Department of Computer Science, University College London, 66-72 Gower St, London, United Kingdom；School of Medicine, University College Dublin, Belfield, Dublin, D04 V1W8, Ireland；CeADAR: Ireland's Centre for AI, University College Dublin, Belfield, Dublin, D04 V1W8, Ireland；Sano Centre for Computational Medicine, Czarnowiejska 36, 30-054, Krakow, Poland；Faculty of Mathematics；Computer Science, Jagiellonian University, S. Łojasiewicza 6, Krakow, Poland；Department of Electronic；Computer Engineering, The Hong Kong University of Science；School of Instrument Science；Engineering, Southeast University, Nanjing, Nanjing 210096, China；College of Artificial Intelligence, Nanjing University of Aeronautics；Academy for Engineering；Technology, Fudan University, Shanghai, Shanghai 200433, China；College of Biomedical Engineering, Fudan University, Shanghai, Shanghai 200433, China；Institute of Science；Technology for Brain-inspired Intelligence, Fudan University, Shanghai, Shanghai 200433, China；Department of Brain Sciences, Imperial College London, London, London SW7 2AZ, United Kingdom；Data Science Institute, Imperial College London, London, London SW7 2AZ, United Kingdom
University of California, Santa Cruz（加州大学圣克鲁兹分校）；University of Oklahoma（俄克拉荷马大学）；Shahid Beheshti University（沙希德·贝赫什提大学）
There are many sources of interference encountered in the electroencephalogram (EEG) recordings, specifically ocular, muscular, and cardiac artifacts. Rejection of EEG artifacts is an essential process in EEG analysis since such artifacts cause many problems in EEG signals analysis. One of the most challenging issues in EEG denoising processes is removing the ocular artifacts where Electrooculographic (EOG), and EEG signals have an overlap in both frequency and time domains. In this paper, we build and train a deep learning model to deal with this challenge and remove the ocular artifacts effe…
Dynamic Magnetic Resonance Imaging (dMRI) is widely used to assess various cardiac conditions such as cardiac motion and blood flow. To accelerate MR acquisition, techniques such as undersampling and Simultaneous Multi-Slice (SMS) are often used. Special reconstruction algorithms are needed to reconstruct multiple SMS image slices from the entangled information. Deep learning (DL)-based methods have shown promising results for single-slice MR reconstruction, but the addition of SMS acceleration raises unique challenges due to the composite k-space signals and the resulting images with strong i…
Shanghai Jiao Tong University（上海交通大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Calgary（卡尔加里大学）
Eliminating ghosting artifacts due to moving objects is a challenging problem in high dynamic range (HDR) imaging. In this letter, we present a hybrid model consisting of a convolutional encoder and a Transformer decoder to generate ghost-free HDR images. In the encoder, a context aggregation network and non-local attention block are adopted to optimize multi-scale features and capture both global and local dependencies of multiple low dynamic range (LDR) images. The decoder based on Swin Transformer is utilized to improve the reconstruction capability of the proposed model. Motivated by the p…
