An investigation into the causes of race bias in AI-based cine CMR segmentation
School of Biomedical Engineering & Imaging Sciences, King's College London（伦敦国王学院生物医学工程与影像科学学院）；HeartFlow Inc（HeartFlow公司）；Guy’s and St Thomas’ Hospital（盖伊和圣托马斯医院）；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
Tiarna Lee, Esther Puyol-Anton, Bram Ruijsink, Sebastien Roujol, Theodore Barfoot, Shaheim Ogbomo-Harmitt, Miaojing Shi, Andrew P. King
Artificial intelligence (AI) methods are being used increasingly for the automated segmentation of cine cardiac magnetic resonance (CMR) imaging. However, these methods have been shown to be subject to race bias, i.e. they exhibit different levels of performance for different races depending on the (im)balance of the data used to train the AI model. In this paper we investigate the source of this bias, seeking to understand its root cause(s) so that it can be effectively mitigated. We perform a series of classification and segmentation experiments on short-axis cine CMR images acquired from Black and White subjects from the UK Biobank and apply AI interpretability methods to understand the results. In the classification experiments, we found that race can be predicted with high accuracy from the images alone, but less accurately from ground truth segmentations, suggesting that the distributional shift between races, which is often the cause of AI bias, is mostly image-based rather than segmentation-based. The interpretability methods showed that most attention in the classification models was focused on non-heart regions, such as subcutaneous fat. Cropping the images tightly around the heart reduced classification accuracy to around chance level. Similarly, race can be predicted from the latent representations of a biased segmentation model, suggesting that race information is encoded in the model. Cropping images tightly around the heart reduced but did not eliminate segmentation bias. We also investigate the influence of possible confounders on the bias observed.
National University of Singapore（新加坡国立大学）；National University Heart Centre（新加坡国立大学心脏中心）
Automatic segmentation of myocardium in Late Gadolinium Enhanced (LGE) Cardiac MR (CMR) images is often difficult due to the intensity heterogeneity resulting from accumulation of contrast agent in infarcted areas. In this paper, we propose an automatic segmentation framework that fully utilizes shared information between corresponding cine and LGE images of a same patient. Given myocardial contours in cine CMR images, the proposed framework achieves accurate segmentation of LGE CMR images in a coarse-to-fine manner. Affine registration is first performed between the corresponding cine and LGE…
CineMyoPS: Segmenting Myocardial Pathologies from Cine Cardiac MR
School of Medical Imaging, Fujian Medical University（福建医科大学医学影像学院）；College of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）；Department of Biomedical Engineering, National University of Singapore（新加坡国立大学生物医学工程系）；School of Data Science, Fudan University（复旦大学数据科学学院）；Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院仁济医院放射科）
Cine cardiac magnetic resonance (CMR) imaging is considered the gold standard for cardiac function evaluation. However, cine CMR acquisition is inherently slow and in recent decades considerable effort has been put into accelerating scan times without compromising image quality or the accuracy of derived results. In this paper, we present a fully-automated, quality-controlled integrated framework for reconstruction, segmentation and downstream analysis of undersampled cine CMR data. The framework enables active acquisition of radial k-space data, in which acquisition can be stopped as soon as …
ECGFlowCMR: Pretraining with ECG-Generated Cine CMR Helps Cardiac Disease Classification and Phenotype Prediction
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Polytechnic Institute of Zhejiang University, Zhejiang University（浙江大学 polytechnic 院）；Institute of Computing Technology, University of the Chinese Academy of Sciences（中国科学院计算技术研究所）；National Institute of Health Data Science, Peking University（北京大学健康数据科学国家研究院）；Institute of Microelectronics, University of the Chinese Academy of Sciences（中国科学院微电子研究所）；Department of Cardiology, Zhejiang University（浙江大学心内科部）
King’s College London（伦敦国王学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健基金会信托）；University Medical Center Utrecht（乌得勒支大学医学中心）
Current artificial intelligence (AI) algorithms for short-axis cardiac magnetic resonance (CMR) segmentation achieve human performance for slices situated in the middle of the heart. However, an often-overlooked fact is that segmentation of the basal and apical slices is more difficult. During manual analysis, differences in the basal segmentations have been reported as one of the major sources of disagreement in human interobserver variability. In this work, we aim to investigate the performance of AI algorithms in segmenting basal and apical slices and design strategies to improve their segm…
Brigham and Women's Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Bariloche Atomic Center（巴里洛切原子中心）；CONICET（阿根廷国家科学与技术研究委员会）；Instituto Balseiro（巴尔塞罗研究所）；Universidad Nacional de Cuyo（库约国立大学）；Sanatorio San Carlos（圣卡洛斯疗养院）；INTECNUS（INTECNUS研究所）；CNEA – Comisión Nacional de Energía Atómica（阿根廷国家原子能委员会）
Background: Artificial intelligence techniques have demonstrated great potential in cardiology, especially to detect imperceptible patterns for the human eye. In this sense, these techniques seem to be adequate to identify patterns in the myocardial texture which could lead to characterize and quantify fibrosis. Purpose: The aim of this study was to postulate a new artificial intelligence method to identify fibrosis in cine cardiac magnetic resonance (CMR) imaging. Methods: A retrospective observational study was carried out in a population of 75 subjects from a clinical center of San Carlos d…
In the clinical environment, myocardial infarction (MI) as one com-mon cardiovascular disease is mainly evaluated based on the late gadolinium enhancement (LGE) cardiac magnetic resonance images (CMRIs). The auto-matic segmentations of left ventricle (LV), right ventricle (RV), and left ven-tricular myocardium (LVM) in the LGE CMRIs are desired for the aided diag-nosis in clinic. To accomplish this segmentation task, this paper proposes a modified U-net architecture by combining multi-sequence CMRIs, including the cine, LGE, and T2-weighted CMRIs. The cine and T2-weighted CMRIs are used to ass…
Assessment of Deep Learning Segmentation for Real-Time Free-Breathing Cardiac Magnetic Resonance Imaging at Rest and Under Exercise Stress
University Medical Center Göttingen（哥廷根大学医学中心）；German Centre for Cardiovascular Research (DZHK)（德国心血管研究中心）；Clinic of Cardiology and Pneumology, University Medical Center Göttingen（哥廷根大学医学中心心脏病与肺病科）；Graz University of Technology（格拉茨技术大学）；Institute of Biomedical Imaging, Graz University of Technology（格拉茨技术大学生物医学影像研究所）
Learning from Acquisition: Metadata-driven Multimodal Pre-training for Cardiac MRI
Department of Surgery & Cancer（外科与癌症系）；Bioengineering Department and Imperial-X（生物工程系和Imperial-X）；National Heart and Lung Institute（国家心脏和肺研究所）；Cardiovascular Research Centre（心血管研究中心）；School of Biomedical Engineering & Imaging Sciences（生物医学工程与成像科学学院）
An AI tool for automated analysis of large-scale unstructured clinical cine CMR databases
King’s College London（伦敦国王学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯NHS基金会信托）；Duke University（杜克大学）；Barts Health NHS Trust（巴特健康NHS基金会信托）；Alan Turing Institute（阿兰·图灵研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）
Imperial College London（帝国理工学院）；Technical University Munich（慕尼黑工业大学）；University of Edinburgh（爱丁堡大学）；MRC London Institute of Medical Sciences（MRC伦敦医学科学研究所）
Recovering the 3D motion of the heart from cine cardiac magnetic resonance (CMR) imaging enables the assessment of regional myocardial function and is important for understanding and analyzing cardiovascular disease. However, 3D cardiac motion estimation is challenging because the acquired cine CMR images are usually 2D slices which limit the accurate estimation of through-plane motion. To address this problem, we propose a novel multi-view motion estimation network (MulViMotion), which integrates 2D cine CMR images acquired in short-axis and long-axis planes to learn a consistent 3D motion fi…
Purpose: To develop a fully automatic method for extraction and directionality determination of respiratory signal in free-breathing, real-time (RT) cardiac MRI. Methods: The respiratory signal is extracted by a principal component analysis method from RT cine images. Then, a two-step procedure is used to determine the directionality (sign) of the respiratory signal. First, the signal polarity of all slices is made consistent with a reference slice. Second, a global sign correction is performed by maximizing the correlation of the respiratory signal with the zeroth-moment center curve. The pro…
Accelerating Cardiac MRI Reconstruction with CMRatt: An Attention-Driven Approach
ML-Labs Dublin City University（都柏林城市大学ML实验室）；Insight SFI Research Centre for Data Analytics Dublin City University（都柏林城市大学Insight SFI数据分析研究中心）；School of Medicine University College Dublin（都柏林大学学院医学院）
Guy’s and St Thomas’ Hospital NHS Foundation Trust（盖伊和圣托马斯医院NHS基金会信托）；King’s College London（伦敦国王学院）；Imperial College London（帝国理工学院）
In fully sampled cardiac MR (CMR) acquisitions, motion can lead to corruption of k-space lines, which can result in artefacts in the reconstructed images. In this paper, we propose a method to automatically detect and correct motion-related artefacts in CMR acquisitions during reconstruction from k-space data. Our correction method is inspired by work on undersampled CMR reconstruction, and uses deep learning to optimize a data-consistency term for under-sampled k-space reconstruction. Our main methodological contribution is the addition of a detection network to classify motion-corrupted k-sp…
Fuzhou University（福州大学）；University of Oxford（牛津大学）；Fudan University（复旦大学）；Zhongshan Hospital, Fudan University（复旦大学附属中山医院）；Shanghai Medical School, Fudan University（复旦大学上海医学院）；Shanghai Institute of Medical Imaging（上海医学影像研究所）
Myocardial pathology segmentation (MyoPS) is critical for the risk stratification and treatment planning of myocardial infarction (MI). Multi-sequence cardiac magnetic resonance (MS-CMR) images can provide valuable information. For instance, balanced steady-state free precession cine sequences present clear anatomical boundaries, while late gadolinium enhancement and T2-weighted CMR sequences visualize myocardial scar and edema of MI, respectively. Existing methods usually fuse anatomical and pathological information from different CMR sequences for MyoPS, but assume that these images have bee…
A versatile foundation model for cine cardiac magnetic resonance image analysis tasks
UCL Hawkes Institute, Department of Medical Physics and Biomedical Engineering, University College London（伦敦大学霍克斯研究所、医学物理与生物医学工程系、伦敦大学学院）；InstaDeep；Department of Computing, Imperial College London（伦敦帝国学院计算机系）；Department of Brain Sciences, Imperial College London（伦敦帝国学院脑科学系）；Institute of Cardiovascular Sciences, University College London（伦敦大学学院心血管科学研究所）；Barts Heart Centre, Barts Health NHS Trust（巴特斯心脏中心、巴特斯健康 NHS信托）
Fully Automated Assessment of Cardiac Coverage in Cine Cardiovascular Magnetic Resonance Images using an Explainable Deep Visual Salient Region Detection Model
King's College London（伦敦国王学院）；Guy’s and St Thomas’ Hospital（盖伊和圣托马斯医院）
In computer vision there has been significant research interest in assessing potential demographic bias in deep learning models. One of the main causes of such bias is imbalance in the training data. In medical imaging, where the potential impact of bias is arguably much greater, there has been less interest. In medical imaging pipelines, segmentation of structures of interest plays an important role in estimating clinical biomarkers that are subsequently used to inform patient management. Convolutional neural networks (CNNs) are starting to be used to automate this process. We present the fir…
Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins Hospital（约翰斯·霍普金斯医院）
Visualizing disease-induced scarring and fibrosis in the heart on cardiac magnetic resonance (CMR) imaging with contrast enhancement (LGE) is paramount in characterizing disease progression and quantifying pathophysiological substrates of arrhythmias. However, segmentation and scar/fibrosis identification from LGE-CMR is an intensive manual process prone to large inter-observer variability. Here, we present a novel fully-automated anatomically-informed deep learning solution for left ventricle (LV) and scar/fibrosis segmentation and clinical feature extraction from LGE-CMR. The technology invo…
Fudan University（复旦大学）；Imperial College London（帝国理工学院）；University of Oxford（牛津大学）；Zhongshan Hospital Affiliated to Fudan University（复旦大学附属中山医院）
The quality of cardiac magnetic resonance (CMR) imaging is susceptible to respiratory motion artifacts. The model robustness of automated segmentation techniques in face of real-world respiratory motion artifacts is unclear. This manuscript describes the design of extreme cardiac MRI analysis challenge under respiratory motion (CMRxMotion Challenge). The challenge aims to establish a public benchmark dataset to assess the effects of respiratory motion on image quality and examine the robustness of segmentation models. The challenge recruited 40 healthy volunteers to perform different breath-ho…
Imperial College London（帝国理工学院）；National University of Defense Technology（国防科技大学）；MRC London Institute of Medical Sciences（英国医学研究委员会伦敦医学科学研究所）；Technical University Munich（慕尼黑工业大学）
3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and diagnosis of cardiovascular diseases. Most of the previous methods focus on estimating pixel-/voxel-wise motion fields in the full image space, which ignore the fact that motion estimation is mainly relevant and useful within the object of interest, e.g., the heart. In this work, we model the heart as a 3D geometric mesh and propose a novel deep learning-based method that can estimate 3D motion of the heart mesh from 2D short- and long-axis CMR images. By developing a …
National Heart, Lung and Blood Institute, National Institutes of Health（美国国立卫生研究院心肺血液研究所）；Barts Heart Centre, Barts Health NHS Trust（巴特健康国民保健信托巴特心脏中心）；National Amyloidosis Centre, Royal Free Hospital（皇家自由医院国家淀粉样变性病中心）
Purpose: To develop a convolutional neural network (CNN) solution for robust landmark detection in cardiac MR images. Methods: This retrospective study included cine, LGE and T1 mapping scans from two hospitals. The training set included 2,329 patients and 34,019 images. A hold-out test set included 531 patients and 7,723 images. CNN models were developed to detect two mitral valve plane and apical points on long-axis (LAX) images. On short-axis (SAX) images, anterior and posterior RV insertion points and LV center were detected. Model outputs were compared to manual labels by two operators fo…
National University of Singapore（新加坡国立大学）；National University Heart Centre（新加坡国立大学心脏中心）；National University Hospital（新加坡国立大学医院）
Automatic segmentation of the left ventricle (LV) in late gadolinium enhanced (LGE) cardiac MR (CMR) images is difficult due to the intensity heterogeneity arising from accumulation of contrast agent in infarcted myocardium. In this paper, we present a comprehensive framework for automatic 3D segmentation of the LV in LGE CMR images. Given myocardial contours in cine images as a priori knowledge, the framework initially propagates the a priori segmentation from cine to LGE images via 2D translational registration. Two meshes representing respectively endocardial and epicardial surfaces are the…
DeepMesh: Mesh-based Cardiac Motion Tracking using Deep Learning
Imperial College London（帝国理工学院）；University of Birmingham（伯明翰大学）；MRC London Institute of Medical Sciences（英国医学研究理事会伦敦医学科学研究所）；Technical University Munich（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河畔临床教学中心）
本文提出 DeepMesh 学习框架，利用多视图二维 CMR 图像重建个体心脏三维网格并估计逐顶点运动场，在 UK Biobank 上优于现有图像与网格类方法。
Extreme Cardiac MRI Analysis under Respiratory Motion: Results of the CMRxMotion Challenge
Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, Shanghai 200032, China；Shanghai Key Laboratory of MICCAI, Fudan University, Shanghai, Shanghai 200032, China；Department of Electrical；Electronic Engineering \& I-X, Imperial College London, London, London SW7 2AZ, United Kingdom；Department of Radiology, Zhongshan Hospital Affiliated to Fudan University, Shanghai, Shanghai 200032, China；Department of Computing, Imperial College London, London, London SW7 2AZ, United Kingdom；School of Computer Science, University of Sheffield, Sheffield, S1 4DP, United Kingdom；Department of Engineering Science, University of Oxford, Oxford, OX2 0ES, United Kingdom；Shanghai Pudong Hospital；Human Phenome Institute, Fudan University, Shanghai, 201203, China；School of Computer Science, University of Nottingham, Nottingham, NG8 1BB, United Kingdom；Diagnostic Imaging, University of Alberta, Edmonton, AB T6G 1K4, Canada；Department of Computer Science；Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong 000000, China；The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology；Reproduction, Maastricht University, 6220 MD Maastricht, The Netherlands；Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven 5612 AZ, The Netherlands；Department of Radiology, Northwestern University, 737 N. Michigan Ave, Suite 1600, Chicago 60611, United States；United Imaging Research, 393 Middle Huaxia Road, Pudong, Shanghai 201210, China；Division of Image Processing, Department of Radiology, Leiden University Medical Center, PO Box 9600, Leiden 2300 RC, The Netherlands；Université Bourgogne Europe, CNRS, ICMUB UMR 6302, 21000 Dijon, France；Istanbul Technical University, Maslak, 34467, İstanbul, Türkiye；Computer Science, Technical University of Darmstadt, Karolinenpl. 5, 64289 Darmstadt, Germany；Lung Institute, Faculty of Medicine, Imperial College London, Guy Scadding Building, Cale Street, London, SW3 6LY,United Kingdom；Hawkes Institute, Department of Computer Science, University College London, 66-72 Gower St, London, United Kingdom；School of Medicine, University College Dublin, Belfield, Dublin, D04 V1W8, Ireland；CeADAR: Ireland's Centre for AI, University College Dublin, Belfield, Dublin, D04 V1W8, Ireland；Sano Centre for Computational Medicine, Czarnowiejska 36, 30-054, Krakow, Poland；Faculty of Mathematics；Computer Science, Jagiellonian University, S. Łojasiewicza 6, Krakow, Poland；Department of Electronic；Computer Engineering, The Hong Kong University of Science；School of Instrument Science；Engineering, Southeast University, Nanjing, Nanjing 210096, China；College of Artificial Intelligence, Nanjing University of Aeronautics；Academy for Engineering；Technology, Fudan University, Shanghai, Shanghai 200433, China；College of Biomedical Engineering, Fudan University, Shanghai, Shanghai 200433, China；Institute of Science；Technology for Brain-inspired Intelligence, Fudan University, Shanghai, Shanghai 200433, China；Department of Brain Sciences, Imperial College London, London, London SW7 2AZ, United Kingdom；Data Science Institute, Imperial College London, London, London SW7 2AZ, United Kingdom
King's College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）
Cine cardiac magnetic resonance (CMR) has become the gold standard for the non-invasive evaluation of cardiac function. In particular, it allows the accurate quantification of functional parameters including the chamber volumes and ejection fraction. Deep learning has shown the potential to automate the requisite cardiac structure segmentation. However, the lack of robustness of deep learning models has hindered their widespread clinical adoption. Due to differences in the data characteristics, neural networks trained on data from a specific scanner are not guaranteed to generalise well to dat…
