Broadly speaking, the objective in cardiac image segmentation is to delineate the outer and inner walls of the heart to segment out either the entire or parts of the organ boundaries. This paper will focus on MR images as they are the most widely used in cardiac segmentation -- as a result of the accurate morphological information and better soft tissue contrast they provide. This cardiac segmentation information is very useful as it eases physical measurements that provides useful metrics for cardiac diagnosis such as infracted volumes, ventricular volumes, ejection fraction, myocardial mass, cardiac movement, and the like. But, this task is difficult due to the intensity and texture similarities amongst the different cardiac and background structures on top of some noisy artifacts present in MR images. Thus far, various researchers have proposed different techniques to solve some of the pressing issues. This seminar paper presents an overview of representative medical image segmentation techniques. The paper also highlights preferred approaches for segmentation of the four cardiac chambers: the left ventricle (LV), right ventricle (RV), left atrium (LA) and right atrium (RA), on short axis image planes.
Rochester Institute of Technology（罗切斯特理工学院）；National Institute of Allergy and Infectious Diseases（国家过敏和传染病研究所）；MSC LLC.（MSC有限责任公司）
Cardiac image segmentation is a critical process for generating personalized models of the heart and for quantifying cardiac performance parameters. Several convolutional neural network (CNN) architectures have been proposed to segment the heart chambers from cardiac cine MR images. Here we propose a multi-task learning (MTL)-based regularization framework for cardiac MR image segmentation. The network is trained to perform the main task of semantic segmentation, along with a simultaneous, auxiliary task of pixel-wise distance map regression. The proposed distance map regularizer is a decoder …
Rutgers University（罗格斯大学）；Nvidia Corporation（英伟达公司）；New York University（纽约大学）
The 3D volumetric shape of the heart's left ventricle (LV) myocardium (MYO) wall provides important information for diagnosis of cardiac disease and invasive procedure navigation. Many cardiac image segmentation methods have relied on detection of region-of-interest as a pre-requisite for shape segmentation and modeling. With segmentation results, a 3D surface mesh and a corresponding point cloud of the segmented cardiac volume can be reconstructed for further analyses. Although state-of-the-art methods (e.g., U-Net) have achieved decent performance on cardiac image segmentation in terms of ac…
Using Foundation Models as Pseudo-Label Generators for Pre-Clinical 4D Cardiac CT Segmentation
Yale University（耶鲁大学）；University of Pennsylvania（宾夕法尼亚大学）；Washington University School of Medicine in St Louis（华盛顿大学圣路易斯医学学院）；University of South Carolina School of Medicine（南卡罗来纳大学医学院）；University of Colorado Boulder（科罗拉多大学波德分校）
University of Oslo（奥斯陆大学）；GE Vingmed Ultrasound（GE维ingmed超声公司）；GE Research（通用电气研究中心）
Accurate and consistent predictions of echocardiography parameters are important for cardiovascular diagnosis and treatment. In particular, segmentations of the left ventricle can be used to derive ventricular volume, ejection fraction (EF) and other relevant measurements. In this paper we propose a new automated method called EchoGraphs for predicting ejection fraction and segmenting the left ventricle by detecting anatomical keypoints. Models for direct coordinate regression based on Graph Convolutional Networks (GCNs) are used to detect the keypoints. GCNs can learn to represent the cardiac…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）
Current state-of-the-art deep learning segmentation methods have not yet made a broad entrance into the clinical setting in spite of high demand for such automatic methods. One important reason is the lack of reliability caused by models that fail unnoticed and often locally produce anatomically implausible results that medical experts would not make. This paper presents an automatic image segmentation method based on (Bayesian) dilated convolutional networks (DCNN) that generate segmentation masks and spatial uncertainty maps for the input image at hand. The method was trained and evaluated u…
Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images
Xi’an Jiaotong University（西安交通大学）；National Engineering Laboratory for Big Data Algorithm and Analysis Technology（大数据算法与分析技术国家工程实验室）；Shanghai Jiaotong University（上海交通大学）
多图谱分割方法是生物医学领域中应用最广泛的图像分割技术之一。在这类方法中存在两个主要挑战，即，图谱选择和标签融合。在本文中，我们提出了一种新的多图谱分割方法，该方法在深度学习框架中制定多图谱分割，以更好地解决这些挑战。所提出的方法，被称为深度融合网络（DFN），是一种深度架构，它集成了一个特征提取子网和一个非本地的基于补丁的标签融合（NL-PLF）子网在一个单一的网络。通过端到端训练来学习网络参数，以自动学习深度特征，从而在NL-PLF框架中实现最佳性能。所学习的深度特征被进一步用于定义用于图谱选择的相似性度量。通过对SATA-13和LV-09两个公共心脏MR数据集的左心室分割评估，我们的方法在SATA-13数据集上的平均Dice度量（ADM）达到0.833，在LV-09数据集上的心外膜分割ADM达到0.95，与其他自动左心室分割方法相比，我们的方法是有利的。我们还在MICCAI 2013 SATA Segmentation Challenge的Cardiac Atlas Project（CAP）测试集上测试了我们的方法，我们的方法在ADM中达到了0.815，在撰写本文时排名最高。
Due to the imbalanced and limited data, semi-supervised medical image segmentation methods often fail to produce superior performance for some specific tailed classes. Inadequate training for those particular classes could introduce more noise to the generated pseudo labels, affecting overall learning. To alleviate this shortcoming and identify the under-performing classes, we propose maintaining a confidence array that records class-wise performance during training. A fuzzy fusion of these confidence scores is proposed to adaptively prioritize individual confidence metrics in every sample rat…
NVIDIA（英伟达）；MGH and BWH Center for Clinical Data Science（麻省总医院与布列根和妇女医院临床数据科学中心）
We propose a 4D convolutional neural network (CNN) for the segmentation of retrospective ECG-gated cardiac CT, a series of single-channel volumetric data over time. While only a small subset of volumes in the temporal sequence is annotated, we define a sparse loss function on available labels to allow the network to leverage unlabeled images during training and generate a fully segmented sequence. We investigate the accuracy of the proposed 4D network to predict temporally consistent segmentations and compare with traditional 3D segmentation approaches. We demonstrate the feasibility of the 4D…
Convolutional Feature Noise Reduction for 2D Cardiac MR Image Segmentation
College of Computer Science, Sichuan Normal University（四川师范大学计算机科学学院）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机科学与人工智能学院）；Visual Computing and Virtual Reality Key Laboratory of Sichuan Province（四川省视觉计算与虚拟现实重点实验室）
Compositional Segmentation of Cardiac Images Leveraging Metadata
School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）；Queen Mary’s Digital Environment Research Institute (DERI)（玛丽女王数字环境研究所）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；School of Engineering and Materials Science, Queen Mary University of London（伦敦玛丽女王大学工程与材料科学学院）
Imperial College London（伦敦帝国理工学院）；Queen Mary University of London（伦敦玛丽女王大学）
In the recent years, convolutional neural networks have transformed the field of medical image analysis due to their capacity to learn discriminative image features for a variety of classification and regression tasks. However, successfully learning these features requires a large amount of manually annotated data, which is expensive to acquire and limited by the available resources of expert image analysts. Therefore, unsupervised, weakly-supervised and self-supervised feature learning techniques receive a lot of attention, which aim to utilise the vast amount of available data, while at the …
Biomedical Image Analysis Group, Department of Computing, Imperial College London（帝国理工学院计算机系生物医学图像分析组）；Division of Brain Sciences, Dept. of Medicine, Imperial College London（帝国理工学院医学部脑科学系）；GlaxoSmithKline Research and Development（葛兰素史克研发部）；William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London（伦敦玛丽女王大学威廉·哈维研究所（NIHR巴茨生物医学研究中心））；Barts Heart Centre, Barts Health NHS Trust（巴茨健康国民医疗服务信托基金巴茨心脏中心）；Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford（牛津大学拉德克利夫医学部心血管医学系）；Department of Radiology, Severance Hospital, Yonsei University College of Medicine（延世大学医学院塞弗伦斯医院放射科）；UK Dementia Research Institute, Imperial College London（帝国理工学院英国痴呆症研究所）
Background: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools, e.g. image segmentation methods, are employed to derive quantitative measures or biomarkers for later analyses. Manual inspection and visual QC of each segmentation isn't feasible at large scale. However, it's important to be able to automatically detect when a segmentation method fails so as to avoid inclusion of wrong measurements into subsequent analyses which could lead to incorrect conclusions. Met…
School of Engineering, University of Edinburgh（爱丁堡大学工程学院）；Canon Medical Research Europe Ltd.（佳能医疗欧洲研究有限公司）；The Alan Turing Institute（艾伦·图灵研究所）
Robust cardiac image segmentation is still an open challenge due to the inability of the existing methods to achieve satisfactory performance on unseen data of different domains. Since the acquisition and annotation of medical data are costly and time-consuming, recent work focuses on domain adaptation and generalization to bridge the gap between data from different populations and scanners. In this paper, we propose two data augmentation methods that focus on improving the domain adaptation and generalization abilities of state-to-the-art cardiac segmentation models. In particular, our "Resol…
Crop and Couple: cardiac image segmentation using interlinked specialist networks
Queen Mary University of London（伦敦玛丽女王大学）；Queen Mary's Digital Environment Research Institute (DERI)（伦敦玛丽女王大学数字环境研究所）；University College London（伦敦大学学院）
Rutgers University（罗格斯大学）；SenseBrain Research（SenseBrain研究院）；University of Arkansas for Medical Sciences（阿肯色大学医学院）；Lund University（隆德大学）；New York University（纽约大学）
Joint 2D cardiac segmentation and 3D volume reconstruction are fundamental to building statistical cardiac anatomy models and understanding functional mechanisms from motion patterns. However, due to the low through-plane resolution of cine MR and high inter-subject variance, accurately segmenting cardiac images and reconstructing the 3D volume are challenging. In this study, we propose an end-to-end latent-space-based framework, DeepRecon, that generates multiple clinically essential outcomes, including accurate image segmentation, synthetic high-resolution 3D image, and 3D reconstructed volu…
Deep learning methods have reached state-of-the-art performance in cardiac image segmentation. Currently, the main bottleneck towards their effective translation into clinics requires assuring continuous high model performance and segmentation results. In this work, we present a novel learning framework to monitor the performance of heart segmentation models in the absence of ground truth. Formulated as an anomaly detection problem, the monitoring framework allows deriving surrogate quality measures for a segmentation and allows flagging suspicious results. We propose two different types of qu…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Lenovo Research（联想研究院）；Lenovo Ltd.（联想有限公司）
Accurate segmentation of cardiac structures can assist doctors to diagnose diseases, and to improve treatment planning, which is highly demanded in the clinical practice. However, the shortage of annotation and the variance of the data among different vendors and medical centers restrict the performance of advanced deep learning methods. In this work, we present a fully automatic method to segment cardiac structures including the left (LV) and right ventricle (RV) blood pools, as well as for the left ventricular myocardium (MYO) in MRI volumes. Specifically, we design a semi-supervised learnin…
Imperial College London（伦敦帝国学院）；City, University of London（伦敦城市大学）；University of Birmingham（伯明翰大学）
Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the chall…
Semantic segmentation using convolutional neural networks (CNNs) is the state-of-the-art for many medical image segmentation tasks including myocardial segmentation in cardiac MR images. However, the predicted segmentation maps obtained from such standard CNN do not allow direct quantification of regional shape properties such as regional wall thickness. Furthermore, the CNNs lack explicit shape constraints, occasionally resulting in unrealistic segmentations. In this paper, we use a CNN to predict shape parameters of an underlying statistical shape model of the myocardium learned from a train…
LGE CMR is an efficient technology for detecting infarcted myocardium. An efficient and objective ventricle segmentation method in LGE can benefit the location of the infarcted myocardium. In this paper, we proposed an automatic framework for LGE image segmentation. There are just 5 labeled LGE volumes with about 15 slices of each volume. We adopted histogram match, an invariant of rotation registration method, on the other labeled modalities to achieve effective augmentation of the training data. A CNN segmentation model was trained based on the augmented training data by leave-one-out strate…
Semantic segmentation using convolutional neural networks (CNNs) is the state-of-the-art for many medical segmentation tasks including left ventricle (LV) segmentation in cardiac MR images. However, a drawback is that these CNNs lack explicit shape constraints, occasionally resulting in unrealistic segmentations. In this paper, we perform LV and myocardial segmentation by regression of pose and shape parameters derived from a statistical shape model. The integrated shape model regularizes predicted segmentations and guarantees realistic shapes. Furthermore, in contrast to semantic segmentation…
King's College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）
Cine cardiac magnetic resonance (CMR) has become the gold standard for the non-invasive evaluation of cardiac function. In particular, it allows the accurate quantification of functional parameters including the chamber volumes and ejection fraction. Deep learning has shown the potential to automate the requisite cardiac structure segmentation. However, the lack of robustness of deep learning models has hindered their widespread clinical adoption. Due to differences in the data characteristics, neural networks trained on data from a specific scanner are not guaranteed to generalise well to dat…
Multi-Disease-Aware Training Strategy for Cardiac MR Image Segmentation
College of Computer Science, Sichuan Normal University（四川师范大学计算机科学学院）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算与人工智能学院）；West China Hospital, Sichuan University（四川大学华西医院）
Gaze-Assisted Human-Centric Domain Adaptation for Cardiac Ultrasound Image Segmentation
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Medical Faculty Heidelberg, Heidelberg University（海德堡大学海德堡医学院）
Cardiac magnetic resonance imaging (cMRI) is an integral part of diagnosis in many heart related diseases. Recently, deep neural networks have demonstrated successful automatic segmentation, thus alleviating the burden of time-consuming manual contouring of cardiac structures. Moreover, frameworks such as nnU-Net provide entirely automatic model configuration to unseen datasets enabling out-of-the-box application even by non-experts. However, current studies commonly neglect the clinically realistic scenario, in which a trained network is applied to data from a different domain such as deviati…
In this work, we implement a fully convolutional segmenter featuring both a learned group structure and a regularized weight-pruner to reduce the high computational cost in volumetric image segmentation. We validated our framework on the ACDC dataset featuring one healthy and four pathology groups imaged throughout the cardiac cycle. Our technique achieved Dice scores of 96.8% (LV blood-pool), 93.3% (RV blood-pool) and 90.0% (LV Myocardium) with five-fold cross-validation and yielded similar clinical parameters as those estimated from the ground truth segmentation data. Based on these results,…
Medical image segmentation has significantly benefitted thanks to deep learning architectures. Furthermore, semi-supervised learning (SSL) has recently been a growing trend for improving a model's overall performance by leveraging abundant unlabeled data. Moreover, learning multiple tasks within the same model further improves model generalizability. To generate smoother and accurate segmentation masks from 3D cardiac MR images, we present a Multi-task Cross-task learning consistency approach to enforce the correlation between the pixel-level (segmentation) and the geometric-level (distance ma…
Bursa Technical University（布尔萨技术大学）；Imperial College London（伦敦帝国学院）
Recently, deep networks have shown impressive performance for the segmentation of cardiac Magnetic Resonance Imaging (MRI) images. However, their achievement is proving slow to transition to widespread use in medical clinics because of robustness issues leading to low trust of clinicians to their results. Predicting run-time quality of segmentation masks can be useful to warn clinicians against poor results. Despite its importance, there are few studies on this problem. To address this gap, we propose a quality control method based on the agreement across decoders of a multi-view network, TMS-…
Sano Centre for Computational Medicine（萨诺计算医学中心）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Cardiac Magnetic Resonance Imaging is commonly used for the assessment of the cardiac anatomy and function. The delineations of left and right ventricle blood pools and left ventricular myocardium are important for the diagnosis of cardiac diseases. Unfortunately, the movement of a patient during the CMR acquisition procedure may result in motion artifacts appearing in the final image. Such artifacts decrease the diagnostic quality of CMR images and force redoing of the procedure. In this paper, we present a Multi-task Swin UNEt TRansformer network for simultaneous solving of two tasks in the …
Imperial College London（帝国理工学院）；MRC London Institute of Medical Sciences（MRC伦敦医学科学研究所）；National Heart and Lung Institute（国家心肺研究所）
Deep learning approaches have achieved state-of-the-art performance in cardiac magnetic resonance (CMR) image segmentation. However, most approaches have focused on learning image intensity features for segmentation, whereas the incorporation of anatomical shape priors has received less attention. In this paper, we combine a multi-task deep learning approach with atlas propagation to develop a shape-constrained bi-ventricular segmentation pipeline for short-axis CMR volumetric images. The pipeline first employs a fully convolutional network (FCN) that learns segmentation and landmark localisat…
Nanjing University of Science and Technology（南京理工大学）
Convolutional Neural Networks (CNNs) have achieved high accuracy for cardiac structure segmentation if training cases and testing cases are from the same distribution. However, the performance would be degraded if the testing cases are from a distinct domain (e.g., new MRI scanners, clinical centers). In this paper, we propose a histogram matching (HM) data augmentation method to eliminate the domain gap. Specifically, our method generates new training cases by using HM to transfer the intensity distribution of testing cases to existing training cases. The proposed method is quite simple and c…
Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）
With large-scale well-labeled datasets, deep learning has shown significant success in medical image segmentation. However, it is challenging to acquire abundant annotations in clinical practice due to extensive expertise requirements and costly labeling efforts. Recently, contrastive learning has shown a strong capacity for visual representation learning on unlabeled data, achieving impressive performance rivaling supervised learning in many domains. In this work, we propose a novel multi-scale multi-view global-local contrastive learning (MMGL) framework to thoroughly explore global and loca…
Deep models often suffer from severe performance drop due to the appearance shift in the real clinical setting. Most of the existing learning-based methods rely on images from multiple sites/vendors or even corresponding labels. However, collecting enough unknown data to robustly model segmentation cannot always hold since the complex appearance shift caused by imaging factors in daily application. In this paper, we propose a novel style-invariant method for cardiac image segmentation. Based on the zero-shot style transfer to remove appearance shift and test-time augmentation to explore divers…
In this work, we attempt the segmentation of cardiac structures in late gadolinium-enhanced (LGE) magnetic resonance images (MRI) using only minimal supervision in a two-step approach. In the first step, we register a small set of five LGE cardiac magnetic resonance (CMR) images with ground truth labels to a set of 40 target LGE CMR images without annotation. Each manually annotated ground truth provides labels of the myocardium and the left ventricle (LV) and right ventricle (RV) cavities, which are used as atlases. After multi-atlas label fusion by majority voting, we possess noisy labels fo…
Imperial College London（帝国理工学院）；Queen Mary University of London（伦敦玛丽女王大学）
Cardiac MR image segmentation is essential for the morphological and functional analysis of the heart. Inspired by how experienced clinicians assess the cardiac morphology and function across multiple standard views (i.e. long- and short-axis views), we propose a novel approach which learns anatomical shape priors across different 2D standard views and leverages these priors to segment the left ventricular (LV) myocardium from short-axis MR image stacks. The proposed segmentation method has the advantage of being a 2D network but at the same time incorporates spatial context from multiple, com…
With the advent of Cardiac Cine Magnetic Resonance (CMR) Imaging, there has been a paradigm shift in medical technology, thanks to its capability of imaging different structures within the heart without ionizing radiation. However, it is very challenging to conduct pre-operative planning of minimally invasive cardiac procedures without accurate segmentation and identification of the left ventricle (LV), right ventricle (RV) blood-pool, and LV-myocardium. Manual segmentation of those structures, nevertheless, is time-consuming and often prone to error and biased outcomes. Hence, automatic and c…
Istanbul Technical University（伊斯坦布尔理工大学）；King’s College London（伦敦国王学院）
Segmenting anatomical structures in medical images has been successfully addressed with deep learning methods for a range of applications. However, this success is heavily dependent on the quality of the image that is being segmented. A commonly neglected point in the medical image analysis community is the vast amount of clinical images that have severe image artefacts due to organ motion, movement of the patient and/or image acquisition related issues. In this paper, we discuss the implications of image motion artefacts on cardiac MR segmentation and compare a variety of approaches for joint…
Unsupervised domain adaptation (UDA) methods intend to reduce the gap between source and target domains by using unlabeled target domain and labeled source domain data, however, in the medical domain, target domain data may not always be easily available, and acquiring new samples is generally time-consuming. This restricts the development of UDA methods for new domains. In this paper, we explore the potential of UDA in a more challenging while realistic scenario where only one unlabeled target patient sample is available. We call it Few-shot Unsupervised Domain adaptation (FUDA). We first gen…
