LU-Net: a multi-task network to improve the robustness of segmentation of left ventriclular structures by deep learning in 2D echocardiography
Univ Lyon（里昂大学）；INSA-Lyon（里昂国立应用科学学院）；Université Claude Bernard Lyon 1（里昂第一大学克洛德·贝尔纳大学）；UJM-Saint Etienne（圣艾蒂安大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Norwegian University of Science and Technology（挪威科技大学）；Centre Hospitalier de Saint-Etienne（圣艾蒂安中心医院）；St. Olavs Hospital（圣奥拉夫医院）；University of Sherbrooke（舍布鲁克大学）
Sarah Leclerc, Erik Smistad, Andreas Østvik, Frederic Cervenansky, Florian Espinosa, Torvald Espeland, Erik Andreas Rye Berg, Thomas Grenier, Carole Lartizien, … 展开作者
Sarah Leclerc, Erik Smistad, Andreas Østvik, Frederic Cervenansky, Florian Espinosa, Torvald Espeland, Erik Andreas Rye Berg, Thomas Grenier, Carole Lartizien, Pierre-Marc Jodoin, Lasse Lovstakken, Olivier Bernard
Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semi-automatically in clinical routine, and is thus prone to inter- and intra-observer variability. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a lack of robustness. In this work, we introduce an end-to-end multi-task network designed to improve the overall accuracy of cardiac segmentation while enhancing the estimation of clinical indices and reducing the number of outliers. Results obtained on a large open access dataset show that our method outperforms the current best performing deep learning solution and achieved an overall segmentation accuracy lower than the intra-observer variability for the epicardial border (i.e. on average a mean absolute error of 1.5mm and a Hausdorff distance of 5.1mm) with 11% of outliers. Moreover, we demonstrate that our method can closely reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.96 and a mean absolute error of 7.6ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.83 and an absolute mean error of 5.0%, producing scores that are slightly below the intra-observer margin. Based on this observation, areas for improvement are suggested.
Following the successful application of the U-Net to medical images, there have been different encoder-decoder models proposed as an improvement to the original U-Net for segmenting echocardiographic images. This study aims to examine the performance of the state-of-the-art proposed models as well as the original U-Net model by applying them to segment the endocardium of the Left Ventricle in 2D automatically. The prediction outputs of the models are used to evaluate the performance of the models by comparing the automated results against the expert annotations (gold standard). Our results rev…
University of Alberta（阿尔伯塔大学）；Mazankowski Alberta Heart Institute（马赞科夫斯基阿尔伯塔心脏研究所）
Purpose: Echocardiography is commonly used as a non-invasive imaging tool in clinical practice for the assessment of cardiac function. However, delineation of the left ventricle is challenging due to the inherent properties of ultrasound imaging, such as the presence of speckle noise and the low signal-to-noise ratio. Methods: We propose a semi-automated segmentation algorithm for the delineation of the left ventricle in temporal 3D echocardiography sequences. The method requires minimal user interaction and relies on a diffeomorphic registration approach. Advantages of the method include no d…
Ultromics Ltd（Ultromics有限公司）；King’s College London（伦敦国王学院）；Sherbrooke University（舍布鲁克大学）
We propose a new method to automatically contour the left ventricle on 2D echocardiographic images. Unlike most existing segmentation methods, which are based on predicting segmentation masks, we focus at predicting the endocardial contour and the key landmark points within this contour (basal points and apex). This provides a representation that is closer to how experts perform manual annotations and hence produce results that are physiologically more plausible. Our proposed method uses a two-headed network based on the U-Net architecture. One head predicts the 7 contour points, and the other…
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Contrastive learning has proven useful in many applications where access to labelled data is limited. The lack of annotated data is particularly problematic in medical image segmentation as it is difficult to have clinical experts manually annotate large volumes of data such as cardiac structures in ultrasound images of the heart. In this paper, We propose a self supervised contrastive learning method to segment the left ventricle from echocardiography when limited annotated images exist. Furthermore, we study the effect of contrastive pretraining on two well-known segmentation networks, UNet …
University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）
Transthoracic echo is one of the most common means of cardiac studies in the clinical routines. During the echo exam, the sonographer captures a set of standard cross sections (echo views) of the heart. Each 2D echo view cuts through the 3D cardiac geometry via a unique plane. Consequently, different views share some limited information. In this work, we investigate the feasibility of generating a 2D echo view using another view based on adversarial generative models. The objective optimized to train the view-conversion model is based on the ideas introduced by LSGAN, PatchGAN and Conditional …
Bayesian Optimization (BO) is a well-studied hyperparameter tuning technique that is more efficient than grid search for high-cost, high-parameter machine learning problems. Echocardiography is a ubiquitous modality for evaluating heart structure and function in cardiology. In this work, we use BO to optimize the architectural and training-related hyperparameters of a previously published deep fully convolutional neural network model for multi-structure segmentation in echocardiography. In a fair comparison, the resulting model outperforms this recent state-of-the-art on the annotated CAMUS da…
The University of Winnipeg（温尼伯大学）；Qatar University（卡塔尔大学）；Hamad Medical Corporation（哈马德医疗公司）
Cardiac imaging known as echocardiography is a non-invasive tool utilized to produce data including images and videos, which cardiologists use to diagnose cardiac abnormalities in general and myocardial infarction (MI) in particular. Echocardiography machines can deliver abundant amounts of data that need to be quickly analyzed by cardiologists to help them make a diagnosis and treat cardiac conditions. However, the acquired data quality varies depending on the acquisition conditions and the patient's responsiveness to the setup instructions. These constraints are challenging to doctors especi…
Med Fanavarn Plus Co.（梅德法纳瓦尔恩普拉斯公司）；Sharif University of Technology（谢里夫理工大学）；Rajaie Cardiovascular Medical and Research Center（拉贾伊心血管医疗与研究中心）；Iran University of Medical Science（伊朗医科大学）；Geneva University Hospital（日内瓦大学医院）；The University of British Columbia（不列颠哥伦比亚大学）；KU Leuven（鲁汶大学）
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed this problem through an encoder-decoder path. Despite outstanding overall performance, U-net ignores the contribution of all semantic strengths in the segmentation procedure. In the present study, we h…
ETH Zurich（苏黎世联邦理工学院）；University Hospital Zurich（苏黎世大学医院）
The segmentation of the mitral valve annulus and leaflets specifies a crucial first step to establish a machine learning pipeline that can support physicians in performing multiple tasks, e.g.\ diagnosis of mitral valve diseases, surgical planning, and intraoperative procedures. Current methods for mitral valve segmentation on 2D echocardiography videos require extensive interaction with annotators and perform poorly on low-quality and noisy videos. We propose an automated and unsupervised method for the mitral valve segmentation based on a low dimensional embedding of the echocardiography vid…
The University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）
Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecting cardiovascular disease at point-of-care. Echo imaging quality and acquisition time is highly dependent on the operator's experience level. Recent developments have shown the possibility of automating echo image quality quantification by mapping an expert's assessment of qualit…
Tampere University（坦佩雷大学）；Qatar University（卡塔尔大学）；Hamad Medical Corporation（哈马德医疗公司）
Myocardial infarction (MI), or commonly known as heart attack, is a life-threatening health problem worldwide from which 32.4 million people suffer each year. Early diagnosis and treatment of MI are crucial to prevent further heart tissue damages or death. The earliest and most reliable sign of ischemia is regional wall motion abnormality (RWMA) of the affected part of the ventricular muscle. Echocardiography can easily, inexpensively, and non-invasively exhibit the RWMA. In this article, we introduce a three-phase approach for early MI detection in low-quality echocardiography: 1) segmentatio…
King's College London（伦敦国王学院）；Guy’s and St Thomas' Hospital（盖伊和圣托马斯医院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Royal Brompton Hospital (Guy’s and St Thomas' NHS Foundation Trust)（皇家布朗普顿医院（盖伊和圣托马斯NHS基金会信托））
Left ventricular (LV) function is an important factor in terms of patient management, outcome, and long-term survival of patients with heart disease. The most recently published clinical guidelines for heart failure recognise that over reliance on only one measure of cardiac function (LV ejection fraction) as a diagnostic and treatment stratification biomarker is suboptimal. Recent advances in AI-based echocardiography analysis have shown excellent results on automated estimation of LV volumes and LV ejection fraction. However, from time-varying 2-D echocardiography acquisition, a richer descr…
A Unified Approach for Comprehensive Analysis of Various Spectral and Tissue Doppler Echocardiography
Ontact Health（Ontact健康公司）；Yonsei University College of Medicine（延世大学医学院）；CONNECT-AI Research Center, Yonsei University College of Medicine（延世大学医学院CONNECT-AI研究中心）；Seoul National University Bundang Hospital（首尔国立大学盆唐医院）；Severance Cardiovascular Hospital, Yonsei University Health System（延世大学医疗系统世福兰斯心血管医院）
Echocardiography to Cardiac MRI View Transformation for Real-Time Blind Restoration
Tampere University（坦佩雷大学）；Qatar University（卡塔尔大学）；VTT Technical Research Centre of Finland（芬兰国家技术研究中心）；Hamad Medical Corporation Hospital（哈马德医疗公司医院）；Hayatabad Medical Complex（海亚特阿巴德医疗中心）
Delineation of the cardiac structures from 2D echocardiographic images is a common clinical task to establish a diagnosis. Over the past decades, the automation of this task has been the subject of intense research. In this paper, we evaluate how far the state-of-the-art encoder-decoder deep convolutional neural network methods can go at assessing 2D echocardiographic images, i.e segmenting cardiac structures as well as estimating clinical indices, on a dataset especially designed to answer this objective. We therefore introduce the Cardiac Acquisitions for Multi-structure Ultrasound Segmentat…
We recently published a deep learning study on the potential of encoder-decoder networks for the segmentation of the 2D CAMUS ultrasound dataset. We propose in this abstract an extension of the evaluation criteria to anatomical assessment, as traditional geometric and clinical metrics in cardiac segmentation do not take into account the anatomical correctness of the predicted shapes. The completed study sheds a new light on the ranking of models.
Phase Unwrapping of Color Doppler Echocardiography using Deep Learning
CREATIS（CREATIS实验室）；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；INSA Lyon（里昂国立应用科学学院）；University of Lyon 1（里昂第一大学）；Institut Universitaire de France（法兰西大学研究院）
Automatic Cardiac Pathology Recognition in Echocardiography Images Using Higher Order Dynamic Mode Decomposition and a Vision Transformer for Small Datasets
Universidad Politécnica de Madrid（马德里理工大学）；Centro Nacional de Investigaciones Cardiovasculares (CNIC)（国家心血管研究中心）；Universidad Complutense de Madrid（马德里康普顿斯大学）；Center for Computational Simulation (CCS)（计算仿真中心）
Compute-Optimal Network Design for Echocardiography Myocardial Segmentation and Perfusion Quantification using Neural Scaling Laws
Department of Bioengineering, Imperial College London（生物工程系，帝国理工学院伦敦分校）；National Heart and Lung Institute, Imperial College London（国家心脏和肺 institute，帝国理工学院伦敦分校）；Guy’s and St. Thomas’ NHS Foundation Trust（圣泰莫斯国家健康服务信托基金）
MSV-Mamba: A Multiscale Vision Mamba Network for Echocardiography Segmentation
School of Computer and Information Engineering, Shanghai Polytechnic University（上海第二工业大学计算机与信息工程学院）；The First Affiliated Hospital of Jinan University（暨南大学附属第一医院）；Fudan University Huashan Hospital（复旦大学附属华山医院）
Peng Cheng Laboratory（鹏城实验室）；Beihang University（北京航空航天大学）；Capital Medical University（首都医科大学）；Beijing Anzhen Hospital（北京安贞医院）；Beijing Advanced Innovation Center for Big Data-Based Precision Medicine（北京大数据精准医疗先进创新中心）
As an important scan plane, four chamber view is routinely performed in both second trimester perinatal screening and fetal echocardiographic examinations. The biometrics in this plane including cardio-thoracic ratio (CTR) and cardiac axis are usually measured by sonographers for diagnosing congenital heart disease. However, due to the commonly existing artifacts like acoustic shadowing, the traditional manual measurements not only suffer from the low efficiency, but also with the inconsistent results depending on the operators' skills. In this paper, we present an anchor-free ellipse detectio…
We present a novel multimodal deep learning framework for cardiac resynchronisation therapy (CRT) response prediction from 2D echocardiography and cardiac magnetic resonance (CMR) data. The proposed method first uses the `nnU-Net' segmentation model to extract segmentations of the heart over the full cardiac cycle from the two modalities. Next, a multimodal deep learning classifier is used for CRT response prediction, which combines the latent spaces of the segmentation models of the two modalities. At inference time, this framework can be used with 2D echocardiography data only, whilst taking…
GHOST-CAT: An Efficient and Practical Network for Mesh Generation from 3D Echocardiography
Auckland Bioengineering Institute, University of Auckland（奥克兰大学生物工程研究所）；Faculty of Informatics, Telkom University（印尼电信大学信息学院）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与成像科学系）；Department of Engineering Science and Biomedical Engineering, University of Auckland（奥克兰大学生命科学工程系）
University of Notre Dame（圣母大学）；Boston Children’s Hospital, Harvard Medical School（哈佛医学院波士顿儿童医院）；Guangdong Cardiovascular Institute, Guangdong Provincial People’s Hospital（广东省人民医院广东省心血管病研究所）；University of Pittsburgh（匹兹堡大学）
Automatic myocardial segmentation of contrast echocardiography has shown great potential in the quantification of myocardial perfusion parameters. Segmentation quality control is an important step to ensure the accuracy of segmentation results for quality research as well as its clinical application. Usually, the segmentation quality control happens after the data acquisition. At the data acquisition time, the operator could not know the quality of the segmentation results. On-the-fly segmentation quality control could help the operator to adjust the ultrasound probe or retake data if the qual…
Echocardiography (echo) is an indispensable tool in a cardiologist's diagnostic armamentarium. To date, almost all echocardiographic parameters require time-consuming manual labeling and measurements by an experienced echocardiographer and exhibit significant variability, owing to the noisy and artifact-laden nature of echo images. For example, mitral inflow (MI) Doppler is used to assess left ventricular (LV) diastolic function, which is of paramount clinical importance to distinguish between different cardiac diseases. In the current work we present a fully automated workflow which leverages…
A Semi-Supervised Approach with Error Reflection for Echocardiography Segmentation
Shanghai University（上海大学）；Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）
Extraction of volumetric indices from echocardiography: which deep learning solution for clinical use?
Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安让·莫内大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；University of Sherbrooke（舍布鲁克大学）；Hospices Civils de Lyon（里昂公立医院集团）
Echocardiography (echo) is a common means of evaluating cardiac conditions. Due to the label scarcity, semi-supervised paradigms in automated echo analysis are getting traction. One of the most sought-after problems in echo is the segmentation of cardiac structures (e.g. chambers). Accordingly, we propose an echocardiogram generation approach using generative adversarial networks with a conditional patch-based discriminator. In this work, we validate the feasibility of GAN-enhanced echo generation with different conditions (segmentation masks), namely, the left ventricle, ventricular myocardiu…
University of Sherbrooke（舍布鲁克大学）；Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Institut Universitaire de France（法国大学研究院）
Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axi…
Standard views in two-dimensional echocardiography are well established but the quality of acquired images are highly dependent on operator skills and are assessed subjectively. This study is aimed at providing an objective assessment pipeline for echocardiogram image quality by defining a new set of domain-specific quality indicators. Consequently, image quality assessment can thus be automated to enhance clinical measurements, interpretation, and real-time optimization. We have developed deep neural networks for the automated assessment of echocardiographic frame which were randomly sampled …
IBM Research – Almaden Research Center（IBM阿尔马登研究中心）；Eastern Virginia Medical School（东弗吉尼亚医学院）
Congenital heart disease (CHD) is the most common congenital abnormality associated with birth defects in the United States. Despite training efforts and substantial advancement in ultrasound technology over the past years, CHD remains an abnormality that is frequently missed during prenatal ultrasonography. Therefore, computer-aided detection of CHD can play a critical role in prenatal care by improving screening and diagnosis. Since many CHDs involve structural abnormalities, automatic segmentation of anatomical structures is an important step in the analysis of fetal echocardiograms. While …
A Novel Data Augmentation Tool for Enhancing Machine Learning Classification: A New Application of the Higher Order Dynamic Mode Decomposition for Improved Cardiac Disease Identification
Universidad Politécnica de Madrid（马德里理工大学）；ETSI Aeronáutica y del Espacio（航空航天高级技术工程学院）；ETSI Telecomunicación（电信高级技术工程学院）；Universidad Complutense de Madrid（马德里康普顿斯大学）；Centro Nacional de Investigaciones Cardiovasculares (CNIC)（国家心血管研究中心）；Center for Computational Simulation (CCS)（计算仿真中心）
GE Vingmed Ultrasound, GE Healthcare（通用电气医疗集团GE Vingmed超声部门）；University of Oslo（奥斯陆大学）；Norwegian Computing Center（挪威计算中心）
Two-dimensional echocardiography (2DE) measurements of left ventricle (LV) dimensions are highly significant markers of several cardiovascular diseases. These measurements are often used in clinical care despite suffering from large variability between observers. This variability is due to the challenging nature of accurately finding the correct temporal and spatial location of measurement endpoints in ultrasound images. These images often contain fuzzy boundaries and varying reflection patterns between frames. In this work, we present a convolutional neural network (CNN) based approach to aut…
NVIDIA（英伟达）；Lenovo Research（联想研究院）；Piedmont Heart Institute（皮埃蒙特心脏研究所）；Tencent Medical AI Lab（腾讯医疗人工智能实验室）；University of California, Irvine（加州大学欧文分校）
Echocardiography has become routinely used in the diagnosis of cardiomyopathy and abnormal cardiac blood flow. However, manually measuring myocardial motion and cardiac blood flow from echocardiogram is time-consuming and error-prone. Computer algorithms that can automatically track and quantify myocardial motion and cardiac blood flow are highly sought after, but have not been very successful due to noise and high variability of echocardiography. In this work, we propose a neural multi-scale self-supervised registration (NMSR) method for automated myocardial and cardiac blood flow dense track…
University of West London（西伦敦大学）；Imperial College（帝国理工学院）
Echocardiography image quality assessment is not a trivial issue in transthoracic examination. As the in vivo examination of heart structures gained prominence in cardiac diagnosis, it has been affirmed that accurate diagnosis of the left ventricle functions is hugely dependent on the quality of echo images. Up till now, visual assessment of echo images is highly subjective and requires specific definition under clinical pathologies. While poor-quality images impair quantifications and diagnosis, the inherent variations in echocardiographic image quality standards indicates the complexity face…
IMC-PINN-FE: A Physics-Informed Neural Network for Patient-Specific Left Ventricular Finite Element Modeling with Image Motion Consistency and Biomechanical Parameter Estimation
Department of Bioengineering, Imperial College London（帝国理工学院生物工程系）
Daegu Gyeongbuk Institute of Science & Technology（大邱庆北科学技术院）；Yonsei University College of Medicine（延世大学医学院）
Background and Objective: Incomplete Kawasaki disease (KD) has often been misdiagnosed due to a lack of the clinical manifestations of classic KD. However, it is associated with a markedly higher prevalence of coronary artery lesions. Identifying coronary artery lesions by echocardiography is important for the timely diagnosis of and favorable outcomes in KD. Moreover, similar to KD, coronavirus disease 2019, currently causing a worldwide pandemic, also manifests with fever; therefore, it is crucial at this moment that KD should be distinguished clearly among the febrile diseases in children. …
Interpretable and backpropagation-free Green Learning for efficient multi-task echocardiographic segmentation and classification
Harvard Medical School and Massachusetts General Hospital（哈佛医学院和麻省总医院）；Graduate Institute of Biomedical Electronics and Bioinformatics（生物医学电子与生物信息学研究生院）；University of Southern California（南加州大学）
Fuzhou University（福州大学）；College of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）
Automatic segmentation of left atrial (LA) scars from late gadolinium enhanced CMR images is a crucial step for atrial fibrillation (AF) recurrence analysis. However, delineating LA scars is tedious and error-prone due to the variation of scar shapes. In this work, we propose a boundary-aware LA scar segmentation network, which is composed of two branches to segment LA and LA scars, respectively. We explore the inherent spatial relationship between LA and LA scars. By introducing a Sobel fusion module between the two segmentation branches, the spatial information of LA boundaries can be propag…
Towards Robust Cardiac Segmentation using Graph Convolutional Networks
Norwegian University of Science and Technology（挪威科技大学）；University of Oslo（奥斯陆大学）；GE Research（GE研究中心）；St. Olavs hospital（圣奥拉夫斯医院）；Levanger hospital（莱旺厄尔医院）；SINTEF Health（辛特福健康研究院）
Hospital Virgen de la Arrixaca（拉阿里克萨卡圣母医院）；University of Murcia（穆尔西亚大学）
Left ventricular non-compaction (LVNC) is a rare cardiomyopathy characterized by abnormal trabeculations in the left ventricle cavity. Although traditional computer vision approaches exist for LVNC diagnosis, deep learning-based tools could not be found in the literature. In this paper, a first approach using convolutional neural networks (CNNs) is presented. Four CNNs are trained to automatically segment the compacted and trabecular areas of the left ventricle for a population of patients diagnosed with Hypertrophic cardiomyopathy. Inference results confirm that deep learning-based approaches…
