Cardiac Output Prediction from Echocardiograms: Self-Supervised Learning with Limited Data
University of Turin, Italy（都灵大学，意大利）；LINKS Foundation, Italy（LINKS基金会，意大利）
Adson Duarte, Davide Vitturini, Emanuele Milillo, Andrea Bragagnolo, Carlo Alberto Barbano, Riccardo Renzulli, Michele Cannito, Federico Giacobbe, Francesco Bru… 展开作者
Adson Duarte, Davide Vitturini, Emanuele Milillo, Andrea Bragagnolo, Carlo Alberto Barbano, Riccardo Renzulli, Michele Cannito, Federico Giacobbe, Francesco Bruno, Ovidio de Filippo, Fabrizio D'Ascenzo, Marco Grangetto
Cardiac Output (CO) is a key parameter in the diagnosis and management of cardiovascular diseases. However, its accurate measurement requires right-heart catheterization, an invasive and time-consuming procedure, motivating the development of reliable non-invasive alternatives using echocardiography. In this work, we propose a self-supervised learning (SSL) pretraining strategy based on SimCLR to improve CO prediction from apical four-chamber echocardiographic videos. The pretraining is performed using the same limited dataset available for the downstream task, demonstrating the potential of SSL even under data scarcity. Our results show that SSL mitigates overfitting and improves representation learning, achieving an average Pearson correlation of 0.41 on the test set and outperforming PanEcho, a model trained on over one million echocardiographic exams. Source code is available at https://github.com/EIDOSLAB/cardiac-output.
S2MNet: Speckle-To-Mesh Net for Three-Dimensional Cardiac Morphology Reconstruction via Echocardiogram
Department of Statistics, University of Georgia（佐治亚大学统计学系）；Department of Statistics, Havard University（哈佛大学统计学系）；Institute of Geriatric Medicine, Chinese Academy of Medical Sciences, Beijing Hospital（中国医学科学院老年医学研究所）
CardiacNet: Learning to Reconstruct Abnormalities for Cardiac Disease Assessment from Echocardiogram Videos
The Hong Kong University of Science and Technology（香港科技大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
Ultromics Ltd.（奥特罗米克斯有限公司）；Metalynx Ltd.（梅塔林克斯有限公司）；Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）
Analysis of cardiac ultrasound images is commonly performed in routine clinical practice for quantification of cardiac function. Its increasing automation frequently employs deep learning networks that are trained to predict disease or detect image features. However, such models are extremely data-hungry and training requires labelling of many thousands of images by experienced clinicians. Here we propose the use of contrastive learning to mitigate the labelling bottleneck. We train view classification models for imbalanced cardiac ultrasound datasets and show improved performance for views/cl…
EchoFlow: A Foundation Model for Cardiac Ultrasound Image and Video Generation
Imperial College London（帝国理工学院）；University of Birmingham（伯明翰大学）；Ultromics Ltd.（奥特罗米克斯有限公司）；University of Oxford（牛津大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
NVIDIA（英伟达）；Lenovo Research（联想研究院）；Piedmont Heart Institute（皮埃蒙特心脏研究所）；Tencent Medical AI Lab（腾讯医疗人工智能实验室）；University of California, Irvine（加州大学欧文分校）
Echocardiography has become routinely used in the diagnosis of cardiomyopathy and abnormal cardiac blood flow. However, manually measuring myocardial motion and cardiac blood flow from echocardiogram is time-consuming and error-prone. Computer algorithms that can automatically track and quantify myocardial motion and cardiac blood flow are highly sought after, but have not been very successful due to noise and high variability of echocardiography. In this work, we propose a neural multi-scale self-supervised registration (NMSR) method for automated myocardial and cardiac blood flow dense track…
EchoingECG: An Electrocardiogram Cross-Modal Model for Echocardiogram Tasks
Peter Munk Cardiac Centre, University Health Network (UHN)（彼得·默克心脏中心，大学健康网络）；Department of Medical Biophysics, UofT（医学生物物理学系）；Ted Rogers Centre for Heart Research, UHN（泰德·罗杰斯心脏病研究中心，大学健康网络）；Department of Computer Science, University of Toronto (UofT)（计算机科学系，多伦多大学）；Toronto General Hospital Research Institute, UHN（多伦多总医院研究 institute）；Department of Medical Imaging, UofT（医学影像学系）；Vector Institute, Toronto（向量研究所）
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology Shenzhen Research Institute（香港科技大学深圳研究院）
Left-ventricular ejection fraction (LVEF) is an important indicator of heart failure. Existing methods for LVEF estimation from video require large amounts of annotated data to achieve high performance, e.g. using 10,030 labeled echocardiogram videos to achieve mean absolute error (MAE) of 4.10. Labeling these videos is time-consuming however and limits potential downstream applications to other heart diseases. This paper presents the first semi-supervised approach for LVEF prediction. Unlike general video prediction tasks, LVEF prediction is specifically related to changes in the left ventric…
ETH Zürich（苏黎世联邦理工学院）；University Children’s Hospital Regensburg (KUNO)（雷根斯堡大学儿童医院（KUNO））
Pulmonary hypertension (PH) in newborns and infants is a complex condition associated with several pulmonary, cardiac, and systemic diseases contributing to morbidity and mortality. Therefore, accurate and early detection of PH is crucial for successful management. Using echocardiography, the primary diagnostic tool in pediatrics, human assessment is both time-consuming and expertise-demanding, raising the need for an automated approach. In this work, we present an interpretable multi-view video-based deep learning approach to predict PH for a cohort of 194 newborns using echocardiograms. We u…
Westfälische Hochschule – University of Applied Sciences（威斯特法伦应用科学大学）
Cardiac image segmentation is a powerful tool in regard to diagnostics and treatment of cardiovascular diseases. Purely feature-based detection of anatomical structures like the mitral valve is a laborious task due to specifically required feature engineering and is especially challenging in echocardiograms, because of their inherently low contrast and blurry boundaries between some anatomical structures. With the publication of further annotated medical datasets and the increase in GPU processing power, deep learning-based methods in medical image segmentation became more feasible in the past…
Multimodal Foundation Models For Echocardiogram Interpretation
Cedars-Sinai Medical Center（西达赛奈医疗中心）；University of California Los Angeles（加州大学洛杉矶分校）；University of California, San Francisco（加州大学旧金山分校）；San Francisco Veterans Affairs Medical Center（旧金山退伍军人事务医疗中心）
Tufts University（塔夫茨大学）；CVAI Solutions（CVAI解决方案公司）；Tufts Medical Center（塔夫茨医疗中心）
Semi-supervised image classification has shown substantial progress in learning from limited labeled data, but recent advances remain largely untested for clinical applications. Motivated by the urgent need to improve timely diagnosis of life-threatening heart conditions, especially aortic stenosis, we develop a benchmark dataset to assess semi-supervised approaches to two tasks relevant to cardiac ultrasound (echocardiogram) interpretation: view classification and disease severity classification. We find that a state-of-the-art method called MixMatch achieves promising gains in heldout accura…
Cardiac segmentation is an essential step for the diagnosis of cardiovascular diseases. However, pixel-wise dense labeling is both costly and time-consuming. Scribble, as a form of sparse annotation, is more accessible than full annotations. However, it's particularly challenging to train a segmentation network with weak supervision from scribbles. To tackle this problem, we propose a new scribble-guided method for cardiac segmentation, based on the Positive-Unlabeled (PU) learning framework and global consistency regularization, and termed as ShapePU. To leverage unlabeled pixels via PU learn…
University of California, San Francisco（加州大学旧金山分校）
Prenatal diagnosis of tetralogy of Fallot (TOF) and hypoplastic left heart syndrome (HLHS), two serious congenital heart defects, improves outcomes and can in some cases facilitate in utero interventions. In practice, however, the fetal diagnosis rate for these lesions is only 30-50 percent in community settings. Improving fetal diagnosis of congenital heart disease is therefore critical. Deep learning is a cutting-edge machine learning technique for finding patterns in images but has not yet been applied to prenatal diagnosis of congenital heart disease. Using 685 retrospectively collected ec…
Due to the imbalanced and limited data, semi-supervised medical image segmentation methods often fail to produce superior performance for some specific tailed classes. Inadequate training for those particular classes could introduce more noise to the generated pseudo labels, affecting overall learning. To alleviate this shortcoming and identify the under-performing classes, we propose maintaining a confidence array that records class-wise performance during training. A fuzzy fusion of these confidence scores is proposed to adaptively prioritize individual confidence metrics in every sample rat…
Imperial College London（伦敦帝国理工学院）；Queen Mary University of London（伦敦玛丽女王大学）
In the recent years, convolutional neural networks have transformed the field of medical image analysis due to their capacity to learn discriminative image features for a variety of classification and regression tasks. However, successfully learning these features requires a large amount of manually annotated data, which is expensive to acquire and limited by the available resources of expert image analysts. Therefore, unsupervised, weakly-supervised and self-supervised feature learning techniques receive a lot of attention, which aim to utilise the vast amount of available data, while at the …
Stanford University（斯坦福大学）；Houston Methodist DeBakey Heart Centre（休斯顿卫理公会德贝基心脏中心）；Spectrum Health Grand Rapids（大急流城Spectrum Health）；Columbia University（哥伦比亚大学）
Non-invasive and cost effective in nature, the echocardiogram allows for a comprehensive assessment of the cardiac musculature and valves. Despite progressive improvements over the decades, the rich temporally resolved data in echocardiography videos remain underutilized. Human reads of echocardiograms reduce the complex patterns of cardiac wall motion, to a small list of measurements of heart function. Furthermore, all modern echocardiography artificial intelligence (AI) systems are similarly limited by design - automating measurements of the same reductionist metrics rather than utilizing th…
Self-Supervised Temporal Regularization for Landmark-Based Cardiac Segmentation with Automatic AHA Regional Mapping
Biomedical Image Technologies, ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain（生物医学图像技术，电信工程学院，马德里理工大学，马德里，西班牙）；Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto de Salud Carlos III, Madrid, Spain（生物工程、生物材料和纳米医学网络生物医学研究中心（CIBER-BBN），卡洛斯三世健康研究所，马德里，西班牙）；APOLO Biotech, Ciudad Autónoma de Buenos Aires, Argentina（APOLO Biotech，布宜诺斯艾利斯自治市，阿根廷）；Departamento de Computación, Universidad de Buenos Aires, Ciudad Autónoma de Buenos Aires, Argentina（计算系，布宜诺斯艾利斯大学，布宜诺斯艾利斯自治市，阿根廷）；Instituto de Ciencias de la Computación (ICC), CONICET-Universidad de Buenos Aires, Ciudad Autónoma de Buenos Aires, Argentina（计算机科学研究所（ICC），CONICET-布宜诺斯艾利斯大学，布宜诺斯艾利斯自治市，阿根廷）
Automatic diagnosis of cardiac magnetic resonance images based on semi-supervised learning
School of Information and Electrical Engineering, Hunan University of Science and Technology（湖南科技大学信息与电气工程学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Guilin Medical University（桂林医学院）；Rucheng County Hospital of Traditional Chinese Medicine（汝城县中医医院）
Demographic-Aware Self-Supervised Anomaly Detection Pretraining for Equitable Rare Cardiac Diagnosis
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；National University of Defense Technology（国防科技大学）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（瑞金医院，上海交通大学医学院）；Renji Hospital, Shanghai Jiao Tong University School of Medicine（仁济医院，上海交通大学医学院）
Development of Automated Neural Network Prediction for Echocardiographic Left ventricular Ejection Fraction
University of Birmingham（伯明翰大学）；Manchester University NHS Foundation Trust（曼彻斯特大学NHS基金会信托）；University Hospitals Birmingham NHS Foundation Trust（伯明翰大学医院NHS基金会信托）；Manchester Metropolitan University（曼彻斯特城市大学）；Loughborough University（拉夫堡大学）
Yale University（耶鲁大学）；Northeastern University（东北大学）；University of Washington（华盛顿大学）
Reliable motion estimation and strain analysis using 3D+time echocardiography (4DE) for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. However, motion estimation is difficult due to the low-SNR that stems from the inherent image properties of 4DE, and intelligent regularization is critical for producing reliable motion estimates. In this work, we incorporated the notion of domain adaptation into a supervised neural network regularization framework. We first propose an unsupervised autoencoder network with biomechanical constra…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Lenovo Research（联想研究院）；Lenovo Ltd.（联想有限公司）
Accurate segmentation of cardiac structures can assist doctors to diagnose diseases, and to improve treatment planning, which is highly demanded in the clinical practice. However, the shortage of annotation and the variance of the data among different vendors and medical centers restrict the performance of advanced deep learning methods. In this work, we present a fully automatic method to segment cardiac structures including the left (LV) and right ventricle (RV) blood pools, as well as for the left ventricular myocardium (MYO) in MRI volumes. Specifically, we design a semi-supervised learnin…
Automatic segmentation of the heart cavity is an essential task for the diagnosis of cardiac diseases. In this paper, we propose a semi-supervised segmentation setup for leveraging unlabeled data to segment Left-ventricle, Right-ventricle, and Myocardium. We utilize an enhanced version of residual U-Net architecture on a large-scale cardiac MRI dataset. Handling the class imbalanced data issue using dice loss, the enhanced supervised model is able to achieve better dice scores in comparison with a vanilla U-Net model. We applied several augmentation techniques including histogram matching to i…
Columbia University（哥伦比亚大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Imperial College London（帝国理工学院）；Institut Polytechnique de Paris（巴黎理工学院）
Automatically identifying the structural substrates underlying cardiac abnormalities can potentially provide real-time guidance for interventional procedures. With the knowledge of cardiac tissue substrates, the treatment of complex arrhythmias such as atrial fibrillation and ventricular tachycardia can be further optimized by detecting arrhythmia substrates to target for treatment (i.e., adipose) and identifying critical structures to avoid. Optical coherence tomography (OCT) is a real-time imaging modality that aids in addressing this need. Existing approaches for cardiac image analysis main…
The University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）
In echocardiography (echo), an electrocardiogram (ECG) is conventionally used to temporally align different cardiac views for assessing critical measurements. However, in emergencies or point-of-care situations, acquiring an ECG is often not an option, hence motivating the need for alternative temporal synchronization methods. Here, we propose Echo-SyncNet, a self-supervised learning framework to synchronize various cross-sectional 2D echo series without any external input. The proposed framework takes advantage of both intra-view and inter-view self supervisions. The former relies on spatiote…
Deformable Image Registration for Self-supervised Cardiac Phase Detection in Multi-View Multi-Disease Cardiac Magnetic Resonance Images
Department of Internal Medicine III, Heidelberg University Hospital（海德堡大学第三内科部门医院）；Medical Faculty of Heidelberg University（海德堡大学医学院）；DZHK (German Centre for Cardiovascular Research)（德国心血管研究中心）；Division of Pediatric Cardiology, Department of Pediatrics, UT Southwestern /Children’s Health（儿童心脏病科，德克萨斯西南医学中心/儿童健康）；Department of Cardiothoracic, Transplantation and Vascular Surgery, Hannover Medical School（汉诺威医学院心胸外科、移植外科和血管外科部门）
本文提出一种自监督深度学习方法，通过多视角多疾病心脏磁共振图像检测五个关键帧，提高了心脏相位检测的准确率，适用于SAX和4CH cine CMR。
Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification
Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine（放射科与放射科学系，约翰霍普金斯大学医学院）；Department of Electrical and Computer Engineering, Johns Hopkins University（电气与计算机工程系，约翰霍普金斯大学）；Department of Computer Science, University of North Carolina at Chapel Hill（计算机科学系，北卡罗来纳大学教堂山分校）；Department of Radiology, University of Colorado Denver Anschutz Medical Campus（放射科，科罗拉多大学丹佛分校安舒茨医学中心）；Department of Radiology, Second Xiangya Hospital, Central South University（放射科，中南大学湘雅医院）；Department of Cardiology, Johns Hopkins University School of Medicine（心血管科，约翰霍普金斯大学医学院）；Department of Diagnostic Imaging, Brown University Health（诊断影像科，布朗大学健康中心）
The Chinese University of Hong Kong（香港中文大学）；Stanford University（斯坦福大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Medical image annotations are prohibitively time-consuming and expensive to obtain. To alleviate annotation scarcity, many approaches have been developed to efficiently utilize extra information, e.g.,semi-supervised learning further exploring plentiful unlabeled data, domain adaptation including multi-modality learning and unsupervised domain adaptation resorting to the prior knowledge from additional modality. In this paper, we aim to investigate the feasibility of simultaneously leveraging abundant unlabeled data and well-established cross-modality data for annotation-efficient medical imag…
Training deep learning models on cardiac magnetic resonance imaging (CMR) can be a challenge due to the small amount of expert generated labels and inherent complexity of data source. Self-supervised contrastive learning (SSCL) has recently been shown to boost performance in several medical imaging tasks. However, it is unclear how much the pre-trained representation reflects the primary organ of interest compared to spurious surrounding tissue. In this work, we evaluate the optimal method of incorporating prior knowledge of anatomy into a SSCL training paradigm. Specifically, we evaluate usin…
Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）
With large-scale well-labeled datasets, deep learning has shown significant success in medical image segmentation. However, it is challenging to acquire abundant annotations in clinical practice due to extensive expertise requirements and costly labeling efforts. Recently, contrastive learning has shown a strong capacity for visual representation learning on unlabeled data, achieving impressive performance rivaling supervised learning in many domains. In this work, we propose a novel multi-scale multi-view global-local contrastive learning (MMGL) framework to thoroughly explore global and loca…
Anomaly Detection in Electrocardiograms: Advancing Clinical Diagnosis Through Self-Supervised Learning
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）
We propose a method to classify cardiac pathology based on a novel approach to extract image derived features to characterize the shape and motion of the heart. An original semi-supervised learning procedure, which makes efficient use of a large amount of non-segmented images and a small amount of images segmented manually by experts, is developed to generate pixel-wise apparent flow between two time points of a 2D+t cine MRI image sequence. Combining the apparent flow maps and cardiac segmentation masks, we obtain a local apparent flow corresponding to the 2D motion of myocardium and ventricu…
Heidelberg University Hospital（海德堡大学医院）；DZHK (German Centre for Cardiovascular Research)（德国心血管研究中心）；UT Southwestern Medical Center（德克萨斯大学西南医学中心）；German Competence Network for Congenital Heart Defects（德国先天性心脏病能力网络）；Hannover Medical School（汉诺威医学院）
Cardiac magnetic resonance (CMR) sequences visualise the cardiac function voxel-wise over time. Simultaneously, deep learning-based deformable image registration is able to estimate discrete vector fields which warp one time step of a CMR sequence to the following in a self-supervised manner. However, despite the rich source of information included in these 3D+t vector fields, a standardised interpretation is challenging and the clinical applications remain limited so far. In this work, we show how to efficiently use a deformable vector field to describe the underlying dynamic process of a car…
The Hong Kong University of Science and Technology（香港科技大学）
Deep regression is an important problem with numerous applications. These range from computer vision tasks such as age estimation from photographs, to medical tasks such as ejection fraction estimation from echocardiograms for disease tracking. Semi-supervised approaches for deep regression are notably under-explored compared to classification and segmentation tasks, however. Unlike classification tasks, which rely on thresholding functions for generating class pseudo-labels, regression tasks use real number target predictions directly as pseudo-labels, making them more sensitive to prediction…
Faculty of Electrical & Computer Engineering, Technion - IIT（以色列理工学院电气与计算机工程学院）；Faculty of Biomedical Engineering, Technion - IIT（以色列理工学院生物医学工程学院）；The May-Blum-Dahl MRI Research Center, Faculty of Biomedical Engineering, Technion - IIT（以色列理工学院生物医学工程学院梅-布-达尔磁共振成像研究中心）；Bnai Zion Medical Center（巴奈锡安医疗中心）
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；University of Leeds（利兹大学）
Left ventricle segmentation and morphological assessment are essential for improving diagnosis and our understanding of cardiomyopathy, which in turn is imperative for reducing risk of myocardial infarctions in patients. Convolutional neural network (CNN) based methods for cardiac magnetic resonance (CMR) image segmentation rely on supervision with pixel-level annotations, and may not generalize well to images from a different domain. These methods are typically sensitive to variations in imaging protocols and data acquisition. Since annotating multi-sequence CMR images is tedious and subject …
IMT School for Advanced Studies Lucca（IMT卢卡高级研究学院）；School of Engineering, University of Edinburgh（爱丁堡大学工程学院）
There has been an increasing focus in learning interpretable feature representations, particularly in applications such as medical image analysis that require explainability, whilst relying less on annotated data (since annotations can be tedious and costly). Here we build on recent innovations in style-content representations to learn anatomy, imaging characteristics (appearance) and temporal correlations. By introducing a self-supervised objective of predicting future cardiac phases we improve disentanglement. We propose a temporal transformer architecture that given an image conditioned on …
