A Computer Vision Pipeline for Automated Determination of Cardiac Structure and Function and Detection of Disease by Two-Dimensional Echocardiography
Cardiovascular Research Institute, University of California, San Francisco（加州大学旧金山分校心血管研究所）；Department of Electrical Engineering and Computer Science, University of California at Berkeley（加州大学伯克利分校电气工程与计算机科学系）；Department of Medicine, University of California, San Francisco（加州大学旧金山分校医学系）；Division of Cardiology, Department of Medicine, and Feinberg Cardiovascular Research Institute（医学系心脏病学科及范伯格心血管研究所）
Jeffrey Zhang, Sravani Gajjala, Pulkit Agrawal, Geoffrey H. Tison, Laura A. Hallock, Lauren Beussink-Nelson, Eugene Fan, Mandar A. Aras, ChaRandle Jordan, Kirst… 展开作者
Jeffrey Zhang, Sravani Gajjala, Pulkit Agrawal, Geoffrey H. Tison, Laura A. Hallock, Lauren Beussink-Nelson, Eugene Fan, Mandar A. Aras, ChaRandle Jordan, Kirsten E. Fleischmann, Michelle Melisko, Atif Qasim, Alexei Efros, Sanjiv J. Shah, Ruzena Bajcsy, Rahul C. Deo
Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost serial assessment of cardiac function in the primary care and rural setting. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram (echo) interpretation. Our approach entailed: 1) preprocessing; 2) convolutional neural networks (CNN) for view identification, image segmentation, and phasing of the cardiac cycle; 3) quantification of chamber volumes and left ventricular mass; 4) particle tracking to compute longitudinal strain; and 5) targeted disease detection. CNNs accurately identified views (e.g. 99% for apical 4-chamber) and segmented individual cardiac chambers. Cardiac structure measurements agreed with study report values (e.g. mean absolute deviations (MAD) of 7.7 mL/kg/m2 for left ventricular diastolic volume index, 2918 studies). We computed automated ejection fraction and longitudinal strain measurements (within 2 cohorts), which agreed with commercial software-derived values [for ejection fraction, MAD=5.3%, N=3101 studies; for strain, MAD=1.5% (n=197) and 1.6% (n=110)], and demonstrated applicability to serial monitoring of breast cancer patients for trastuzumab cardiotoxicity. Overall, we found that, compared to manual measurements, automated measurements had superior performance across seven internal consistency metrics with an average increase in the Spearman correlation coefficient of 0.05 (p=0.02). Finally, we developed disease detection algorithms for hypertrophic cardiomyopathy and cardiac amyloidosis, with C-statistics of 0.93 and 0.84, respectively. Our pipeline lays the groundwork for using automated interpretation to support point-of-care handheld cardiac ultrasound and large-scale analysis of the millions of echos archived within healthcare systems.
University of Oslo（奥斯陆大学）；GE Vingmed Ultrasound（GE维ingmed超声公司）；GE Research（通用电气研究中心）
Accurate and consistent predictions of echocardiography parameters are important for cardiovascular diagnosis and treatment. In particular, segmentations of the left ventricle can be used to derive ventricular volume, ejection fraction (EF) and other relevant measurements. In this paper we propose a new automated method called EchoGraphs for predicting ejection fraction and segmenting the left ventricle by detecting anatomical keypoints. Models for direct coordinate regression based on Graph Convolutional Networks (GCNs) are used to detect the keypoints. GCNs can learn to represent the cardiac…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Deep fully convolutional neural network (FCN) based architectures have shown great potential in medical image segmentation. However, such architectures usually have millions of parameters and inadequate number of training samples leading to over-fitting and poor generalization. In this paper, we present a novel highly parameter and memory efficient FCN based architecture for medical image analysis. We propose a novel up-sampling path which incorporates long skip and short-cut connections to overcome the feature map explosion in FCN like architectures. In order to processes the input images at …
Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels
The University of Manchester（曼彻斯特大学）；Kaohsiung Chang Gung Memorial Hospital（高雄长庚纪念医院）；Chang Gung University College of Medicine（长庚大学医学院）；Cheng Shiu University（正修科技大学）；National Yang Ming Chiao Tung University（国立阳明交通大学）；The Hong Kong University of Science and Technology（香港科技大学）
Automated Interpretable 2D Video Extraction from 3D Echocardiography
University of California, Los Angeles（加州大学洛杉矶分校）；Kaiser Permanente Division of Research（凯撒医疗集团研究部）；Cedars-Sinai Medical Center（西达赛西医学中心）；Stanford University（斯坦福大学）
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Contrastive learning has proven useful in many applications where access to labelled data is limited. The lack of annotated data is particularly problematic in medical image segmentation as it is difficult to have clinical experts manually annotate large volumes of data such as cardiac structures in ultrasound images of the heart. In this paper, We propose a self supervised contrastive learning method to segment the left ventricle from echocardiography when limited annotated images exist. Furthermore, we study the effect of contrastive pretraining on two well-known segmentation networks, UNet …
Multi-Disease-Aware Training Strategy for Cardiac MR Image Segmentation
College of Computer Science, Sichuan Normal University（四川师范大学计算机科学学院）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算与人工智能学院）；West China Hospital, Sichuan University（四川大学华西医院）
A Semi-Supervised Approach with Error Reflection for Echocardiography Segmentation
Shanghai University（上海大学）；Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）
King's College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）
Cine cardiac magnetic resonance (CMR) has become the gold standard for the non-invasive evaluation of cardiac function. In particular, it allows the accurate quantification of functional parameters including the chamber volumes and ejection fraction. Deep learning has shown the potential to automate the requisite cardiac structure segmentation. However, the lack of robustness of deep learning models has hindered their widespread clinical adoption. Due to differences in the data characteristics, neural networks trained on data from a specific scanner are not guaranteed to generalise well to dat…
EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function
Medical Technology & E-Health, Akershus University Hospital, Norway（医疗技术与电子健康，阿克ershus大学医院，挪威）；Department of Technology Systems, University of Oslo, Norway（技术系统系，奥斯陆大学，挪威）；Department of Cardiology, Akershus University Hospital, Norway（心脏病学系，阿克ershus大学医院，挪威）
Echocardiography to Cardiac MRI View Transformation for Real-Time Blind Restoration
Tampere University（坦佩雷大学）；Qatar University（卡塔尔大学）；VTT Technical Research Centre of Finland（芬兰国家技术研究中心）；Hamad Medical Corporation Hospital（哈马德医疗公司医院）；Hayatabad Medical Complex（海亚特阿巴德医疗中心）
Nanjing University of Science and Technology（南京理工大学）
Convolutional Neural Networks (CNNs) have achieved high accuracy for cardiac structure segmentation if training cases and testing cases are from the same distribution. However, the performance would be degraded if the testing cases are from a distinct domain (e.g., new MRI scanners, clinical centers). In this paper, we propose a histogram matching (HM) data augmentation method to eliminate the domain gap. Specifically, our method generates new training cases by using HM to transfer the intensity distribution of testing cases to existing training cases. The proposed method is quite simple and c…
Latent Motion Profiling for Annotation-free Cardiac Phase Detection in Adult and Fetal Echocardiography Videos
Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Hong Kong Centre for Cerebro-Cardiovascular Health Engineering（香港脑心血管健康工程中心）；Department of Maternal and Child Health and Urological Sciences, Sapienza University of Rome（罗马萨皮恩扎大学妇幼保健与泌尿科学系）；Nuffield Department of Women’s and Reproductive Health, University of Oxford（牛津大学努尔菲尔德妇女与生殖健康系）
Orientation-Robust Latent Motion Trajectory Learning for Annotation-free Cardiac Phase Detection in Fetal Echocardiography
Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford（生物医学工程研究所，工程科学系，牛津大学）；Nuffield Department of Women’s and Reproductive Health, University of Oxford（妇女与生殖健康纽菲尔德部门，牛津大学）
China University of Petroleum（中国石油大学）；Qingdao Women and Children’s Hospital（青岛市妇女儿童医院）
Echocardiography is a powerful prenatal examination tool for early diagnosis of fetal congenital heart diseases (CHDs). The four-chamber (FC) view is a crucial and easily accessible ultrasound (US) image among echocardiography images. Automatic analysis of FC views contributes significantly to the early diagnosis of CHDs. The first step to automatically analyze fetal FC views is locating the fetal four crucial chambers of heart in a US image. However, it is a greatly challenging task due to several key factors, such as numerous speckles in US images, the fetal cardiac chambers with small size …
King's College London（伦敦国王学院）；Guy’s and St Thomas' Hospital（盖伊和圣托马斯医院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Royal Brompton Hospital (Guy’s and St Thomas' NHS Foundation Trust)（皇家布朗普顿医院（盖伊和圣托马斯NHS基金会信托））
Left ventricular (LV) function is an important factor in terms of patient management, outcome, and long-term survival of patients with heart disease. The most recently published clinical guidelines for heart failure recognise that over reliance on only one measure of cardiac function (LV ejection fraction) as a diagnostic and treatment stratification biomarker is suboptimal. Recent advances in AI-based echocardiography analysis have shown excellent results on automated estimation of LV volumes and LV ejection fraction. However, from time-varying 2-D echocardiography acquisition, a richer descr…
IBM Research – Almaden Research Center（IBM阿尔马登研究中心）；Eastern Virginia Medical School（东弗吉尼亚医学院）
Congenital heart disease (CHD) is the most common congenital abnormality associated with birth defects in the United States. Despite training efforts and substantial advancement in ultrasound technology over the past years, CHD remains an abnormality that is frequently missed during prenatal ultrasonography. Therefore, computer-aided detection of CHD can play a critical role in prenatal care by improving screening and diagnosis. Since many CHDs involve structural abnormalities, automatic segmentation of anatomical structures is an important step in the analysis of fetal echocardiograms. While …
The University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）
In echocardiography (echo), an electrocardiogram (ECG) is conventionally used to temporally align different cardiac views for assessing critical measurements. However, in emergencies or point-of-care situations, acquiring an ECG is often not an option, hence motivating the need for alternative temporal synchronization methods. Here, we propose Echo-SyncNet, a self-supervised learning framework to synchronize various cross-sectional 2D echo series without any external input. The proposed framework takes advantage of both intra-view and inter-view self supervisions. The former relies on spatiote…
Extraction of volumetric indices from echocardiography: which deep learning solution for clinical use?
Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安让·莫内大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；University of Sherbrooke（舍布鲁克大学）；Hospices Civils de Lyon（里昂公立医院集团）
Cardiac valve event timing in echocardiography using deep learning and triplane recordings
GE Vingmed Ultrasound；Norwegian University of Science and Technology（挪威科技大学）；Oslo University Hospital, Rikshospitalet（奥斯陆大学医院（雷克沙医院））；University of Oslo（奥斯陆大学）；SINTEF Digital（SINTEF数字研究所）；St. Olavs hospital（圣奥拉夫斯医院）
GDKVM: Echocardiography Video Segmentation via Spatiotemporal Key-Value Memory with Gated Delta Rule
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University（香港理工大学智能健康中心）
Peng Cheng Laboratory（鹏城实验室）；Beihang University（北京航空航天大学）；Capital Medical University（首都医科大学）；Beijing Anzhen Hospital（北京安贞医院）；Beijing Advanced Innovation Center for Big Data-Based Precision Medicine（北京大数据精准医疗先进创新中心）
As an important scan plane, four chamber view is routinely performed in both second trimester perinatal screening and fetal echocardiographic examinations. The biometrics in this plane including cardio-thoracic ratio (CTR) and cardiac axis are usually measured by sonographers for diagnosing congenital heart disease. However, due to the commonly existing artifacts like acoustic shadowing, the traditional manual measurements not only suffer from the low efficiency, but also with the inconsistent results depending on the operators' skills. In this paper, we present an anchor-free ellipse detectio…
Automatic Cardiac Pathology Recognition in Echocardiography Images Using Higher Order Dynamic Mode Decomposition and a Vision Transformer for Small Datasets
Universidad Politécnica de Madrid（马德里理工大学）；Centro Nacional de Investigaciones Cardiovasculares (CNIC)（国家心血管研究中心）；Universidad Complutense de Madrid（马德里康普顿斯大学）；Center for Computational Simulation (CCS)（计算仿真中心）
EchoPrime: A Multi-Video View-Informed Vision-Language Model for Comprehensive Echocardiography Interpretation
Cedars-Sinai Medical Center（西达赛奈医疗中心）；University of California Los Angeles（加州大学洛杉矶分校）；Stanford University（斯坦福大学）；University of California, San Francisco（加州大学旧金山分校）；San Francisco Veterans Affairs Medical Center（旧金山退伍军人事务医疗中心）
Neural Implicit 3D Cardiac Shape Reconstruction from Sparse CT Angiography Slices Mimicking 2D Transthoracic Echocardiography Views
Department of Biomedical Engineering & Physics, Amsterdam UMC（阿姆斯特丹大学医学中心生物医学工程与物理系）；qurAI group, Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息研究所qurAI组）；Department of Radiology, Mayo Clinic（梅奥诊所放射科）；Department of Biomedical Signals and Systems, University of Twente（特温特大学生物医学信号与系统系）；Department of Cardiology, Amsterdam UMC（阿姆斯特丹大学医学中心心脏病科）；Department of Radiology & Nuclear Medicine, Amsterdam UMC（阿姆斯特丹大学医学中心放射与核医学科）
A Novel Data Augmentation Tool for Enhancing Machine Learning Classification: A New Application of the Higher Order Dynamic Mode Decomposition for Improved Cardiac Disease Identification
Universidad Politécnica de Madrid（马德里理工大学）；ETSI Aeronáutica y del Espacio（航空航天高级技术工程学院）；ETSI Telecomunicación（电信高级技术工程学院）；Universidad Complutense de Madrid（马德里康普顿斯大学）；Centro Nacional de Investigaciones Cardiovasculares (CNIC)（国家心血管研究中心）；Center for Computational Simulation (CCS)（计算仿真中心）
The University of Winnipeg（温尼伯大学）；Qatar University（卡塔尔大学）；Hamad Medical Corporation（哈马德医疗公司）
Cardiac imaging known as echocardiography is a non-invasive tool utilized to produce data including images and videos, which cardiologists use to diagnose cardiac abnormalities in general and myocardial infarction (MI) in particular. Echocardiography machines can deliver abundant amounts of data that need to be quickly analyzed by cardiologists to help them make a diagnosis and treat cardiac conditions. However, the acquired data quality varies depending on the acquisition conditions and the patient's responsiveness to the setup instructions. These constraints are challenging to doctors especi…
Echocardiography Segmentation Using Neural ODE-based Diffeomorphic Registration Field
University of Engineering and Technology, Vietnam National University（越南国立大学工程技术大学）；VinUniversity（文梅大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
University of Sherbrooke（舍布鲁克大学）；Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Institut Universitaire de France（法国大学研究院）
Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axi…
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；The Hamlyn Centre for Robotic Surgery, Imperial College of London（伦敦帝国理工学院哈姆林机器人手术中心）；Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（上海交通大学图像处理与模式识别研究所）；Shanghai Xinhua Hospital（上海新华医院）
Echocardiography plays an important part in diagnostic aid in cardiac diseases. A critical step in echocardiography-aided diagnosis is to extract the standard planes since they tend to provide promising views to present different structures that are benefit to diagnosis. To this end, this paper proposes a spatial-temporal embedding framework to extract the standard view planes from 4D STIC (spatial-temporal image corre- lation) volumes. The proposed method is comprised of three stages, the frame smoothing, spatial-temporal embedding and final classification. In first stage, an L 0 smoothing fi…
EchoVLM: Measurement-Grounded Multimodal Learning for Echocardiography
Georgia Institute of Technology（佐治亚理工学院）；Siemens Healthineers（西门子医疗）；Siemens Healthcare Limited（西门子医疗有限公司）；Emory University（埃默里大学）；University of Southern California（南加州大学）
Echo4DIR: 4D Implicit Heart Reconstruction from 2D Echocardiography Videos
Department of Biomedical Engineering, National University of Singapore, Singapore（新加坡国立大学生物医学工程系）；School of Information Science and Engineering, Yunnan University, Kunming, China（云南大学信息科学与工程学院）
University of Alberta（阿尔伯塔大学）；Mazankowski Alberta Heart Institute（马赞科夫斯基阿尔伯塔心脏研究所）
Purpose: Echocardiography is commonly used as a non-invasive imaging tool in clinical practice for the assessment of cardiac function. However, delineation of the left ventricle is challenging due to the inherent properties of ultrasound imaging, such as the presence of speckle noise and the low signal-to-noise ratio. Methods: We propose a semi-automated segmentation algorithm for the delineation of the left ventricle in temporal 3D echocardiography sequences. The method requires minimal user interaction and relies on a diffeomorphic registration approach. Advantages of the method include no d…
Continuous 3D Myocardial Motion Tracking via Echocardiography
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；Medical School, Nanjing University（南京大学医学院）；Zhongshan Hospital, Fudan University（复旦大学中山医院）；College of Optical Sciences, University of Arizona（亚利桑那大学光学科学学院）
NepAl Applied Mathematics and Informatics Institute for Research (NAAMII)（尼泊尔应用数学与信息学研究所（NAAMII））
2D echocardiography is the most common imaging modality for cardiovascular diseases. The portability and relatively low-cost nature of Ultrasound (US) enable the US devices needed for performing echocardiography to be made widely available. However, acquiring and interpreting cardiac US images is operator dependent, limiting its use to only places where experts are present. Recently, Deep Learning (DL) has been used in 2D echocardiography for automated view classification, and structure and function assessment. Although these recent works show promise in developing computer-guided acquisition …
Unsupervised 4D Cardiac Motion Tracking with Spatiotemporal Optical Flow Networks
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；DJI Technology Co., Ltd.（大疆创新科技有限公司）；Shenzhen University of Advanced Technology（深圳理工大学）
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；Department of Radiology, University Medical Center Utrecht（乌得勒支大学医学中心放射科）
Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features derived from the obtained segmentations were used in a Random Forest classifier to label patients as suffering from dilated cardiomyopathy, hypertrophic cardiomyopathy, heart failure following myocard…
Comparison of Loss Functions for Robust Deep Learning-based Echocardiography Segmentation when Learning with Partially Labelled Data from Multiple Domains
School of Biomedical Engineering & Imaging Sciences, King’s College London, UK（英国伦敦国王学院生物医学工程与影像科学学院）
