Accurate and consistent predictions of echocardiography parameters are important for cardiovascular diagnosis and treatment. In particular, segmentations of the left ventricle can be used to derive ventricular volume, ejection fraction (EF) and other relevant measurements. In this paper we propose a new automated method called EchoGraphs for predicting ejection fraction and segmenting the left ventricle by detecting anatomical keypoints. Models for direct coordinate regression based on Graph Convolutional Networks (GCNs) are used to detect the keypoints. GCNs can learn to represent the cardiac shape based on local appearance of each keypoint, as well as global spatial and temporal structures of all keypoints combined. We evaluate our EchoGraphs model on the EchoNet benchmark dataset. Compared to semantic segmentation, GCNs show accurate segmentation and improvements in robustness and inference runtime. EF is computed simultaneously to segmentations and our method also obtains state-of-the-art ejection fraction estimation. Source code is available online: https://github.com/guybenyosef/EchoGraphs.
Gaze-Assisted Human-Centric Domain Adaptation for Cardiac Ultrasound Image Segmentation
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
Broadly speaking, the objective in cardiac image segmentation is to delineate the outer and inner walls of the heart to segment out either the entire or parts of the organ boundaries. This paper will focus on MR images as they are the most widely used in cardiac segmentation -- as a result of the accurate morphological information and better soft tissue contrast they provide. This cardiac segmentation information is very useful as it eases physical measurements that provides useful metrics for cardiac diagnosis such as infracted volumes, ventricular volumes, ejection fraction, myocardial mass,…
Cardiovascular Research Institute, University of California, San Francisco（加州大学旧金山分校心血管研究所）；Department of Electrical Engineering and Computer Science, University of California at Berkeley（加州大学伯克利分校电气工程与计算机科学系）；Department of Medicine, University of California, San Francisco（加州大学旧金山分校医学系）；Division of Cardiology, Department of Medicine, and Feinberg Cardiovascular Research Institute（医学系心脏病学科及范伯格心血管研究所）
Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost serial assessment of cardiac function in the primary care and rural setting. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram (echo) interpretation. Our approach entailed: 1) preprocessing; 2) convolutional neural networks (CNN) for view identification, image segmentation, and phasing of the cardiac cycle; 3) quantification of chamber volumes and left ventricular mass; 4) parti…
Compositional Segmentation of Cardiac Images Leveraging Metadata
School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）；Queen Mary’s Digital Environment Research Institute (DERI)（玛丽女王数字环境研究所）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；School of Engineering and Materials Science, Queen Mary University of London（伦敦玛丽女王大学工程与材料科学学院）
Imperial College London（伦敦帝国学院）；City, University of London（伦敦城市大学）；University of Birmingham（伯明翰大学）
Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the chall…
Technion - Israel Institute of Technology（以色列理工学院）；University of Waterloo（滑铁卢大学）；Hillel Yaffe Medical Center（希勒尔·雅菲医疗中心）
Cardiac ultrasound imaging requires a high frame rate in order to capture rapid motion. This can be achieved by multi-line acquisition (MLA), where several narrow-focused received lines are obtained from each wide-focused transmitted line. This shortens the acquisition time at the expense of introducing block artifacts. In this paper, we propose a data-driven learning-based approach to improve the MLA image quality. We train an end-to-end convolutional neural network on pairs of real ultrasound cardiac data, acquired through MLA and the corresponding single-line acquisition (SLA). The network …
Eindhoven University of Technology（埃因霍温理工大学）；Shandong University of Science and Technology（山东科技大学）；Philips Research（飞利浦研究院）
Accurate and efficient catheter segmentation in 3D ultrasound (US) is essential for cardiac intervention. Currently, the state-of-the-art segmentation algorithms are based on convolutional neural networks (CNNs), which achieved remarkable performances in a standard Cartesian volumetric data. Nevertheless, these approaches suffer the challenges of low efficiency and GPU unfriendly image size. Therefore, such difficulties and expensive hardware requirements become a bottleneck to build accurate and efficient segmentation models for real clinical application. In this paper, we propose a novel Fru…
King's College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）
Cine cardiac magnetic resonance (CMR) has become the gold standard for the non-invasive evaluation of cardiac function. In particular, it allows the accurate quantification of functional parameters including the chamber volumes and ejection fraction. Deep learning has shown the potential to automate the requisite cardiac structure segmentation. However, the lack of robustness of deep learning models has hindered their widespread clinical adoption. Due to differences in the data characteristics, neural networks trained on data from a specific scanner are not guaranteed to generalise well to dat…
John Radcliffe Hospital（约翰·拉德克利夫医院）；Ultromics Ltd（Ultromics有限公司）；Friedrich--Alexander University Erlangen--Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Imperial College London（帝国理工学院）
Cardiac ultrasound imaging is used to diagnose various heart diseases. Common analysis pipelines involve manual processing of the video frames by expert clinicians. This suffers from intra- and inter-observer variability. We propose a novel approach to ultrasound video analysis using a transformer architecture based on a Residual Auto-Encoder Network and a BERT model adapted for token classification. This enables videos of any length to be processed. We apply our model to the task of End-Systolic (ES) and End-Diastolic (ED) frame detection and the automated computation of the left ventricular …
Eindhoven University of Technology（埃因霍温理工大学）；Philips Research（飞利浦研究院）
Fast and accurate catheter detection in cardiac catheterization using harmless 3D ultrasound (US) can improve the efficiency and outcome of the intervention. However, the low image quality of US requires extra training for sonographers to localize the catheter. In this paper, we propose a catheter detection method based on a pre-trained VGG network, which exploits 3D information through re-organized cross-sections to segment the catheter by a shared fully convolutional network (FCN), which is called a Direction-Fused FCN (DF-FCN). Based on the segmented image of DF-FCN, the catheter can be loc…
The Research Center of Biomedical Engineering (CREB)（生物医学工程研究中心 (CREB)）；Universitat Politècnica de Cataluya（加泰罗尼亚理工大学）；École Normale Supérieure（巴黎高等师范学院）；Institute for Bioengineering of Catalonia（加泰罗尼亚生物工程研究所）；Radboud University Nijmegen Medical Centre（拉德堡德大学奈梅亨医学中心）；Eidgenössische Technische Hochschule Zürich - ETH Zurich（苏黎世联邦理工学院）
Cardiac motion estimation is an important diagnostic tool to detect heart diseases and it has been explored with modalities such as MRI and conventional ultrasound (US) sequences. US cardiac motion estimation still presents challenges because of the complex motion patterns and the presence of noise. In this work, we propose a novel approach to estimate the cardiac motion using ultrafast ultrasound data. -- Our solution is based on a variational formulation characterized by the L2-regularized class. The displacement is represented by a lattice of b-splines and we ensure robustness by applying a…
EchoFlow: A Foundation Model for Cardiac Ultrasound Image and Video Generation
Imperial College London（帝国理工学院）；University of Birmingham（伯明翰大学）；Ultromics Ltd.（奥特罗米克斯有限公司）；University of Oxford（牛津大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；National University of Singapore（新加坡国立大学）
Romanian Institute of Science and Technology（罗马尼亚科学技术研究所）；University of Medicine and Pharmacy of Craiova（克拉约瓦医药大学）
The present paper puts forward an incipient study that uses a traditional segmentation method based on Zernike moments for extracting significant features from frames of fetal echocardiograms from first trimester color Doppler examinations. A distance based approach is then used on the obtained indicators to classify frames of three given categories that should be present in a normal heart condition. The computational tool shows promise in supporting the obstetrician in a rapid recognition of heart views during screening.
Unified Review and Benchmark of Deep Segmentation Architectures for Cardiac Ultrasound on CAMUS
Department of Computer Science；Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea；Department of Semiconductor System Engineering, Sejong University, Seoul, 05006, Republic of Korea；Department of Operations Research；Statistics, Faculty of Organizational Sciences, University of Belgrade, Belgrade, Serbia；Department of Industrial Engineering \& Management, Yuan Ze University, Taoyuan City 320315, Taiwan；Department of Applied Mathematical Science, College of Science；Technology, Korea University, Sejong 30019, Republic of Korea
Hannover Medical School（汉诺威医学院）；German Center for Lung Research（德国肺研究中心）；Johannes Gutenberg University（约翰内斯·古腾堡大学）
Background: Cardiac MRI derived biventricular mass and function parameters, such as end-systolic volume (ESV), end-diastolic volume (EDV), ejection fraction (EF), stroke volume (SV), and ventricular mass (VM) are clinically well established. Image segmentation can be challenging and time-consuming, due to the complex anatomy of the human heart. Objectives: This study introduces $ν$-net (/nju:n$\varepsilon$t/) -- a deep learning approach allowing for fully-automated high quality segmentation of right (RV) and left ventricular (LV) endocardium and epicardium for extraction of cardiac function pa…
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；The Hamlyn Centre for Robotic Surgery, Imperial College of London（伦敦帝国理工学院哈姆林机器人手术中心）；Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（上海交通大学图像处理与模式识别研究所）；Shanghai Xinhua Hospital（上海新华医院）
Echocardiography plays an important part in diagnostic aid in cardiac diseases. A critical step in echocardiography-aided diagnosis is to extract the standard planes since they tend to provide promising views to present different structures that are benefit to diagnosis. To this end, this paper proposes a spatial-temporal embedding framework to extract the standard view planes from 4D STIC (spatial-temporal image corre- lation) volumes. The proposed method is comprised of three stages, the frame smoothing, spatial-temporal embedding and final classification. In first stage, an L 0 smoothing fi…
With superiorities on low cost, portability, and free of radiation, echocardiogram is a widely used imaging modality for left ventricle (LV) function quantification. However, automatic LV segmentation and motion tracking is still a challenging task. In addition to fuzzy border definition, low contrast, and abounding artifacts on typical ultrasound images, the shape and size of the LV change significantly in a cardiac cycle. In this work, we propose a temporal affine network (TAN) to perform image analysis in a warped image space, where the shape and size variations due to the cardiac motion as…
DCD: A Semantic Segmentation Model for Fetal Ultrasound Four-Chamber View
School of Information and Communication（信息与通信学院）；Guilin University of Electronic Technology（桂林电子科技大学）；Faculty of Humanities and Arts（人文艺术学院）；Macau University of Science and Technology（澳门科技大学）；Guangxi Key Laboratory of Machine Vision and Intelligent Control（广西机器视觉与智能控制重点实验室）；Wuzhou University（梧州大学）；Faculty of Innovation Engineering（创新工程学院）；Department of Public Education（公共教育系）；Wuzhou Medical College（梧州医学院）；College of Big Data and Software Engineering（大数据与软件工程学院）；Shenzhen Maternity and Child Healthcare Hospital（深圳妇幼保健医院）；Southern Medical University（南方医科大学）；School of Microelectronics（微电子学院）
GE Vingmed Ultrasound, GE Healthcare（通用电气医疗集团GE Vingmed超声部门）；University of Oslo（奥斯陆大学）；Norwegian Computing Center（挪威计算中心）
Two-dimensional echocardiography (2DE) measurements of left ventricle (LV) dimensions are highly significant markers of several cardiovascular diseases. These measurements are often used in clinical care despite suffering from large variability between observers. This variability is due to the challenging nature of accurately finding the correct temporal and spatial location of measurement endpoints in ultrasound images. These images often contain fuzzy boundaries and varying reflection patterns between frames. In this work, we present a convolutional neural network (CNN) based approach to aut…
Imperial College London（伦敦帝国学院）；King’s College London（伦敦国王学院）；St Thomas’ Hospital（圣托马斯医院）
Prenatal screening with ultrasound can lower neonatal mortality significantly for selected cardiac abnormalities. However, the need for human expertise, coupled with the high volume of screening cases, limits the practically achievable detection rates. In this paper we discuss the potential for deep learning techniques to aid in the detection of congenital heart disease (CHD) in fetal ultrasound. We propose a pipeline for automated data curation and classification. During both training and inference, we exploit an auxiliary view classification task to bias features toward relevant cardiac stru…
Anatomical Prior-Driven Framework for Autonomous Robotic Cardiac Ultrasound Standard View Acquisition
State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology（华中科技大学智能制造装备与技术国家重点实验室）；School of Biomedical Engineering, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）生物医学工程学院）；Information Intelligence Lab, Department of Electrical Engineering, National Chung Hsing University（中原大学电子工程系信息智能实验室）；Institute of Medical Equipment Science and Engineering, Huazhong University of Science and Technology（华中科技大学医学装备科学与工程研究院）；Department of Ultrasound Medicine, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属同济医院超声医学科）；Institute of Systems Science (ISS), National University of Singapore (NUS)（新加坡国立大学系统科学研究所）
Ultromics Ltd.（奥特罗米克斯有限公司）；Metalynx Ltd.（梅塔林克斯有限公司）；Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）
Analysis of cardiac ultrasound images is commonly performed in routine clinical practice for quantification of cardiac function. Its increasing automation frequently employs deep learning networks that are trained to predict disease or detect image features. However, such models are extremely data-hungry and training requires labelling of many thousands of images by experienced clinicians. Here we propose the use of contrastive learning to mitigate the labelling bottleneck. We train view classification models for imbalanced cardiac ultrasound datasets and show improved performance for views/cl…
Benchmarking Self-Supervised Models for Cardiac Ultrasound View Classification
Department of Systems；Computer Engineering, Carleton University, Ottawa, Ontario, Canada Department of Methodological；Implementation Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada Department of Acute Care Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada Department of Obstetrics；Gynecology, University of Ottawa, Ottawa, Ontario, Canada School of Epidemiology；Public Health, University of Ottawa, Ottawa, Ontario, Canada Department of Obstetrics, Gynecology \& Newborn Care, The Ottawa Hospital, Ottawa, Ontario, Canada International；Global Health Office, University of Ottawa, Ottawa, Ontario, Canada College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound（医学超声关键技术研发实验室）；School of Biomedical Engineering, Medical School, Shenzhen University（生物医学工程学院）；Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院）；Centre for CIMIM, Manchester University（CIMIM研究中心）；Center for Cardiovascular Disease（心血管疾病中心）；The Affiliated Suzhou Hospital of Nanjing Medical University（南京医科大学附属苏州医院）；Suzhou Municipal Hospital, Gusu School, Suzhou, China（苏州 municipal 医院）；The First Affiliated Hospital of Kunming Medical University（昆明医科大学第一附属医院）；The First Affiliated Hospital of Anhui Medical University North（安徽医科大学北院）；The Second People’s Hospital（第二人民医院）；The First Affiliated Hospital of Anhui Medical University（安徽医科大学第一附属医院）；The People's Hospital of Tongren City（铜仁市人民医院）；The First People's Hospital of Taicang City（太仓市第一人民医院）；Guangdong Provincial People’s Hospital（广东省人民医院）
Manipal Institute of Technology, Manipal Academy of Higher Education（马尼帕尔高等教育学院马尼帕尔理工学院）
Segmentation of the left ventricle in cardiac magnetic resonance imaging MRI scans enables cardiologists to calculate the volume of the left ventricle and subsequently its ejection fraction. The ejection fraction is a measurement that expresses the percentage of blood leaving the heart with each contraction. Cardiologists often use ejection fraction to determine one's cardiac function. We propose multiscale template matching technique for detection and an elliptical active disc for automated segmentation of the left ventricle in MR images. The elliptical active disc optimizes the local energy …
Cardiac motion analysis from B-mode ultrasound sequence is a key task in assessing the health of the heart. The paper proposes a new methodology for cardiac motion analysis based on the temporal behaviour of points of interest on the myocardium. We define a new signal called the Temporal Flow Graph (TFG) which depicts the movement of a point of interest over time. It is a graphical representation derived from a flow field and describes the temporal evolution of a point. We prove that TFG for an object undergoing periodic motion is also periodic. This principle can be utilized to derive both gl…
King’s College London（伦敦国王学院）；Ultromics Ltd.（Ultromics有限公司）
Accurate geometric quantification of the human heart is a key step in the diagnosis of numerous cardiac diseases, and in the management of cardiac patients. Ultrasound imaging is the primary modality for cardiac imaging, however acquisition requires high operator skill, and its interpretation and analysis is difficult due to artifacts. Reconstructing cardiac anatomy in 3D can enable discovery of new biomarkers and make imaging less dependent on operator expertise, however most ultrasound systems only have 2D imaging capabilities. We propose both a simple alteration to the Pix2Vox++ networks fo…
Automated Interpretable 2D Video Extraction from 3D Echocardiography
University of California, Los Angeles（加州大学洛杉矶分校）；Kaiser Permanente Division of Research（凯撒医疗集团研究部）；Cedars-Sinai Medical Center（西达赛西医学中心）；Stanford University（斯坦福大学）
Myocardial Strain Drift Correction in Deep Learning Based Ultrasound Tracking
University of Sherbrooke（舍布鲁克大学）；INSA（法国国立应用科学学院）；Université Lyon 1（里昂第一大学）；CREATIS（CREATIS实验室）；Institut Universitaire de France (IUF)（法国大学研究院（IUF））；NTNU（挪威科技大学）；St. Olavs Hospital（圣奥拉夫医院）；SINTEF Digital（SINTEF数字研究所）；Levanger Hospital（莱旺厄尔医院）；Nord-Trøndelag Hospital Trust（北特伦德拉格医院信托）；Hôpital Croix-Rousse（克鲁瓦鲁斯医院）；Hospices Civils de Lyon（里昂民用医院集团）；Hôpital Lyon Sud（里昂南部医院）
Left ventricular segmentation is essential for measuring left ventricular function indices. Segmentation of one or several images requires an initial guess of the contour. It is hypothesized here that creating an initial guess by first detecting anatomical markers, would lead to correct detection of the endocardium. The first step of the algorithm presented here includes automatic detection of the mitral valve. Next, the apex is detected in the same frame. The valve is then tracked throughout the cardiac cycle. Contours passing from the apex to each valve corner are then found using a dynamic …
We present an automatic method to describe clinically useful information about scanning, and to guide image interpretation in ultrasound (US) videos of the fetal heart. Our method is able to jointly predict the visibility, viewing plane, location and orientation of the fetal heart at the frame level. The contributions of the paper are three-fold: (i) a convolutional neural network architecture is developed for a multi-task prediction, which is computed by sliding a 3x3 window spatially through convolutional maps. (ii) an anchor mechanism and Intersection over Union (IoU) loss are applied for i…
End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning
organization= Concordia Institute for Information Systems Engineering, Concordia University , city= Montreal , country= Canada；organization= Department of Mechanical, Industrial；Aerospace Engineering, Concordia University , city= Montreal , country= Canada；organization= Department of Medicine, Laval University , city= Quebec , country= Canada；organization= Department of Electrical, Computer；Software Engineering, Ontario Tech University , city= Oshawa , country= Canada；organization= Department of Computer Science, Khalifa University , city= Abu Dhabi , country= UAE；organization= Artificial Intelligence \& Cyber Systems Research Center, Department of CSM, Lebanese American University , city= Beirut , country= Lebanon；organization= Department of Electrical Engineering；Computer Science, Khalifa University , city= Abu Dhabi , country= UAE
Echocardiography Segmentation Using Neural ODE-based Diffeomorphic Registration Field
University of Engineering and Technology, Vietnam National University（越南国立大学工程技术大学）；VinUniversity（文梅大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Imperial College London（帝国理工学院）；Friedrich–Alexander University Erlangen–Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Spotify（Spotify公司）；University College London（伦敦大学学院）；Ultromics Ltd（Ultromics有限公司）；King’s College London（伦敦国王学院）；John Radcliffe Hospital（约翰·拉德克利夫医院）
Causally-enabled machine learning frameworks could help clinicians to identify the best course of treatments by answering counterfactual questions. We explore this path for the case of echocardiograms by looking into the variation of the Left Ventricle Ejection Fraction, the most essential clinical metric gained from these examinations. We combine deep neural networks, twin causal networks and generative adversarial methods for the first time to build D'ARTAGNAN (Deep ARtificial Twin-Architecture GeNerAtive Networks), a novel causal generative model. We demonstrate the soundness of our approac…
Segmentation of the left ventricle (LV) from cardiac magnetic resonance imaging (MRI) datasets is an essential step for calculation of clinical indices such as ventricular volume and ejection fraction. In this work, we employ deep learning algorithms combined with deformable models to develop and evaluate a fully automatic segmentation tool for the LV from short-axis cardiac MRI datasets. The method employs deep learning algorithms to learn the segmentation task from the ground true data. Convolutional networks are employed to automatically detect the LV chamber in MRI dataset. Stacked autoenc…
