2D echocardiography is the most common imaging modality for cardiovascular diseases. The portability and relatively low-cost nature of Ultrasound (US) enable the US devices needed for performing echocardiography to be made widely available. However, acquiring and interpreting cardiac US images is operator dependent, limiting its use to only places where experts are present. Recently, Deep Learning (DL) has been used in 2D echocardiography for automated view classification, and structure and function assessment. Although these recent works show promise in developing computer-guided acquisition and automated interpretation of echocardiograms, most of these methods do not model and estimate uncertainty which can be important when testing on data coming from a distribution further away from that of the training data. Uncertainty estimates can be beneficial both during the image acquisition phase (by providing real-time feedback to the operator on acquired image's quality), and during automated measurement and interpretation. The performance of uncertainty models and quantification metric may depend on the prediction task and the models being compared. Hence, to gain insight of uncertainty modelling for left ventricular segmentation from US images, we compare three ensembling based uncertainty models quantified using four different metrics (one newly proposed) on state-of-the-art baseline networks using two publicly available echocardiogram datasets. We further demonstrate how uncertainty estimation can be used to automatically reject poor quality images and improve state-of-the-art segmentation results.
Harbin Institute of Technology（哈尔滨工业大学）；The Fourth Hospital of Harbin Medical University（哈尔滨医科大学附属第四医院）；Western University（西安大略大学）；University of Manchester（曼彻斯特大学）；Digital Image Group (DIG)（数字图像组(DIG)）
3D left ventricle (LV) segmentation on echocardiography is very important for diagnosis and treatment of cardiac disease. It is not only because of that echocardiography is a real-time imaging technology and widespread in clinical application, but also because of that LV segmentation on 3D echocardiography can provide more full volume information of heart than LV segmentation on 2D echocardiography. However, 3D LV segmentation on echocardiography is still an open and challenging task owing to the lower contrast, higher noise and data dimensionality, limited annotation of 3D echocardiography. I…
GDKVM: Echocardiography Video Segmentation via Spatiotemporal Key-Value Memory with Gated Delta Rule
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University（香港理工大学智能健康中心）
BOTM: Echocardiography Segmentation via Bi-directional Optimal Token Matching
School of Computing and Mathematical Sciences, University of Leicester（计算与数学科学学院，莱斯特大学）；Centre for AI, AstraZeneca（人工智能中心，阿斯利康）；Information Engineering Department The Chinese University of Hong Kong（信息工程系，香港中文大学）；Data Science Institute Imperial College London（数据科学研究所，伦敦帝国理工学院）
University of Notre Dame（圣母大学）；Guangdong Provincial People’s Hospital（广东省人民医院）
Most existing deep learning-based frameworks for image segmentation assume that a unique ground truth is known and can be used for performance evaluation. This is true for many applications, but not all. Myocardial segmentation of Myocardial Contrast Echocardiography (MCE), a critical task in automatic myocardial perfusion analysis, is an example. Due to the low resolution and serious artifacts in MCE data, annotations from different cardiologists can vary significantly, and it is hard to tell which one is the best. In this case, how can we find a good way to evaluate segmentation performance …
Hierarchical Spatio-temporal Segmentation Network for Ejection Fraction Estimation in Echocardiography Videos
PCA Lab（PCA实验室）；Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education（教育部高维信息智能感知与系统重点实验室）；School of Computer Science and Engineering（计算机科学与工程学院）；Nanjing University of Science and Technology（南京理工大学）
Imperial College London（帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）
Myocardial contrast echocardiography (MCE) is an imaging technique that assesses left ventricle function and myocardial perfusion for the detection of coronary artery diseases. Automatic MCE perfusion quantification is challenging and requires accurate segmentation of the myocardium from noisy and time-varying images. Random forests (RF) have been successfully applied to many medical image segmentation tasks. However, the pixel-wise RF classifier ignores contextual relationships between label outputs of individual pixels. RF which only utilizes local appearance features is also susceptible to …
New York University（纽约大学）；University of Rochester（罗切斯特大学）；Siemens Healthineers（西门子医疗）
Using catheter ablation to treat atrial fibrillation increasingly relies on intracardiac echocardiography (ICE) for an anatomical delineation of the left atrium and the pulmonary veins that enter the atrium. However, it is a challenge to build an automatic contouring algorithm because ICE is noisy and provides only a limited 2D view of the 3D anatomy. This work provides the first automatic solution to segment the left atrium and the pulmonary veins from ICE. In this solution, we demonstrate the benefit of building a cross-modality framework that can leverage a database of diagnostic images to …
Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling
University of Sherbrooke（谢布鲁克大学）；INSA（国立应用科学学院）；Universite Claude Bernard Lyon 1（里昂第一大学（克洛德·贝尔纳））；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；CREATIS（CREATIS研究机构）；Ultromics Ltd.（Ultromics有限公司）；King’s College London（伦敦国王学院）
OSA: Echocardiography Video Segmentation via Orthogonalized State Update and Anatomical Prior-aware Feature Enhancement
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University（香港理工大学护理学院智能健康中心）
Echocardiography Segmentation Using Neural ODE-based Diffeomorphic Registration Field
University of Engineering and Technology, Vietnam National University（越南国立大学工程技术大学）；VinUniversity（文梅大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
University of Notre Dame（圣母大学）；Boston Children’s Hospital, Harvard Medical School（哈佛医学院波士顿儿童医院）；Guangdong Cardiovascular Institute, Guangdong Provincial People’s Hospital（广东省人民医院广东省心血管病研究所）；University of Pittsburgh（匹兹堡大学）
Automatic myocardial segmentation of contrast echocardiography has shown great potential in the quantification of myocardial perfusion parameters. Segmentation quality control is an important step to ensure the accuracy of segmentation results for quality research as well as its clinical application. Usually, the segmentation quality control happens after the data acquisition. At the data acquisition time, the operator could not know the quality of the segmentation results. On-the-fly segmentation quality control could help the operator to adjust the ultrasound probe or retake data if the qual…
University of Alberta（阿尔伯塔大学）；Mazankowski Alberta Heart Institute（马赞科夫斯基阿尔伯塔心脏研究所）
Purpose: Echocardiography is commonly used as a non-invasive imaging tool in clinical practice for the assessment of cardiac function. However, delineation of the left ventricle is challenging due to the inherent properties of ultrasound imaging, such as the presence of speckle noise and the low signal-to-noise ratio. Methods: We propose a semi-automated segmentation algorithm for the delineation of the left ventricle in temporal 3D echocardiography sequences. The method requires minimal user interaction and relies on a diffeomorphic registration approach. Advantages of the method include no d…
Bayesian Optimization (BO) is a well-studied hyperparameter tuning technique that is more efficient than grid search for high-cost, high-parameter machine learning problems. Echocardiography is a ubiquitous modality for evaluating heart structure and function in cardiology. In this work, we use BO to optimize the architectural and training-related hyperparameters of a previously published deep fully convolutional neural network model for multi-structure segmentation in echocardiography. In a fair comparison, the resulting model outperforms this recent state-of-the-art on the annotated CAMUS da…
A Semi-Supervised Approach with Error Reflection for Echocardiography Segmentation
Shanghai University（上海大学）；Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）
The University of Winnipeg（温尼伯大学）；Qatar University（卡塔尔大学）；Hamad Medical Corporation（哈马德医疗公司）
Cardiac imaging known as echocardiography is a non-invasive tool utilized to produce data including images and videos, which cardiologists use to diagnose cardiac abnormalities in general and myocardial infarction (MI) in particular. Echocardiography machines can deliver abundant amounts of data that need to be quickly analyzed by cardiologists to help them make a diagnosis and treat cardiac conditions. However, the acquired data quality varies depending on the acquisition conditions and the patient's responsiveness to the setup instructions. These constraints are challenging to doctors especi…
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Contrastive learning has proven useful in many applications where access to labelled data is limited. The lack of annotated data is particularly problematic in medical image segmentation as it is difficult to have clinical experts manually annotate large volumes of data such as cardiac structures in ultrasound images of the heart. In this paper, We propose a self supervised contrastive learning method to segment the left ventricle from echocardiography when limited annotated images exist. Furthermore, we study the effect of contrastive pretraining on two well-known segmentation networks, UNet …
Reinforcement Learning for Unsupervised Domain Adaptation in Spatio-Temporal Echocardiography Segmentation
Department of Computer Science, University of Sherbrooke（谢布鲁克大学计算机科学系）；INSA, Universite Claude Bernard Lyon 1, CNRS UMR 5220, Inserm U1206, CREATIS（里昂1大学INSA、CNRS UMR 5220、Inserm U1206、CREATIS）；Dep. of Software and Information Technology Engineering, École de technologie supérieure（蒙特利尔工程学院软件与信息技术工程系）；Institut Universitaire de France (IUF)（法国国家科学院（IUF））
IBM Research – Almaden Research Center（IBM阿尔马登研究中心）；Eastern Virginia Medical School（东弗吉尼亚医学院）
Congenital heart disease (CHD) is the most common congenital abnormality associated with birth defects in the United States. Despite training efforts and substantial advancement in ultrasound technology over the past years, CHD remains an abnormality that is frequently missed during prenatal ultrasonography. Therefore, computer-aided detection of CHD can play a critical role in prenatal care by improving screening and diagnosis. Since many CHDs involve structural abnormalities, automatic segmentation of anatomical structures is an important step in the analysis of fetal echocardiograms. While …
Comparison of Loss Functions for Robust Deep Learning-based Echocardiography Segmentation when Learning with Partially Labelled Data from Multiple Domains
School of Biomedical Engineering & Imaging Sciences, King’s College London, UK（英国伦敦国王学院生物医学工程与影像科学学院）
Tampere University（坦佩雷大学）；Qatar University（卡塔尔大学）；Hamad Medical Corporation（哈马德医疗公司）
Myocardial infarction (MI), or commonly known as heart attack, is a life-threatening health problem worldwide from which 32.4 million people suffer each year. Early diagnosis and treatment of MI are crucial to prevent further heart tissue damages or death. The earliest and most reliable sign of ischemia is regional wall motion abnormality (RWMA) of the affected part of the ventricular muscle. Echocardiography can easily, inexpensively, and non-invasively exhibit the RWMA. In this article, we introduce a three-phase approach for early MI detection in low-quality echocardiography: 1) segmentatio…
EAGT: Echocardiography Augmentation for Generalisability and Transferability
THRIVE Centre, University of West London（西伦敦大学THRIVE中心）；University of West London（西伦敦大学）；School of Computing and Engineering, University of West London（西伦敦大学计算机与工程学院）
Compute-Optimal Network Design for Echocardiography Myocardial Segmentation and Perfusion Quantification using Neural Scaling Laws
Department of Bioengineering, Imperial College London（生物工程系，帝国理工学院伦敦分校）；National Heart and Lung Institute, Imperial College London（国家心脏和肺 institute，帝国理工学院伦敦分校）；Guy’s and St. Thomas’ NHS Foundation Trust（圣泰莫斯国家健康服务信托基金）
VinUni-Illinois Smart Health Center（文大-伊利诺伊智能健康中心）；VinUniversity（文大）；VNU University of Engineering and Technology（越南国立大学工程技术学院）；Vietnam National Heart Institute（越南国家心脏研究所）；Bach Mai Hospital（白梅医院）；College of Engineering & Computer Science（工程与计算机科学学院）
Early detection and localization of myocardial infarction (MI) can reduce the severity of cardiac damage through timely treatment interventions. In recent years, deep learning techniques have shown promise for detecting MI in echocardiographic images. However, there has been no examination of how segmentation accuracy affects MI classification performance and the potential benefits of using ensemble learning approaches. Our study investigates this relationship and introduces a robust method that combines features from multiple segmentation models to improve MI classification performance by lev…
Ultromics Ltd（Ultromics有限公司）；King’s College London（伦敦国王学院）；Sherbrooke University（舍布鲁克大学）
We propose a new method to automatically contour the left ventricle on 2D echocardiographic images. Unlike most existing segmentation methods, which are based on predicting segmentation masks, we focus at predicting the endocardial contour and the key landmark points within this contour (basal points and apex). This provides a representation that is closer to how experts perform manual annotations and hence produce results that are physiologically more plausible. Our proposed method uses a two-headed network based on the U-Net architecture. One head predicts the 7 contour points, and the other…
Med Fanavarn Plus Co.（梅德法纳瓦尔恩普拉斯公司）；Sharif University of Technology（谢里夫理工大学）；Rajaie Cardiovascular Medical and Research Center（拉贾伊心血管医疗与研究中心）；Iran University of Medical Science（伊朗医科大学）；Geneva University Hospital（日内瓦大学医院）；The University of British Columbia（不列颠哥伦比亚大学）；KU Leuven（鲁汶大学）
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed this problem through an encoder-decoder path. Despite outstanding overall performance, U-net ignores the contribution of all semantic strengths in the segmentation procedure. In the present study, we h…
EchoVLM: Measurement-Grounded Multimodal Learning for Echocardiography
Georgia Institute of Technology（佐治亚理工学院）；Siemens Healthineers（西门子医疗）；Siemens Healthcare Limited（西门子医疗有限公司）；Emory University（埃默里大学）；University of Southern California（南加州大学）
University of Sherbrooke（舍布鲁克大学）；Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Institut Universitaire de France（法国大学研究院）
Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axi…
MSV-Mamba: A Multiscale Vision Mamba Network for Echocardiography Segmentation
School of Computer and Information Engineering, Shanghai Polytechnic University（上海第二工业大学计算机与信息工程学院）；The First Affiliated Hospital of Jinan University（暨南大学附属第一医院）；Fudan University Huashan Hospital（复旦大学附属华山医院）
The University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）
Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecting cardiovascular disease at point-of-care. Echo imaging quality and acquisition time is highly dependent on the operator's experience level. Recent developments have shown the possibility of automating echo image quality quantification by mapping an expert's assessment of qualit…
A Data Augmentation Pipeline to Generate Synthetic Labeled Datasets of 3D Echocardiography Images using a GAN
GE Vingmed Ultrasound - GE Healthcare（GE Vingmed超声 - GE医疗）；CARIM School for Cardiovascular Diseases（CARIM心血管疾病学院）；Maastricht University Medical Center（马斯特里赫特大学医学中心）
Neural Implicit 3D Cardiac Shape Reconstruction from Sparse CT Angiography Slices Mimicking 2D Transthoracic Echocardiography Views
Department of Biomedical Engineering & Physics, Amsterdam UMC（阿姆斯特丹大学医学中心生物医学工程与物理系）；qurAI group, Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息研究所qurAI组）；Department of Radiology, Mayo Clinic（梅奥诊所放射科）；Department of Biomedical Signals and Systems, University of Twente（特温特大学生物医学信号与系统系）；Department of Cardiology, Amsterdam UMC（阿姆斯特丹大学医学中心心脏病科）；Department of Radiology & Nuclear Medicine, Amsterdam UMC（阿姆斯特丹大学医学中心放射与核医学科）
Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
School of Biomedical Engineering, University of British Columbia（不列颠哥伦比亚大学生物医学工程学院）；Department of Electrical and Computer Engineering, University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）；Department of Orthopaedics, University of British Columbia（不列颠哥伦比亚大学骨科）；Department of Radiology, University of British Columbia（不列颠哥伦比亚大学放射学系）；Department of Medicine, University of British Columbia（不列颠哥伦比亚大学医学系）
Extraction of volumetric indices from echocardiography: which deep learning solution for clinical use?
Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安让·莫内大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；University of Sherbrooke（舍布鲁克大学）；Hospices Civils de Lyon（里昂公立医院集团）
Lister Hill National Center for Biomedical Communications（李斯特·希尔国家生物医学交流中心）；National Heart, Lung, and Blood Institute（国家心肺血液研究所）
Accurate segmentation of medical images into anatomically meaningful regions is critical for the extraction of quantitative indices or biomarkers. The common pipeline for segmentation comprises regions of interest detection stage and segmentation stage, which are independent of each other and typically performed using separate deep learning networks. The performance of the segmentation stage highly relies on the extracted set of spatial features and the receptive fields. In this work, we propose an end-to-end network, called Trilateral Attention Network (TaNet), for real-time detection and seg…
University of Copenhagen（哥本哈根大学）；Technical University of Denmark（丹麦技术大学）
Probabilistic image segmentation encodes varying prediction confidence and inherent ambiguity in the segmentation problem. While different probabilistic segmentation models are designed to capture different aspects of segmentation uncertainty and ambiguity, these modelling differences are rarely discussed in the context of applications of uncertainty. We consider two common use cases of segmentation uncertainty, namely assessment of segmentation quality and active learning. We consider four established strategies for probabilistic segmentation, discuss their modelling capabilities, and investi…
Cardiovascular Research Institute, University of California, San Francisco（加州大学旧金山分校心血管研究所）；Department of Electrical Engineering and Computer Science, University of California at Berkeley（加州大学伯克利分校电气工程与计算机科学系）；Department of Medicine, University of California, San Francisco（加州大学旧金山分校医学系）；Division of Cardiology, Department of Medicine, and Feinberg Cardiovascular Research Institute（医学系心脏病学科及范伯格心血管研究所）
Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost serial assessment of cardiac function in the primary care and rural setting. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram (echo) interpretation. Our approach entailed: 1) preprocessing; 2) convolutional neural networks (CNN) for view identification, image segmentation, and phasing of the cardiac cycle; 3) quantification of chamber volumes and left ventricular mass; 4) parti…
Optical coherence tomography (OCT) is commonly used to analyze retinal layers for assessment of ocular diseases. In this paper, we propose a method for retinal layer segmentation and quantification of uncertainty based on Bayesian deep learning. Our method not only performs end-to-end segmentation of retinal layers, but also gives the pixel wise uncertainty measure of the segmentation output. The generated uncertainty map can be used to identify erroneously segmented image regions which is useful in downstream analysis. We have validated our method on a dataset of 1487 images obtained from 15 …
In semi-supervised medical image segmentation, most previous works draw on the common assumption that higher entropy means higher uncertainty. In this paper, we investigate a novel method of estimating uncertainty. We observe that, when assigned different misclassification costs in a certain degree, if the segmentation result of a pixel becomes inconsistent, this pixel shows a relative uncertainty in its segmentation. Therefore, we present a new semi-supervised segmentation model, namely, conservative-radical network (CoraNet in short) based on our uncertainty estimation and separate self-trai…
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Iris segmentation is a deterministic part of the iris recognition system. Unreliable segmentation of iris regions especially the limbic area is still the bottleneck problem, which impedes more accurate recognition. To make further efforts on accurate and reliable iris segmentation, we propose a bilateral self-attention module and design Bilateral Transformer (BiTrans) with hierarchical architecture by exploring spatial and visual relationships. The bilateral self-attention module adopts a spatial branch to capture spatial contextual information without resolution reduction and a visual branch …
