A Cognitive Explainer for Fetal ultrasound images classifier Based on Medical Concepts
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；University Town of Shenzhen, Nanshan District, Shenzhen（深圳大学城南山区深圳校区）；Beijing Tsinghua Changgung Hospital（北京清华大学长庚医院）；Peking University Shenzhen Hospital（北京大学深圳医院）；Shenzhen Baoan District Maternal and Child Health Hospital（深圳宝安区妇幼保健医院）
Fetal standard scan plane detection during 2-D mid-pregnancy examinations is a highly complex task, which requires extensive medical knowledge and years of training. Although deep neural networks (DNN) can assist inexperienced operators in these tasks, their lack of transparency and interpretability limit their application. Despite some researchers have been committed to visualizing the decision process of DNN, most of them only focus on the pixel-level features and do not take into account the medical prior knowledge. In this work, we propose an interpretable framework based on key medical concepts, which provides explanations from the perspective of clinicians' cognition. Moreover, we utilize a concept-based graph convolutional neural(GCN) network to construct the relationships between key medical concepts. Extensive experimental analysis on a private dataset has shown that the proposed method provides easy-to-understand insights about reasoning results for clinicians.
FUSEP: A Multi-Center Benchmark for Diverse Tasks in Early Pregnancy Fetal Ultrasound Screening
Hunan University（湖南大学）；The Hong Kong University of Science and Technology（香港科技大学）；Anhui University（安徽大学）；Shenzhen Maternity and Child Healthcare Hospital（深圳市妇幼保健院）；Sichuan Provincial Maternity and Child Health Care Hospital（四川省妇幼保健院）；National University of Singapore（新加坡国立大学）；Yunnan Maternal and Child Health Hospital（云南省妇幼保健院）；Zhejiang University（浙江大学）
Self-supervised Normality Learning and Divergence Vector-guided Model Merging for Zero-shot Congenital Heart Disease Detection in Fetal Ultrasound Videos
University of Oxford（牛津大学）；University of Technology Nuremberg（纽伦堡工业大学）
Sano Centre for Computational Medicine（萨诺计算医学中心）；Warsaw University of Technology（华沙理工大学）；Fetai Health Ltd.（Fetai健康有限公司）；Medical University of Warsaw（华沙医科大学）；Tooploox（Tooploox公司）
In this paper, we propose an end-to-end multi-task neural network called FetalNet with an attention mechanism and stacked module for spatio-temporal fetal ultrasound scan video analysis. Fetal biometric measurement is a standard examination during pregnancy used for the fetus growth monitoring and estimation of gestational age and fetal weight. The main goal in fetal ultrasound scan video analysis is to find proper standard planes to measure the fetal head, abdomen and femur. Due to natural high speckle noise and shadows in ultrasound data, medical expertise and sonographic experience are requ…
Google Health（谷歌健康）；University of Zambia School of Medicine（赞比亚大学医学院）；University of North Carolina School of Medicine（北卡罗来纳大学医学院）；UNC Global Projects – Zambia, LLC（北卡罗来纳大学全球项目赞比亚有限责任公司）
Despite considerable progress in maternal healthcare, maternal and perinatal deaths remain high in low-to-middle income countries. Fetal ultrasound is an important component of antenatal care, but shortage of adequately trained healthcare workers has limited its adoption. We developed and validated an artificial intelligence (AI) system that uses novice-acquired "blind sweep" ultrasound videos to estimate gestational age (GA) and fetal malpresentation. We further addressed obstacles that may be encountered in low-resourced settings. Using a simplified sweep protocol with real-time AI feedback …
Objective. This work investigates the use of deep convolutional neural networks (CNN) to automatically perform measurements of fetal body parts, including head circumference, biparietal diameter, abdominal circumference and femur length, and to estimate gestational age and fetal weight using fetal ultrasound videos. Approach. We developed a novel multi-task CNN-based spatio-temporal fetal US feature extraction and standard plane detection algorithm (called FUVAI) and evaluated the method on 50 freehand fetal US video scans. We compared FUVAI fetal biometric measurements with measurements made …
Cross-Modal Ultrasound-MRI Learning for Fetal Brain Ventricular Volumetry and Abnormality Screening
Harvard Medical School（哈佛医学院）；Shenzhen University（深圳大学）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）；School of Biomedical Engineering and Informatics, Nanjing Medical University（南京医科大学生物医学工程与信息学院）
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）
Ultrasound imaging is a standard examination during pregnancy that can be used for measuring specific biometric parameters towards prenatal diagnosis and estimating gestational age. Fetal head circumference (HC) is one of the significant factors to determine the fetus growth and health. In this paper, a multi-task deep convolutional neural network is proposed for automatic segmentation and estimation of HC ellipse by minimizing a compound cost function composed of segmentation dice score and MSE of ellipse parameters. Experimental results on fetus ultrasound dataset in different trimesters of …
Technical University of Denmark（丹麦技术大学）；CAMES Rigshospitalet（哥本哈根大学医院CAMES中心）
Confounding information in the form of text or markings embedded in medical images can severely affect the training of diagnostic deep learning algorithms. However, data collected for clinical purposes often have such markings embedded in them. In dermatology, known examples include drawings or rulers that are overrepresented in images of malignant lesions. In this paper, we encounter text and calipers placed on the images found in national databases containing fetal screening ultrasound scans, which correlate with standard planes to be predicted. In order to utilize the vast amounts of data a…
University College London（伦敦大学学院）；Elizabeth Garrett Anderson Institute for Women’s Health, University College London（伦敦大学学院伊丽莎白·加勒特·安德森妇女健康研究所）；NIHR University College London Hospitals Biomedical Research Centre（英国国家健康研究所伦敦大学学院医院生物医学研究中心）
During pregnancy, ultrasound examination in the second trimester can assess fetal size according to standardized charts. To achieve a reproducible and accurate measurement, a sonographer needs to identify three standard 2D planes of the fetal anatomy (head, abdomen, femur) and manually mark the key anatomical landmarks on the image for accurate biometry and fetal weight estimation. This can be a time-consuming operator-dependent task, especially for a trainee sonographer. Computer-assisted techniques can help in automating the fetal biometry computation process. In this paper, we present a uni…
Sagol Brain Institute, Tel Aviv Sourasky Medical Center（特拉维夫苏拉斯基医疗中心萨戈尔脑研究所）；University College London（伦敦大学学院）
Fetal growth assessment from ultrasound is based on a few biometric measurements that are performed manually and assessed relative to the expected gestational age. Reliable biometry estimation depends on the precise detection of landmarks in standard ultrasound planes. Manual annotation can be time-consuming and operator dependent task, and may results in high measurements variability. Existing methods for automatic fetal biometry rely on initial automatic fetal structure segmentation followed by geometric landmark detection. However, segmentation annotations are time-consuming and may be inac…
General Methods Make Great Domain-specific Foundation Models: A Case-study on Fetal Ultrasound
University of Copenhagen（哥本哈根大学）；Technical University of Denmark（丹麦技术大学）；Copenhagen Academy of Medical Education and Simulation (CAMES)（哥本哈根医学教育与模拟学院）；Copenhagen University Hospital Rigshospitalet（哥本哈根大学医院拉格比医院）；Brown University（布朗大学）；Pioneer Centre for AI（先锋人工智能中心）
Sano Centre for Computational Medicine（萨诺计算医学中心）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）；The Medical Centre of Postgraduate Education（研究生教育医学中心）；Poznan University of Medical Sciences（波兹南医科大学）；Medical University of Warsaw（华沙医科大学）；Warsaw University of Technology（华沙理工大学）
Predicting fetal weight at birth is an important aspect of perinatal care, particularly in the context of antenatal management, which includes the planned timing and the mode of delivery. Accurate prediction of weight using prenatal ultrasound is challenging as it requires images of specific fetal body parts during advanced pregnancy which is difficult to capture due to poor quality of images caused by the lack of amniotic fluid. As a consequence, predictions which rely on standard methods often suffer from significant errors. In this paper we propose the Residual Transformer Module which exte…
In this work, we present an empirical study of DCGANs, including hyperparameter heuristics and image quality assessment, as a way to address the scarcity of datasets to investigate fetal head ultrasound. We present experiments to show the impact of different image resolutions, epochs, dataset size input, and learning rates for quality image assessment on four metrics: mutual information (MI), Fréchet inception distance (FID), peak-signal-to-noise ratio (PSNR), and local binary pattern vector (LBPv). The results show that FID and LBPv have stronger relationship with clinical image quality score…
One of the biggest challenges for deep learning algorithms in medical image analysis is the indiscriminate mixing of image properties, e.g. artifacts and anatomy. These entangled image properties lead to a semantically redundant feature encoding for the relevant task and thus lead to poor generalization of deep learning algorithms. In this paper we propose a novel representation disentanglement method to extract semantically meaningful and generalizable features for different tasks within a multi-task learning framework. Deep neural networks are utilized to ensure that the encoded features are…
Università Politecnica delle Marche（马尔凯理工大学）；Università degli Studi di Macerata（马切拉塔大学）；Scuola Superiore Sant’Anna（圣安娜高等学校）
Deep-learning (DL) algorithms are becoming the standard for processing ultrasound (US) fetal images. Despite a large number of survey papers already present in this field, most of them are focusing on a broader area of medical-image analysis or not covering all fetal US DL applications. This paper surveys the most recent work in the field, with a total of 145 research papers published after 2017. Each paper is analyzed and commented on from both the methodology and application perspective. We categorized the papers in (i) fetal standard-plane detection, (ii) anatomical-structure analysis, and …
Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健服务基金会信托）；City, University of London（伦敦城市大学）；Technical University of Munich（慕尼黑工业大学）
Fetal ultrasound screening during pregnancy plays a vital role in the early detection of fetal malformations which have potential long-term health impacts. The level of skill required to diagnose such malformations from live ultrasound during examination is high and resources for screening are often limited. We present an interpretable, atlas-learning segmentation method for automatic diagnosis of Hypo-plastic Left Heart Syndrome (HLHS) from a single `4 Chamber Heart' view image. We propose to extend the recently introduced Image-and-Spatial Transformer Networks (Atlas-ISTN) into a framework t…
Dual Conditioned Diffusion Models for Out-Of-Distribution Detection: Application to Fetal Ultrasound Videos
Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Nuffield Department of Women’s and Reproductive Health, University of Oxford（牛津大学纳菲尔德妇女与生殖健康系）
King’s College London（伦敦国王学院）；University Hospital of Padova（帕多瓦大学医院）；University of Padova（帕多瓦大学）；University of Warwick（华威大学）
The automatic analysis of ultrasound sequences can substantially improve the efficiency of clinical diagnosis. In this work we present our attempt to automate the challenging task of measuring the vascular diameter of the fetal abdominal aorta from ultrasound images. We propose a neural network architecture consisting of three blocks: a convolutional layer for the extraction of imaging features, a Convolution Gated Recurrent Unit (C-GRU) for enforcing the temporal coherence across video frames and exploiting the temporal redundancy of a signal, and a regularized loss function, called \textit{C…
Imperial College London（伦敦帝国理工学院）；Technical University Munich（慕尼黑工业大学）
Deep neural networks exhibit limited generalizability across images with different entangled domain features and categorical features. Learning generalizable features that can form universal categorical decision boundaries across domains is an interesting and difficult challenge. This problem occurs frequently in medical imaging applications when attempts are made to deploy and improve deep learning models across different image acquisition devices, across acquisition parameters or if some classes are unavailable in new training databases. To address this problem, we propose Mutual Information…
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for in…
University of Oxford（牛津大学）；Amazon（亚马逊）；Kings College London（伦敦国王学院）
This paper presents a novel approach to automatic fetal brain biometry motivated by needs in low- and medium- income countries. Specifically, we leverage high-end (HE) ultrasound images to build a biometry solution for low-cost (LC) point-of-care ultrasound images. We propose a novel unsupervised domain adaptation approach to train deep models to be invariant to significant image distribution shift between the image types. Our proposed method, which employs a Dual Adversarial Calibration (DAC) framework, consists of adversarial pathways which enforce model invariance to; i) adversarial perturb…
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this paper we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end…
Automated Fetal Biometry Assessment with Deep Ensembles using Sparse-Sampling of 2D Intrapartum Ultrasound Images
Oxford Machine Learning in NeuroImaging Lab, Department of Computer Science, University of Oxford（牛津大学计算机科学系）；Visual Geometry Group, Department of Engineering Science, University of Oxford（牛津大学工程科学系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算科学与数据科学学院）
Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；NAAMII（NAAMII研究所）
Manual estimation of fetal Head Circumference (HC) from Ultrasound (US) is a key biometric for monitoring the healthy development of fetuses. Unfortunately, such measurements are subject to large inter-observer variability, resulting in low early-detection rates of fetal abnormalities. To address this issue, we propose a novel probabilistic Deep Learning approach for real-time automated estimation of fetal HC. This system feeds back statistics on measurement robustness to inform users how confident a deep neural network is in evaluating suitable views acquired during free-hand ultrasound exami…
Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学系）；National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, Shenzhen University（深圳大学医学部生物医学工程学院）
Point-of-Care Real-Time Signal Quality for Fetal Doppler Ultrasound Using a Deep Learning Approach
Emory University（埃默里大学）；Georgia Institute of Technology（佐治亚理工学院）；Center for Indigenous Health Research, Wuqu’ Kawoq — Maya Health Alliance（原住民健康研究中心，Wuqu’ Kawoq — 玛雅健康联盟）
3D ultrasound (US) can facilitate detailed prenatal examinations for fetal growth monitoring. To analyze a 3D US volume, it is fundamental to identify anatomical landmarks of the evaluated organs accurately. Typical deep learning methods usually regress the coordinates directly or involve heatmap-matching. However, these methods struggle to deal with volumes with large sizes and the highly-varying positions and orientations of fetuses. In this work, we exploit an object detection framework to detect landmarks in 3D fetal facial US volumes. By regressing multiple parameters of the landmark-cent…
King’s College London（伦敦国王学院）；Imperial College London（帝国理工学院）
Ultrasound (US) is the most widely used fetal imaging technique. However, US images have limited capture range, and suffer from view dependent artefacts such as acoustic shadows. Compounding of overlapping 3D US acquisitions into a high-resolution volume can extend the field of view and remove image artefacts, which is useful for retrospective analysis including population based studies. However, such volume reconstructions require information about relative transformations between probe positions from which the individual volumes were acquired. In prenatal US scans, the fetus can move indepen…
Fetal ultrasounds are an essential part of prenatal care and can be used to estimate gestational age (GA). Accurate GA assessment is important for providing appropriate prenatal care throughout pregnancy and identifying complications such as fetal growth disorders. Since derivation of GA from manual fetal biometry measurements (head, abdomen, femur) are operator-dependent and time-consuming, there have been a number of research efforts focused on using artificial intelligence (AI) models to estimate GA using standard biometry images, but there is still room to improve the accuracy and reliabil…
Hamad Bin Khalifa University（哈马德·本·哈利法大学）；Sidra Medical and Research Center（西德拉医疗与研究中心）；University Sains Malaysia（马来西亚理科大学）
Developing innovative informatics approaches aimed to enhance fetal monitoring is a burgeoning field of study in reproductive medicine. Several reviews have been conducted regarding Artificial intelligence (AI) techniques to improve pregnancy outcomes. They are limited by focusing on specific data such as mother's care during pregnancy. This systematic survey aims to explore how artificial intelligence (AI) can assist with fetal growth monitoring via Ultrasound (US) image. We used eight medical and computer science bibliographic databases, including PubMed, Embase, PsycINFO, ScienceDirect, IEE…
Deep learning models with large learning capacities often overfit to medical imaging datasets. This is because training sets are often relatively small due to the significant time and financial costs incurred in medical data acquisition and labelling. Data augmentation is therefore often used to expand the availability of training data and to increase generalization. However, augmentation strategies are often chosen on an ad-hoc basis without justification. In this paper, we present an augmentation policy search method with the goal of improving model classification performance. We include in …
University of Oxford（牛津大学）；Leiden University Medical Center（莱顿大学医学中心）；Shanghai Jiao Tong University（上海交通大学）
Two-dimensional (2D) freehand ultrasound is the mainstay in prenatal care and fetal growth monitoring. The task of matching corresponding cross-sectional planes in the 3D anatomy for a given 2D ultrasound brain scan is essential in freehand scanning, but challenging. We propose AdLocUI, a framework that Adaptively Localizes 2D Ultrasound Images in the 3D anatomical atlas without using any external tracking sensor.. We first train a convolutional neural network with 2D slices sampled from co-aligned 3D ultrasound volumes to predict their locations in the 3D anatomical atlas. Next, we fine-tune …
Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound（国家区域医疗超声关键技术工程实验室）；Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging（广东省生物医学测量与超声成像重点实验室）；School of Biomedical Engineering, Health Science Center, Shenzhen University（深圳大学医学部生物医学工程学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学系）；Department of Ultrasound, Luohu People’s Hospital（深圳市罗湖区人民医院超声科）
Anatomical landmarks are a crucial prerequisite for many medical imaging tasks. Usually, the set of landmarks for a given task is predefined by experts. The landmark locations for a given image are then annotated manually or via machine learning methods trained on manual annotations. In this paper, in contrast, we present a method to automatically discover and localize anatomical landmarks in medical images. Specifically, we consider landmarks that attract the visual attention of humans, which we term visually salient landmarks. We illustrate the method for fetal neurosonographic images. First…
Recent advances in deep learning have achieved promising performance for medical image analysis, while in most cases ground-truth annotations from human experts are necessary to train the deep model. In practice, such annotations are expensive to collect and can be scarce for medical imaging applications. Therefore, there is significant interest in learning representations from unlabelled raw data. In this paper, we propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video without any type of human annotation. We assume that in …
Image representations are commonly learned from class labels, which are a simplistic approximation of human image understanding. In this paper we demonstrate that transferable representations of images can be learned without manual annotations by modeling human visual attention. The basis of our analyses is a unique gaze tracking dataset of sonographers performing routine clinical fetal anomaly screenings. Models of sonographer visual attention are learned by training a convolutional neural network (CNN) to predict gaze on ultrasound video frames through visual saliency prediction or gaze-poin…
