Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for instance, for US-MRI registration or fusion. Our model is self-supervised and end-to-end trainable. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise an extractor to determine shared latent features, which are then used for data synthesis. Since paired data was unavailable for our study (and rare in practice), we propose to enforce the distributions to be similar instead of employing pixel-wise constraints, by adversarial learning in both the image domain and latent space. Furthermore, we propose an adversarial structural constraint to regularise the anatomical structures between the two modalities during the synthesis. A cross-modal attention scheme is proposed to leverage non-local spatial correlations. The feasibility of the approach to produce realistic looking MR images is demonstrated quantitatively and with a qualitative evaluation compared to real fetal MR images.
University of Oxford（牛津大学）；Tel-Aviv University（特拉维夫大学）
Self-supervised contrastive representation learning offers the advantage of learning meaningful visual representations from unlabeled medical datasets for transfer learning. However, applying current contrastive learning approaches to medical data without considering its domain-specific anatomical characteristics may lead to visual representations that are inconsistent in appearance and semantics. In this paper, we propose to improve visual representations of medical images via anatomy-aware contrastive learning (AWCL), which incorporates anatomy information to augment the positive/negative pa…
MIT（麻省理工学院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Fetal MRI is heavily constrained by unpredictable and substantial fetal motion that causes image artifacts and limits the set of viable diagnostic image contrasts. Current mitigation of motion artifacts is predominantly performed by fast, single-shot MRI and retrospective motion correction. Estimation of fetal pose in real time during MRI stands to benefit prospective methods to detect and mitigate fetal motion artifacts where inferred fetal motion is combined with online slice prescription with low-latency decision making. Current developments of deep reinforcement learning (DRL), offer a nov…
Aligning Fetal Anatomy with Kinematic Tree Log-Euclidean PolyRigid Transforms
Computer Science and Artificial Intelligence Lab, MIT, Cambridge, USA（麻省理工学院计算机科学与人工智能实验室）；California Institute of Technology, Pasadena, USA（加州理工学院）；Boston Children’s Hospital and Harvard Medical School, Boston, USA（哈佛医学院波士顿儿童医院）；The Hebrew University of Jerusalem, Jerusalem, Israel（耶路撒冷希伯来大学）
The Challenge of Fetal Cardiac MRI Reconstruction Using Deep Learning
King’s College London（伦敦国王学院）；Technical University of Munich（慕尼黑工业大学）；Imperial College London（帝国理工学院）；Guy’s & St. Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健服务基金会信托）；Evelina London Children’s Hospital（伊夫琳娜伦敦儿童医院）
DCRA-Net: Attention-Enabled Reconstruction Model for Dynamic Fetal Cardiac MRI
King’s College London（伦敦国王学院）；Evelina London Children’s Hospital（伊芙琳娜伦敦儿童医院）；Imperial College London（帝国理工学院）；Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院）
Pose-GuideNet: Automatic Scanning Guidance for Fetal Head Ultrasound from Pose Estimation
Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Nuffield Department of Women’s & Reproductive Health, University of Oxford（牛津大学纳菲尔德妇女与生殖健康系）
AtlasSeg: Atlas Prior Guided Dual-U-Net for Cortical Segmentation in Fetal Brain MRI
College of Biomedical Engineering & Instrument Science, Zhejiang University（浙江大学生物医学工程与仪器科学学院）；Beijing Hospital, National Center of Gerontology, Institute of Geriatric Medicine, Chinese Academy of Medical Sciences（中国医学科学院北京医院国家老年医学中心老年医学研究所）；Shandong Provincial Hospital Affiliated to Shandong First Medical University（山东第一医科大学附属山东省立医院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）
VoxelCloud Inc（体素科技公司）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Shanghai Jiao Tong University（上海交通大学）
Deep learning models tend to underperform in the presence of domain shifts. Domain transfer has recently emerged as a promising approach wherein images exhibiting a domain shift are transformed into other domains for augmentation or adaptation. However, with the absence of paired and annotated images, models merely learned by adversarial loss and cycle consistency loss could result in poor consistency of anatomy structures during the translation. Additionally, the complexity of learning multi-domain transfer could significantly increase with the number of target domains and source images. In t…
Semi-supervised learning methods have achieved excellent performance on standard benchmark datasets using very few labelled images. Anatomy classification in fetal 2D ultrasound is an ideal problem setting to test whether these results translate to non-ideal data. Our results indicate that inclusion of a challenging background class can be detrimental and that semi-supervised learning mostly benefits classes that are already distinct, sometimes at the expense of more similar classes.
School of Biomedical Engineering & Imaging Sciences, King’s College London, UK（伦敦国王学院生物医学工程与成像科学学院）；Department of Radiology, University Hospitals Leuven, Belgium（鲁汶大学医院放射科）；Institute for Women’s Health, University College London, UK（伦敦大学学院妇女健康研究所）；Department of Obstetrics and Gynaecology, University Hospitals Leuven, Belgium（鲁汶大学医院妇产科）；Department of Gynecology and Obstetrics, University Hospitals Tanta, Egypt（坦塔大学医院妇产科）；Department of Biomedical Imaging and Image-guided Therapy Medical University of Vienna, Austria（维也纳医科大学生物医学成像与图像引导治疗系）
The performance of deep neural networks typically increases with the number of training images. However, not all images have the same importance towards improved performance and robustness. In fetal brain MRI, abnormalities exacerbate the variability of the developing brain anatomy compared to non-pathological cases. A small number of abnormal cases, as is typically available in clinical datasets used for training, are unlikely to fairly represent the rich variability of abnormal developing brains. This leads machine learning systems trained by maximizing the average performance to be biased t…
HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation
University of Information Technology（信息科技大学）；Ho Chi Minh University of Technology（胡志明市技术大学）；University of Science（科学大学）；Vietnam National University（越南国家大学）；Boston University（波士顿大学）；Taipei Medical University（台北医学大学）；Posts and Telecommunications Institute of Technology（邮电技术学院）；New Mexico State University（新墨西哥州立大学）
Structure-Preserving Synthesis: MaskGAN for Unpaired MR-CT Translation
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Flinders Health and Medical Research Institute, Flinders University（弗林德斯大学弗林德斯健康与医学研究所）
Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）
Fetal heart motion is an important diagnostic indicator for structural detection and functional assessment of congenital heart disease. We propose an approach towards integrating deep convolutional and recurrent architectures that utilize localized spatial and temporal features of different anatomical substructures within a global spatiotemporal context for interpretation of fetal echocardiography videos. We formulate our task as a cardiac structure localization problem with convolutional architectures for aggregating global spatial context and detecting anatomical structures on spatial region…
ContourDiff: Unpaired Medical Image Translation with Structural Consistency
Department of Electrical and Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Department of Biostatistics & Bioinformatics, Duke University（杜克大学生物统计学与生物信息学系）；Department of Radiology, Duke University（杜克大学放射学系）；Department of Computer Science, Duke University（杜克大学计算机科学系）
Recent advances in deep learning have achieved promising performance for medical image analysis, while in most cases ground-truth annotations from human experts are necessary to train the deep model. In practice, such annotations are expensive to collect and can be scarce for medical imaging applications. Therefore, there is significant interest in learning representations from unlabelled raw data. In this paper, we propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video without any type of human annotation. We assume that in …
Imperial College London（帝国理工学院）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）；Southest University（东南大学）；Nankai University（南开大学）
Although chest X-ray (CXR) offers a 2D projection with overlapped anatomies, it is widely used for clinical diagnosis. There is clinical evidence supporting that decomposing an X-ray image into different components (e.g., bone, lung and soft tissue) improves diagnostic value. We hereby propose a decomposition generative adversarial network (DecGAN) to anatomically decompose a CXR image but with unpaired data. We leverage the anatomy knowledge embedded in CT, which features a 3D volume with clearly visible anatomies. Our key idea is to embed CT priori decomposition knowledge into the latent spa…
University College London（伦敦大学学院）；Elizabeth Garrett Anderson Institute for Women’s Health, University College London（伦敦大学学院伊丽莎白·加勒特·安德森妇女健康研究所）；NIHR University College London Hospitals Biomedical Research Centre（英国国家健康研究所伦敦大学学院医院生物医学研究中心）
During pregnancy, ultrasound examination in the second trimester can assess fetal size according to standardized charts. To achieve a reproducible and accurate measurement, a sonographer needs to identify three standard 2D planes of the fetal anatomy (head, abdomen, femur) and manually mark the key anatomical landmarks on the image for accurate biometry and fetal weight estimation. This can be a time-consuming operator-dependent task, especially for a trainee sonographer. Computer-assisted techniques can help in automating the fetal biometry computation process. In this paper, we present a uni…
Foundation Model-driven Key Anatomy Frame Selection for Blind-sweep Ultrasound Fetal Birth Weight Estimation
Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University（深圳大学医学超声图像计算实验室）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；School of Biomedical Engineering and Informatics, Nanjing Medical University（南京医科大学生物医学工程与信息学院）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；Shandong Provincial Maternal and Child Health Care Hospital Affiliated to Qingdao University（青岛大学附属山东省妇幼保健院）；Jiangxi Maternal and Child Health Hospital（江西省妇幼保健院）；School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）
Z 2 Sky Technologies Inc.（Z 2 天工科技公司）；University of Rochester（罗切斯特大学）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院计算技术研究所 中国科学院智能信息处理重点实验室）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advance Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）
Spinal surgery planning necessitates automatic segmentation of vertebrae in cone-beam computed tomography (CBCT), an intraoperative imaging modality that is widely used in intervention. However, CBCT images are of low-quality and artifact-laden due to noise, poor tissue contrast, and the presence of metallic objects, causing vertebra segmentation, even manually, a demanding task. In contrast, there exists a wealth of artifact-free, high quality CT images with vertebra annotations. This motivates us to build a CBCT vertebra segmentation model using unpaired CT images with annotations. To overco…
Anatomy-preserving unpaired cone-beam CT refinement for image-guided radiotherapy using pseudo-label guided diffusion
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
Shadow Reduction in Ultrasound Imaging Using Differentiable Simulation and Radiance Field Decomposition
University of Oxford（牛津大学）；University of Amsterdam（阿姆斯特丹大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔朗根-纽伦堡大学）；Imperial College London（伦敦帝国理工学院）；University of Copenhagen（哥本哈根大学）
University of Electronic Science and Technology of China（电子科技大学）；SenseTime Research（商汤科技研究院）
Despite that deep learning has achieved state-of-the-art performance for medical image segmentation, its success relies on a large set of manually annotated images for training that are expensive to acquire. In this paper, we propose an annotation-efficient learning framework for segmentation tasks that avoids annotations of training images, where we use an improved Cycle-Consistent Generative Adversarial Network (GAN) to learn from a set of unpaired medical images and auxiliary masks obtained either from a shape model or public datasets. We first use the GAN to generate pseudo labels for our …
Origin Health India（印度Origin Health）；Bangalore Fetal Medicine Center（班加罗尔胎儿医学中心）；Prime Imaging and Prenatal Diagnostics（Prime影像与产前诊断中心）；Kasturba Medical College, Manipal Academy of Higher Education（马尼帕尔高等教育学院卡斯特尔巴医学院）；Origin Health；Universiti Malaya（马来亚大学）；Aarthi Scans and Labs（Aarthi扫描与实验室）
Quality assessment of prenatal ultrasonography is essential for the screening of fetal central nervous system (CNS) anomalies. The interpretation of fetal brain structures is highly subjective, expertise-driven, and requires years of training experience, limiting quality prenatal care for all pregnant mothers. With recent advancement in Artificial Intelligence (AI), specifically deep learning (DL), assistance in precise anatomy identification through semantic segmentation essential for the reliable assessment of growth and neurodevelopment, and detection of structural abnormalities have been p…
Biologically Inspired Deep Learning Approaches for Fetal Ultrasound Image Classification
Skolkovo Institute of Science and Technology（斯克尔科沃科学与技术研究所）；BIMAI-Lab, Biomedically Informed Artificial Intelligence Laboratory, University of Sharjah（沙迦大学BIMAI实验室，生物医学引导人工智能实验室）
Efficient Feature Extraction Using Light-Weight CNN Attention-Based Deep Learning Architectures for Ultrasound Fetal Plane Classification
Amrita School of Artificial Intelligence（阿米特学校人工智能学院）；Amrita Vishwa Vidyapeetham（阿米特世界大学）；Center for Artificial Intelligence（人工智能中心）；Prince Mohammed Bin Fahd University（普里姆·穆罕默德·本·法赫德大学）
International Institute of Information Technology（国际信息技术研究所）
Biological processes like growth, aging, and disease progression are generally studied with follow-up scans taken at different time points, i.e., with image time series (TS) based analysis. Comparison between TS representing a biological process of two individuals/populations is of interest. A metric to quantify the difference between TS is desirable for such a comparison. The two TS represent the evolution of two different subject/population average anatomies through two paths. A method to untangle and quantify the path and inter-subject anatomy(shape) difference between the TS is presented i…
Accurate and Efficient Fetal Birth Weight Estimation from 3D Ultrasound
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound（医学超声技术国家-地区关键工程技术实验室）；School of Biomedical Engineering（生物医学工程学院）；College of Computer Science and Software Engineering（计算机科学与软件工程学院）；Medical Ultrasound lmage Computing (MUSIC) Lab（医学超声图像计算（MUSIC）实验室）；School of Medicine（医学院）；The First Affiliated Hospital of Xiamen University（厦门大学第一附属医院）；Shenzhen Baoan Women's and Children's Hospital（深圳宝安妇幼医院）；University of California, Berkeley（加州大学伯克利分校）；Yongkang Maternal and Child Health Hospital（永康 maternal and child health hospital）；Shenzhen RayShape Medical Technology Co., Ltd（深圳RayShape医疗科技有限公司）
Identifying and interpreting fetal standard scan planes during 2D ultrasound mid-pregnancy examinations are highly complex tasks which require years of training. Apart from guiding the probe to the correct location, it can be equally difficult for a non-expert to identify relevant structures within the image. Automatic image processing can provide tools to help experienced as well as inexperienced operators with these tasks. In this paper, we propose a novel method based on convolutional neural networks which can automatically detect 13 fetal standard views in freehand 2D ultrasound data as we…
Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）
Standard scan plane detection in fetal brain ultrasound (US) forms a crucial step in the assessment of fetal development. In clinical settings, this is done by manually manoeuvring a 2D probe to the desired scan plane. With the advent of 3D US, the entire fetal brain volume containing these standard planes can be easily acquired. However, manual standard plane identification in 3D volume is labour-intensive and requires expert knowledge of fetal anatomy. We propose a new Iterative Transformation Network (ITN) for the automatic detection of standard planes in 3D volumes. ITN uses a convolutiona…
A large amount of manual segmentation is typically required to train a robust segmentation network so that it can segment objects of interest in a new imaging modality. The manual efforts can be alleviated if the manual segmentation in one imaging modality (e.g., CT) can be utilized to train a segmentation network in another imaging modality (e.g., CBCT/MRI/PET). In this work, we developed an anatomy-constrained contrastive synthetic segmentation network (AccSeg-Net) to train a segmentation network for a target imaging modality without using its ground truth. Specifically, we proposed to use a…
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；Northeastern University（东北大学）
With an aim to increase the capture range and accelerate the performance of state-of-the-art inter-subject and subject-to-template 3D registration, we propose deep learning-based methods that are trained to find the 3D position of arbitrarily oriented subjects or anatomy based on slices or volumes of medical images. For this, we propose regression CNNs that learn to predict the angle-axis representation of 3D rotations and translations using image features. We use and compare mean square error and geodesic loss to train regression CNNs for 3D pose estimation used in two different scenarios: sl…
Dual Conditioned Diffusion Models for Out-Of-Distribution Detection: Application to Fetal Ultrasound Videos
Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Nuffield Department of Women’s and Reproductive Health, University of Oxford（牛津大学纳菲尔德妇女与生殖健康系）
Robustness and Diagnostic Performance of Super-Resolution Fetal Brain MRI
Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia（卢布尔雅那大学医学学院）；University Medical Centre Ljubljana, Ljubljana, Slovenia（卢布尔雅那大学医学中心）；Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia（卢布尔雅那大学电气工程学院）
Biomedical Image Analysis Group, Imperial College London, UK（帝国理工学院伦敦生物医学图像分析组）
Real-world settings often do not allow acquisition of high-resolution volumetric images for accurate morphological assessment and diagnostic. In clinical practice it is frequently common to acquire only sparse data (e.g. individual slices) for initial diagnostic decision making. Thereby, physicians rely on their prior knowledge (or mental maps) of the human anatomy to extrapolate the underlying 3D information. Accurate mental maps require years of anatomy training, which in the first instance relies on normative learning, i.e. excluding pathology. In this paper, we leverage Bayesian Deep Learn…
Trusted Multi-View Deep Learning Classification of Fetal Congenital Heart Disease with Feature-level and Decision-level Fusion
School of Biomedical Engineering, Shanghai Jiao Tong University（生物医学工程学院，上海交通大学）；Shanghai Key Laboratory of Embryo Original Diseases（上海胚胎原发疾病重点实验室）；Shanghai Municipal Key Clinical Specialty（上海市级重点临床专科）
FUSEP: A Multi-Center Benchmark for Diverse Tasks in Early Pregnancy Fetal Ultrasound Screening
Hunan University（湖南大学）；The Hong Kong University of Science and Technology（香港科技大学）；Anhui University（安徽大学）；Shenzhen Maternity and Child Healthcare Hospital（深圳市妇幼保健院）；Sichuan Provincial Maternity and Child Health Care Hospital（四川省妇幼保健院）；National University of Singapore（新加坡国立大学）；Yunnan Maternal and Child Health Hospital（云南省妇幼保健院）；Zhejiang University（浙江大学）
Task-Adaptive 3D Cross-Field MRI Translation via Field-Conditioned Content-Style Pretraining
School of Integrated Circuits and Electronics, Beijing Institute of Technology（北京理工大学集成电路与电子学院）；School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University（清华大学医学院生物医学工程学院）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）
Hamad Bin Khalifa University（哈马德·本·哈利法大学）；Sidra Medical and Research Center（西德拉医疗与研究中心）；University Sains Malaysia（马来西亚理科大学）
Developing innovative informatics approaches aimed to enhance fetal monitoring is a burgeoning field of study in reproductive medicine. Several reviews have been conducted regarding Artificial intelligence (AI) techniques to improve pregnancy outcomes. They are limited by focusing on specific data such as mother's care during pregnancy. This systematic survey aims to explore how artificial intelligence (AI) can assist with fetal growth monitoring via Ultrasound (US) image. We used eight medical and computer science bibliographic databases, including PubMed, Embase, PsycINFO, ScienceDirect, IEE…
University of Oxford（牛津大学）；Leiden University Medical Center（莱顿大学医学中心）；Shanghai Jiao Tong University（上海交通大学）
Two-dimensional (2D) freehand ultrasound is the mainstay in prenatal care and fetal growth monitoring. The task of matching corresponding cross-sectional planes in the 3D anatomy for a given 2D ultrasound brain scan is essential in freehand scanning, but challenging. We propose AdLocUI, a framework that Adaptively Localizes 2D Ultrasound Images in the 3D anatomical atlas without using any external tracking sensor.. We first train a convolutional neural network with 2D slices sampled from co-aligned 3D ultrasound volumes to predict their locations in the 3D anatomical atlas. Next, we fine-tune …
IHF-Harmony: Multi-Modality Magnetic Resonance Images Harmonization using Invertible Hierarchy Flow Model
Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong（健康科技与信息技术系，香港理工大学，香港）；School of Integrated Circuits and Electronics, Beijing Institute of Technology, China（集成电路与电子学院，北京理工大学，中国）；Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong（心理健康研究中心，香港理工大学，香港）；Department of Biomedical Engineering, Johns Hopkins University, USA（生物医学工程系，约翰霍普金斯大学，美国）
Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field of medical image-to-image translation. However, neither modes are ideal. The Pix2Pix mode has excellent performance. But it requires paired and well pixel-wise aligned images, which may not always be achievable due to respiratory motion or anatomy change between times that paired images are acquired. The Cycle-consistency mode is less stringent with training data and works well on unpaired or misaligned images. But its performance may not be optimal. In order to break the dilemma of the existing modes, …
