Cardiac ultrasound simulation for autonomous ultrasound navigation
King’s College London（伦敦国王学院）；Siemens Healthcare Limited（西门子医疗有限公司）；Siemens Healthineers（西门子医疗）；Siemens Healthineers AG（西门子医疗股份公司）
Abdoul Aziz Amadou, Laura Peralta, Paul Dryburgh, Paul Klein, Kaloian Petkov, Richard James Housden, Vivek Singh, Rui Liao, Young-Ho Kim, Florin Christian Ghesu… 展开作者
Abdoul Aziz Amadou, Laura Peralta, Paul Dryburgh, Paul Klein, Kaloian Petkov, Richard James Housden, Vivek Singh, Rui Liao, Young-Ho Kim, Florin Christian Ghesu, Tommaso Mansi, Ronak Rajani, Alistair Young, Kawal Rhode
Ultrasound is well-established as an imaging modality for diagnostic and interventional purposes. However, the image quality varies with operator skills as acquiring and interpreting ultrasound images requires extensive training due to the imaging artefacts, the range of acquisition parameters and the variability of patient anatomies. Automating the image acquisition task could improve acquisition reproducibility and quality but training such an algorithm requires large amounts of navigation data, not saved in routine examinations. Thus, we propose a method to generate large amounts of ultrasound images from other modalities and from arbitrary positions, such that this pipeline can later be used by learning algorithms for navigation. We present a novel simulation pipeline which uses segmentations from other modalities, an optimized volumetric data representation and GPU-accelerated Monte Carlo path tracing to generate view-dependent and patient-specific ultrasound images. We extensively validate the correctness of our pipeline with a phantom experiment, where structures' sizes, contrast and speckle noise properties are assessed. Furthermore, we demonstrate its usability to train neural networks for navigation in an echocardiography view classification experiment by generating synthetic images from more than 1000 patients. Networks pre-trained with our simulations achieve significantly superior performance in settings where large real datasets are not available, especially for under-represented classes. The proposed approach allows for fast and accurate patient-specific ultrasound image generation, and its usability for training networks for navigation-related tasks is demonstrated.
EchoFlow: A Foundation Model for Cardiac Ultrasound Image and Video Generation
Imperial College London（帝国理工学院）；University of Birmingham（伯明翰大学）；Ultromics Ltd.（奥特罗米克斯有限公司）；University of Oxford（牛津大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning
organization= Concordia Institute for Information Systems Engineering, Concordia University , city= Montreal , country= Canada；organization= Department of Mechanical, Industrial；Aerospace Engineering, Concordia University , city= Montreal , country= Canada；organization= Department of Medicine, Laval University , city= Quebec , country= Canada；organization= Department of Electrical, Computer；Software Engineering, Ontario Tech University , city= Oshawa , country= Canada；organization= Department of Computer Science, Khalifa University , city= Abu Dhabi , country= UAE；organization= Artificial Intelligence \& Cyber Systems Research Center, Department of CSM, Lebanese American University , city= Beirut , country= Lebanon；organization= Department of Electrical Engineering；Computer Science, Khalifa University , city= Abu Dhabi , country= UAE
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；National University of Singapore（新加坡国立大学）
Gaze-Assisted Human-Centric Domain Adaptation for Cardiac Ultrasound Image Segmentation
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
Romanian Institute of Science and Technology（罗马尼亚科学技术研究所）；University of Medicine and Pharmacy of Craiova（克拉约瓦医药大学）
The present paper puts forward an incipient study that uses a traditional segmentation method based on Zernike moments for extracting significant features from frames of fetal echocardiograms from first trimester color Doppler examinations. A distance based approach is then used on the obtained indicators to classify frames of three given categories that should be present in a normal heart condition. The computational tool shows promise in supporting the obstetrician in a rapid recognition of heart views during screening.
Imperial College London（伦敦帝国学院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希·亚历山大大学）
This paper presents the use of Multi-Agent Reinforcement Learning (MARL) to perform navigation in 3D anatomical volumes from medical imaging. We utilize Neural Style Transfer to create synthetic Computed Tomography (CT) agent gym environments and assess the generalization capabilities of our agents to clinical CT volumes. Our framework does not require any labelled clinical data and integrates easily with several image translation techniques, enabling cross modality applications. Further, we solely condition our agents on 2D slices, breaking grounds for 3D guidance in much more difficult imagi…
GE Vingmed Ultrasound, GE Healthcare（通用电气医疗集团GE Vingmed超声部门）；University of Oslo（奥斯陆大学）；Norwegian Computing Center（挪威计算中心）
Two-dimensional echocardiography (2DE) measurements of left ventricle (LV) dimensions are highly significant markers of several cardiovascular diseases. These measurements are often used in clinical care despite suffering from large variability between observers. This variability is due to the challenging nature of accurately finding the correct temporal and spatial location of measurement endpoints in ultrasound images. These images often contain fuzzy boundaries and varying reflection patterns between frames. In this work, we present a convolutional neural network (CNN) based approach to aut…
Imperial College London（伦敦帝国学院）；King’s College London（伦敦国王学院）；St Thomas’ Hospital（圣托马斯医院）
Prenatal screening with ultrasound can lower neonatal mortality significantly for selected cardiac abnormalities. However, the need for human expertise, coupled with the high volume of screening cases, limits the practically achievable detection rates. In this paper we discuss the potential for deep learning techniques to aid in the detection of congenital heart disease (CHD) in fetal ultrasound. We propose a pipeline for automated data curation and classification. During both training and inference, we exploit an auxiliary view classification task to bias features toward relevant cardiac stru…
Benchmarking Self-Supervised Models for Cardiac Ultrasound View Classification
Department of Systems；Computer Engineering, Carleton University, Ottawa, Ontario, Canada Department of Methodological；Implementation Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada Department of Acute Care Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada Department of Obstetrics；Gynecology, University of Ottawa, Ottawa, Ontario, Canada School of Epidemiology；Public Health, University of Ottawa, Ottawa, Ontario, Canada Department of Obstetrics, Gynecology \& Newborn Care, The Ottawa Hospital, Ottawa, Ontario, Canada International；Global Health Office, University of Ottawa, Ottawa, Ontario, Canada College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound（医学超声关键技术研发实验室）；School of Biomedical Engineering, Medical School, Shenzhen University（生物医学工程学院）；Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院）；Centre for CIMIM, Manchester University（CIMIM研究中心）；Center for Cardiovascular Disease（心血管疾病中心）；The Affiliated Suzhou Hospital of Nanjing Medical University（南京医科大学附属苏州医院）；Suzhou Municipal Hospital, Gusu School, Suzhou, China（苏州 municipal 医院）；The First Affiliated Hospital of Kunming Medical University（昆明医科大学第一附属医院）；The First Affiliated Hospital of Anhui Medical University North（安徽医科大学北院）；The Second People’s Hospital（第二人民医院）；The First Affiliated Hospital of Anhui Medical University（安徽医科大学第一附属医院）；The People's Hospital of Tongren City（铜仁市人民医院）；The First People's Hospital of Taicang City（太仓市第一人民医院）；Guangdong Provincial People’s Hospital（广东省人民医院）
DreamReg: Belief-Driven World Model for 2D-3D Ultrasound Registration
T Stone Robotics Institute, The Chinese University of Hong Kong（香港中文大学T Stone机器人研究所）；Multi-scale Medical Robotics Center（多尺度医疗机器人中心）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
King’s College London（伦敦国王学院）；Ultromics Ltd.（Ultromics有限公司）
Accurate geometric quantification of the human heart is a key step in the diagnosis of numerous cardiac diseases, and in the management of cardiac patients. Ultrasound imaging is the primary modality for cardiac imaging, however acquisition requires high operator skill, and its interpretation and analysis is difficult due to artifacts. Reconstructing cardiac anatomy in 3D can enable discovery of new biomarkers and make imaging less dependent on operator expertise, however most ultrasound systems only have 2D imaging capabilities. We propose both a simple alteration to the Pix2Vox++ networks fo…
The Hong Kong Polytechnic University（香港理工大学）；Research Institute for Smart Ageing, PolyU（香港理工大学智衰研究所）
The scoliosis progression in adolescents requires close monitoring to timely take treatment measures. Ultrasound imaging is a radiation-free and low-cost alternative in scoliosis assessment to X-rays, which are typically used in clinical practice. However, ultrasound images are prone to speckle noises, making it challenging for sonographers to detect bony features and follow the spine's curvature. This paper introduces a robotic-ultrasound approach for spinal curvature tracking and automatic navigation. A fully connected network with deconvolutional heads is developed to locate the spinous pro…
Imperial College London（帝国理工学院）；Friedrich–Alexander University Erlangen–Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Spotify（Spotify公司）；University College London（伦敦大学学院）；Ultromics Ltd（Ultromics有限公司）；King’s College London（伦敦国王学院）；John Radcliffe Hospital（约翰·拉德克利夫医院）
Causally-enabled machine learning frameworks could help clinicians to identify the best course of treatments by answering counterfactual questions. We explore this path for the case of echocardiograms by looking into the variation of the Left Ventricle Ejection Fraction, the most essential clinical metric gained from these examinations. We combine deep neural networks, twin causal networks and generative adversarial methods for the first time to build D'ARTAGNAN (Deep ARtificial Twin-Architecture GeNerAtive Networks), a novel causal generative model. We demonstrate the soundness of our approac…
Compositional Segmentation of Cardiac Images Leveraging Metadata
School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）；Queen Mary’s Digital Environment Research Institute (DERI)（玛丽女王数字环境研究所）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；School of Engineering and Materials Science, Queen Mary University of London（伦敦玛丽女王大学工程与材料科学学院）
Myocardial Strain Drift Correction in Deep Learning Based Ultrasound Tracking
University of Sherbrooke（舍布鲁克大学）；INSA（法国国立应用科学学院）；Université Lyon 1（里昂第一大学）；CREATIS（CREATIS实验室）；Institut Universitaire de France (IUF)（法国大学研究院（IUF））；NTNU（挪威科技大学）；St. Olavs Hospital（圣奥拉夫医院）；SINTEF Digital（SINTEF数字研究所）；Levanger Hospital（莱旺厄尔医院）；Nord-Trøndelag Hospital Trust（北特伦德拉格医院信托）；Hôpital Croix-Rousse（克鲁瓦鲁斯医院）；Hospices Civils de Lyon（里昂民用医院集团）；Hôpital Lyon Sud（里昂南部医院）
Imperial College London（伦敦帝国学院）；City, University of London（伦敦城市大学）；University of Birmingham（伯明翰大学）
Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the chall…
Atrial Septal Defect Detection in Children Based on Ultrasound Video Using Multiple Instances Learning
Shanghai Children’s Medical Center, School of Medicine, Shanghai Jiaotong University（上海交通大学医学院附属上海儿童医学中心）；Minhang District Centers for Disease Control and Prevention（闵行区疾病预防控制中心）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；East China Normal University（华东师范大学）；University of Shanghai for Science and Technology（上海理工大学）
Applications of ultrasound images have expanded from fetal imaging to abdominal and cardiac diagnosis. Liver-being the largest gland in the body and responsible for metabolic activities requires to be to be diagnosed and therefore subject to utmost injury. Although, ultrasound imaging has developed into three and four dimensions providing higher amount of information; it requires highly trained medical staff due to the image complexity and dimensions it contain. Since 2D ultrasound images are still considered to be the basis of clinical treatments,computer aided automated liver diagnosis is ve…
University of Sherbrooke（舍布鲁克大学）；University of Bourgogne Franche-Comte（勃艮第-弗朗什孔泰大学）；CASIS inc.（CASIS公司）；University of Lyon（里昂大学）
Convolutional neural networks (CNN) have had unprecedented success in medical imaging and, in particular, in medical image segmentation. However, despite the fact that segmentation results are closer than ever to the inter-expert variability, CNNs are not immune to producing anatomically inaccurate segmentations, even when built upon a shape prior. In this paper, we present a framework for producing cardiac image segmentation maps that are guaranteed to respect pre-defined anatomical criteria, while remaining within the inter-expert variability. The idea behind our method is to use a well-trai…
Technion, Israel（以色列理工学院）；University of Waterloo, Canada（滑铁卢大学）
Medical ultrasound (US) is a widespread imaging modality owing its popularity to cost efficiency, portability, speed, and lack of harmful ionizing radiation. In this paper, we demonstrate that replacing the traditional ultrasound processing pipeline with a data-driven, learnable counterpart leads to significant improvement in image quality. Moreover, we demonstrate that greater improvement can be achieved through a learning-based design of the transmitted beam patterns simultaneously with learning an image reconstruction pipeline. We evaluate our method on an in-vivo first-harmonic cardiac ult…
University of Sherbrooke（舍布鲁克大学）；Univ Lyon（里昂大学）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Institut Universitaire de France（法国大学研究院）
Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axi…
China University of Petroleum（中国石油大学）；Qingdao Women and Children’s Hospital（青岛市妇女儿童医院）
Echocardiography is a powerful prenatal examination tool for early diagnosis of fetal congenital heart diseases (CHDs). The four-chamber (FC) view is a crucial and easily accessible ultrasound (US) image among echocardiography images. Automatic analysis of FC views contributes significantly to the early diagnosis of CHDs. The first step to automatically analyze fetal FC views is locating the fetal four crucial chambers of heart in a US image. However, it is a greatly challenging task due to several key factors, such as numerous speckles in US images, the fetal cardiac chambers with small size …
Current commercial matrix transducers for 3D DCE-US do not display side-by-side B-mode and contrast-mode images when capturing volumetric data, thus leaving the operator with no position feedback during lengthy acquisitions. The purpose of this study was to investigate the use of transducer tracking to provide positioning feedback and to re-align images to improve quantification. An interventional tracking system was developed in house using an infrared camera and a 3D-printed tracking target attached to a X6-1 matrix transducer. The system displays a virtual probe on a separate screen and all…
AGE-US: automated gestational age estimation based on fetal ultrasound images
Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS)（智能技术研究中心（CiTIUS））；Universidade de Santiago de Compostela（圣地亚哥大学）；Departamento de Electrónica e Computación, Escola Técnica Superior de Enxeñaría（电子与计算系，工程技术学校）；Universitat Pompeu Fabra（庞培法布拉大学）
Purpose. Given the high level of expertise required for navigation and interpretation of ultrasound images, computational simulations can facilitate the training of such skills in virtual reality. With ray-tracing based simulations, realistic ultrasound images can be generated. However, due to computational constraints for interactivity, image quality typically needs to be compromised. Methods. We propose herein to bypass any rendering and simulation process at interactive time, by conducting such simulations during a non-time-critical offline stage and then learning image translation from cro…
In the modern medical care, venipuncture is an indispensable procedure for both diagnosis and treatment. In this paper, unlike existing solutions that fully or partially rely on professional assistance, we propose VeniBot -- a compact robotic system solution integrating both novel hardware and software developments. For the hardware, we design a set of units to facilitate the supporting, positioning, puncturing and imaging functionalities. For the software, to move towards a full automation, we propose a novel deep learning framework -- semi-ResNeXt-Unet for semi-supervised vein segmentation f…
Testing of Deep Learning Model in Real World Clinical Setting: A Case Study in Obstetric Ultrasound
Technical University of Denmark（技术大学）；Pioneer Centre for AI（先锋人工智能中心）；University of Copenhagen（哥本哈根大学）；Slagelse Hospital（斯拉格尔塞医院）；CAMES Rigshospitalet（CAMES皇家医院）
TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research
University of Strasbourg（斯特拉斯堡大学）；Research Institute against Digestive Cancer (IRCAD)（消化癌症研究所（IRCAD））；Conservatoire National des Arts et Métiers (CNAM)（法国国立工艺学院（CNAM））；Sorbonne University（索邦大学）；Clinique Sainte-Anne（圣安妮诊所）；Clinique Saint Augustin（圣奥古斯丁诊所）；Institute of Image-Guided Surgery (IHU)（图像引导手术研究所（IHU））
