Deep Learning-Based Semantic Segmentation for Real-Time Kidney Imaging and Measurements with Augmented Reality-Assisted Ultrasound
Institute for Artificial Intelligence in Medicine (IKIM), Essen University Hospital (AöR), University of Duisburg-Essen（人工智能医学研究所（IKIM）、埃森大学医院（AöR）、杜伊斯堡-埃森大学）；Center for Virtual and Extended Reality in Medicine (ZvRM), University Hospital Essen (AöR)（医学虚拟与扩展现实中心（ZvRM）、埃森大学医院（AöR））；Institute of Computer Graphics and Vision (ICG), Graz University of Technology（计算机图形与视觉研究所（ICG）、格拉茨技术大学）；Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari（电气与信息工程系（DEI）、巴里理工学院）；Pediatric Clinic II, University Children’s Hospital Essen, University Duisburg-Essen（儿童医院第二儿科诊所、埃森儿童医院、杜伊斯堡-埃森大学）；Medical Faculty, University of Duisburg-Essen（医学系、杜伊斯堡-埃森大学）；Cancer Research Center Cologne Essen (CCCE), West German Cancer Center, University Hospital Essen (AöR)（科隆-埃森癌症研究中心（CCCE）、西德癌症中心、埃森大学医院（AöR））；German Cancer Consortium (DKTK), Partner site University Hospital Essen (AöR)（德国癌症联盟（DKTK）、合作伙伴站点埃森大学医院（AöR））；Technical University Dortmund, Faculty of Physics（多特蒙德技术大学、物理系）；Faculty of Computer Science, University of Duisburg-Essen（计算机科学系、杜伊斯堡-埃森大学）
Gijs Luijten, Roberto Maria Scardigno, Lisle Faray de Paiva, Peter Hoyer, Jens Kleesiek, Domenico Buongiorno, Vitoantonio Bevilacqua, Jan Egger
Ultrasound (US) is widely accessible and radiation-free but has a steep learning curve due to its dynamic nature and non-standard imaging planes. Additionally, the constant need to shift focus between the US screen and the patient poses a challenge. To address these issues, we integrate deep learning (DL)-based semantic segmentation for real-time (RT) automated kidney volumetric measurements, which are essential for clinical assessment but are traditionally time-consuming and prone to fatigue. This automation allows clinicians to concentrate on image interpretation rather than manual measurements. Complementing DL, augmented reality (AR) enhances the usability of US by projecting the display directly into the clinician's field of view, improving ergonomics and reducing the cognitive load associated with screen-to-patient transitions. Two AR-DL-assisted US pipelines on HoloLens-2 are proposed: one streams directly via the application programming interface for a wireless setup, while the other supports any US device with video output for broader accessibility. We evaluate RT feasibility and accuracy using the Open Kidney Dataset and open-source segmentation models (nnU-Net, Segmenter, YOLO with MedSAM and LiteMedSAM). Our open-source GitHub pipeline includes model implementations, measurement algorithms, and a Wi-Fi-based streaming solution, enhancing US training and diagnostics, especially in point-of-care settings.
Ultrasound, because of its low cost, non-ionizing, and non-invasive characteristics, has established itself as a cornerstone radiological examination. Research on ultrasound applications has also expanded, especially with image analysis with machine learning. However, ultrasound data are frequently restricted to closed data sets, with only a few openly available. Despite being a frequently examined organ, the kidney lacks a publicly available ultrasonography data set. The proposed Open Kidney Ultrasound Data Set is the first publicly available set of kidney brightness mode (B-mode) ultrasound …
Department of Radiology, School of Medicine, University of Pennsylvania（宾夕法尼亚大学医学院放射科）；School of Computer and Control Engineering, Yantai University（烟台大学计算机与控制工程学院）；The Children's Hospital of Philadelphia（费城儿童医院）
Purpose: To improve kidney segmentation in clinical ultrasound (US) images, we develop a new graph cuts based method to segment kidney US images by integrating original image intensity information and texture feature maps extracted using Gabor filters. Methods: To handle large appearance variation within kidney images and improve computational efficiency, we build a graph of image pixels close to kidney boundary instead of building a graph of the whole image. To make the kidney segmentation robust to weak boundaries, we adopt localized regional information to measure similarity between image p…
Pyramid Self-Contrastive Learning for Single-shot Test-time Ultrasound Image Denoising
Department of Electrical and Computer Engineering, The University of Hong Kong（香港大学电子与计算机工程系）；Department of Biomedical Engineering, Duke University（达特茅斯大学生物医学工程系）
We develop and approach to unsupervised semantic medical image segmentation that extends previous work with generative adversarial networks. We use existing edge detection methods to construct simple edge diagrams, train a generative model to convert them into synthetic medical images, and construct a dataset of synthetic images with known segmentations using variations on extracted edge diagrams. This synthetic dataset is then used to train a supervised image segmentation model. We test our approach on a clinical dataset of kidney ultrasound images and the benchmark ISIC 2018 skin lesion data…
TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research
University of Strasbourg（斯特拉斯堡大学）；Research Institute against Digestive Cancer (IRCAD)（消化癌症研究所（IRCAD））；Conservatoire National des Arts et Métiers (CNAM)（法国国立工艺学院（CNAM））；Sorbonne University（索邦大学）；Clinique Sainte-Anne（圣安妮诊所）；Clinique Saint Augustin（圣奥古斯丁诊所）；Institute of Image-Guided Surgery (IHU)（图像引导手术研究所（IHU））
Self-Supervised Ultrasound Representation Learning for Renal Anomaly Prediction in Prenatal Imaging
organization= Department of Systems；Computer Engineering, Carleton University , city= Ottawa , state= Ontario , country= Canada；organization= Department of Methodological；Implementation Research, Ottawa Hospital Research Institute , city= Ottawa , state= Ontario , country= Canada；organization= Department of Acute Care Research, Ottawa Hospital Research Institute , city= Ottawa , state= Ontario , country= Canada；organization= Children's Hospital of Eastern Ontario Research Institute , city= Ottawa , state= Ontario , country= Canada；organization= Better Outcomes Registry \& Network Ontario, Children’s Hospital of Eastern , city= Ottawa , state= Ontario , country= Canada；organization= Department of Obstetrics；Gynecology, University of Ottawa , city= Ottawa , state= Ontario , country= Canada；organization= School of Epidemiology；Public Health, University of Ottawa , city= Ottawa , state= Ontario , country= Canada；organization= Department of Obstetrics, Gynecology \& Newborn Care, The Ottawa Hospital , city= Ottawa , state= Ontario , country= Canada；Global Health Office, University of Ottawa , city= Ottawa , state= Ontario , country= Canada；organization= Department of Clinical Science；Translational Medicine, University of Ottawa , city= Ottawa , state= Ontario , country= Canada
Institute of Research and Technology b<>com（b<>com技术研究院）
Purpose. Ability to locate and track ultrasound images in the 3D operating space is of great benefit for multiple clinical applications. This is often accomplished by tracking the probe using a precise but expensive optical or electromagnetic tracking system. Our goal is to develop a simple and low cost augmented reality echography framework using a standard RGB-D Camera. Methods. A prototype system consisting of an Occipital Structure Core RGB-D camera, a specifically-designed 3D marker, and a fast point cloud registration algorithm FaVoR was developed and evaluated on an Ultrasonix ultrasoun…
Boston University School of Medicine（波士顿大学医学院）；Academic Medical Center（学术医学中心）；Leiden University Medical Center（莱顿大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Arkana laboratories（Arkana实验室）；Amsterdam Cardiovascular Sciences（阿姆斯特丹心血管科学）；Veterans Affairs Boston Healthcare System（退伍军人事务部波士顿医疗系统）；Boston University（波士顿大学）
Development of deep learning systems for biomedical segmentation often requires access to expert-driven, manually annotated datasets. If more than a single expert is involved in the annotation of the same images, then the inter-expert agreement is not necessarily perfect, and no single expert annotation can precisely capture the so-called ground truth of the regions of interest on all images. Also, it is not trivial to generate a reference estimate using annotations from multiple experts. Here we present a deep neural network, defined as U-Net-and-a-half, which can simultaneously learn from an…
Twinkling Artifact is a valuable tool in detecting dense objects such as kidney stones, calculi etc., especially when there is no acoustic shadowing and presence of hyperechogenic tissues obstructs visualization. This phenomenon is not completely understood. Different scientific groups have contradictory findings concerning its properties: some of them observed a decrease in Twinkling intensity at elevated pulse repetition frequencies (PRF), while others found Twinkling to be independent from PRF, etc. In this paper we hypothesize that this kind of contradictions can be partially resolved on a…
Non-invasive detection of microvascular alterations in deep tissues in vivo provides critical information for clinical diagnosis and evaluation of a broad-spectrum of pathologies. Recently, the emergence of super-resolution ultrasound localization microscopy (ULM) offers new possibilities for clinical imaging of microvasculature at capillary level. Currently, the clinical utility of ULM on clinical ultrasound scanners is hindered by the technical limitations, such as long data acquisition time, and compromised tracking performance associated with low imaging frame-rate. Here we present an in-h…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins Medical Institute（约翰斯·霍普金斯医学研究所）
Clear identification of bone structures is crucial for ultrasound-guided lumbar interventions, but it can be challenging due to the complex shapes of the self-shadowing vertebra anatomy and the extensive background speckle noise from the surrounding soft tissue structures. Therefore, we propose to use a patch-like wearable ultrasound solution to capture the reflective bone surfaces from multiple imaging angles and create 3D bone representations for interventional guidance. In this work, we will present our method for estimating the vertebra bone surfaces by using a spatiotemporal U-Net archite…
Deep learning (DL) powered biomedical ultrasound imaging is an emerging research field where researchers adapt the image analysis capabilities of DL algorithms to biomedical ultrasound imaging settings. A major roadblock to wider adoption of DL powered biomedical ultrasound imaging is that acquiring large and diverse datasets is expensive in clinical settings, which is a requirement for successful DL implementation. Hence, there is a constant need for developing data-efficient DL techniques to turn DL powered biomedical ultrasound imaging into reality. In this work, we develop a data-efficient…
Ultrafast 3-D Super Resolution Ultrasound using Row-Column Array specific Coherence-based Beamforming and Rolling Acoustic Sub-aperture Processing: In Vitro, In Vivo and Clinical Study
CycleULM: A unified label-free deep learning framework for ultrasound localisation microscopy
Department of Bioengineering, Faculty of Engineering, Imperial College London（生物工程系，工程学院，帝国理工学院伦敦分校）；Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London（电气与电子工程系，工程学院，帝国理工学院伦敦分校）；Faculty of Electrical and Computer Engineering, Escuela Superior Politécnica del Litoral (ESPOL)（电气与计算机工程学院，ESPOL高级理工大学）
Enhanced Real-Time 6-DOF Extended Reality Catheter Tracking for Evaluating Potential Improvement in Efficiency, Precision, and Depth Perception for Cardiac Interventions
