Applying machine learning technologies, especially deep learning, into medical image segmentation is being widely studied because of its state-of-the-art performance and results. It can be a key step to provide a reliable basis for clinical diagnosis, such as 3D reconstruction of human tissues, image-guided interventions, image analyzing and visualization. In this review article, deep-learning-based methods for ultrasound image segmentation are categorized into six main groups according to their architectures and training at first. Secondly, for each group, several current representative algorithms are selected, introduced, analyzed and summarized in detail. In addition, common evaluation methods for image segmentation and ultrasound image segmentation datasets are summarized. Further, the performance of the current methods and their evaluations are reviewed. In the end, the challenges and potential research directions for medical ultrasound image segmentation are discussed.
The Chinese University of Hong Kong（香港中文大学）；University of Cambridge（剑桥大学）；Inception Institute of Artificial Intelligence（启迪人工智能研究院）；The Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Southern University of Science and Technology（南方科技大学）；Chinese Academy of Sciences（中国科学院）
Automatic breast lesion segmentation in ultrasound helps to diagnose breast cancer, which is one of the dreadful diseases that affect women globally. Segmenting breast regions accurately from ultrasound image is a challenging task due to the inherent speckle artifacts, blurry breast lesion boundaries, and inhomogeneous intensity distributions inside the breast lesion regions. Recently, convolutional neural networks (CNNs) have demonstrated remarkable results in medical image segmentation tasks. However, the convolutional operations in a CNN often focus on local regions, which suffer from limit…
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）
Ultrasound imaging is a standard examination during pregnancy that can be used for measuring specific biometric parameters towards prenatal diagnosis and estimating gestational age. Fetal head circumference (HC) is one of the significant factors to determine the fetus growth and health. In this paper, a multi-task deep convolutional neural network is proposed for automatic segmentation and estimation of HC ellipse by minimizing a compound cost function composed of segmentation dice score and MSE of ellipse parameters. Experimental results on fetus ultrasound dataset in different trimesters of …
Ultrasound (US) is one of the most commonly used imaging modalities in both diagnosis and surgical interventions due to its low-cost, safety, and non-invasive characteristic. US image segmentation is currently a unique challenge because of the presence of speckle noise. As manual segmentation requires considerable efforts and time, the development of automatic segmentation algorithms has attracted researchers attention. Although recent methodologies based on convolutional neural networks have shown promising performances, their success relies on the availability of a large number of training d…
BAAF: A benchmark attention adaptive framework for medical ultrasound image segmentation tasks
College of Biomedical Engineering and Technology, Tianjin Medical University（天津医科大学生物医学工程学院）；College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）；Department of Urology, Fourth Medical Center of PLA General Hospital（中国人民解放军第四医学中心泌尿外科科）；Department of Urology, Civil Aviation General Hospital（民航总医院泌尿外科科）；College of Artificial Intelligence, Nankai University（南开大学人工智能学院）；Department of Ultrasonography, Tianjin Medical University Second Hospital（天津医科大学第二医院超声科）
Shenzhen University（深圳大学）；Nanfang Medical University（南方医科大学）；The Chinese University of Hong Kong（香港中文大学）
Volumetric ultrasound has great potentials in promoting prenatal examinations. Automated solutions are highly desired to efficiently and effectively analyze the massive volumes. Segmentation and landmark localization are two key techniques in making the quantitative evaluation of prenatal ultrasound volumes available in clinic. However, both tasks are non-trivial when considering the poor image quality, boundary ambiguity and anatomical variations in volumetric ultrasound. In this paper, we propose an effective framework for simultaneous segmentation and landmark localization in prenatal ultra…
Fine-tuning a network which has been trained on a large dataset is an alternative to full training in order to overcome the problem of scarce and expensive data in medical applications. While the shallow layers of the network are usually kept unchanged, deeper layers are modified according to the new dataset. This approach may not work for ultrasound images due to their drastically different appearance. In this study, we investigated the effect of fine-tuning different layers of a U-Net which was trained on segmentation of natural images in breast ultrasound image segmentation. Tuning the cont…
Breast tumor segmentation provides accurate tumor boundary, and serves as a key step toward further cancer quantification. Although deep learning-based approaches have been proposed and achieved promising results, existing approaches have difficulty in detecting small breast tumors. The capacity to detecting small tumors is particularly important in finding early stage cancers using computer-aided diagnosis (CAD) systems. In this paper, we propose a novel deep learning architecture called Small Tumor-Aware Network (STAN), to improve the performance of segmenting tumors with different size. The…
University of Florida（佛罗里达大学）；US Anesthesia Partners（美国麻醉伙伴公司）
Automated object recognition in medical images can facilitate medical diagnosis and treatment. In this paper, we automatically segmented supraclavicular nerves in ultrasound images to assist in injecting peripheral nerve blocks. Nerve blocks are generally used for pain treatment after surgery, where ultrasound guidance is used to inject local anesthetics next to target nerves. This treatment blocks the transmission of pain signals to the brain, which can help improve the rate of recovery from surgery and significantly decrease the requirement for postoperative opioids. However, Ultrasound Guid…
Imperial College London（帝国理工学院）；The University of Hong Kong（香港大学）
Transfer learning (TL) for medical image segmentation helps deep learning models achieve more accurate performances when there are scarce medical images. This study focuses on completing segmentation of the ribs from lung ultrasound images and finding the best TL technique with U-Net, a convolutional neural network for precise and fast image segmentation. Two approaches of TL were used, using a pre-trained VGG16 model to build the U-Net (V-Unet) and pre-training U-Net network with grayscale natural salient object dataset (X-Unet). Visual results and dice coefficients (DICE) of the models were …
Carnegie Mellon University（卡内基梅隆大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；Cosmetic Surgery Facility LLC（美容外科机构有限责任公司）
We present W-Net, a novel Convolution Neural Network (CNN) framework that employs raw ultrasound waveforms from each A-scan, typically referred to as ultrasound Radio Frequency (RF) data, in addition to the gray ultrasound image to semantically segment and label tissues. Unlike prior work, we seek to label every pixel in the image, without the use of a background class. To the best of our knowledge, this is also the first deep-learning or CNN approach for segmentation that analyses ultrasound raw RF data along with the gray image. International patent(s) pending [PCT/US20/37519]. We chose subc…
Three-dimensional Ultrasound image segmentation methods are surveyed in this paper. The focus of this report is to investigate applications of these techniques and a review of the original ideas and concepts. Although many two-dimensional image segmentation in the literature have been considered as a three-dimensional approach by mistake but we review them as a three-dimensional technique. We select the studies that have addressed the problem of medical three-dimensional Ultrasound image segmentation utilizing their proposed techniques. The evaluation methods and comparison between them are pr…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Breast tumor segmentation is a critical task in computer-aided diagnosis (CAD) systems for breast cancer detection because accurate tumor size, shape and location are important for further tumor quantification and classification. However, segmenting small tumors in ultrasound images is challenging, due to the speckle noise, varying tumor shapes and sizes among patients, and the existence of tumor-like image regions. Recently, deep learning-based approaches have achieved great success for biomedical image analysis, but current state-of-the-art approaches achieve poor performance for segmenting …
Robot-guided catheter insertion has the potential to deliver urgent medical care in situations where medical personnel are unavailable. However, this technique requires accurate and reliable segmentation of anatomical landmarks in the body. For the ultrasound imaging modality, obtaining large amounts of training data for a segmentation model is time-consuming and expensive. This paper introduces RESUS (RESlicing of UltraSound Images), a weak supervision data augmentation technique for ultrasound images based on slicing reconstructed 3D volumes from tracked 2D images. This technique allows us t…
University of Idaho（爱达荷大学）；Utah State University（犹他州立大学）；Harbin Institute of Technology（哈尔滨工业大学）
Breast cancer is one of the leading causes of cancer death among women worldwide. In clinical routine, automatic breast ultrasound (BUS) image segmentation is very challenging and essential for cancer diagnosis and treatment planning. Many BUS segmentation approaches have been studied in the last two decades, and have been proved to be effective on private datasets. Currently, the advancement of BUS image segmentation seems to meet its bottleneck. The improvement of the performance is increasingly challenging, and only few new approaches were published in the last several years. It is the time…
Gaze-Assisted Human-Centric Domain Adaptation for Cardiac Ultrasound Image Segmentation
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
Harbin Institute of Technology（哈尔滨工业大学）；University of Idaho（爱达荷大学）；Utah State University（犹他州立大学）；Affiliated Hospital of Medical College Qingdao University（青岛大学医学院附属医院）；Second Hospital of Hebei Medical University（河北医科大学第二医院）
Breast ultrasound (BUS) image segmentation is challenging and critical for BUS Comput-er-Aided Diagnosis (CAD) systems. Many BUS segmentation approaches have been studied in the last two decades, but the performances of most approaches have been assessed using relatively small private datasets with different quantitative metrics, which results in a discrepancy in performance comparison. Therefore, there is a pressing need for building a benchmark to compare existing methods using a public dataset objectively, to determine the performance of the best breast tumor segmentation algorithm availabl…
University of Electronic Science and Technology of China（电子科技大学）；Institute of Electronic and Information Engineering of UESTC in Guangdong（电子科技大学广东电子信息工程研究院）；Sichuan Provincial Hospital for Women and Children（四川省妇幼保健院）
The identification of nerve is difficult as structures of nerves are challenging to image and to detect in ultrasound images. Nevertheless, the nerve identification in ultrasound images is a crucial step to improve performance of regional anesthesia. In this paper, a network called Brachial Plexus Multi-instance Segmentation Network (BPMSegNet) is proposed to identify different tissues (nerves, arteries, veins, muscles) in ultrasound images. The BPMSegNet has three novel modules. The first is the spatial local contrast feature, which computes contrast features at different scales. The second o…
Shenzhen University（深圳大学）；Luohu People’s Hospital（罗湖人民医院）；Southeast University（东南大学）；University of Leeds（利兹大学）；KU Leuven（鲁汶大学）
Deep segmentation models that generalize to images with unknown appearance are important for real-world medical image analysis. Retraining models leads to high latency and complex pipelines, which are impractical in clinical settings. The situation becomes more severe for ultrasound image analysis because of their large appearance shifts. In this paper, we propose a novel method for robust segmentation under unknown appearance shifts. Our contribution is three-fold. First, we advance a one-stage plug-and-play solution by embedding hierarchical style transfer units into a segmentation architect…
University of Delaware（特拉华大学）；University of Guilan（吉兰大学）；Eindhoven University of Technology（埃因霍温理工大学）；Iran University of Science and Technology（伊朗科学技术大学）
Automated 3-D breast ultrasound (ABUS) is a newfound system for breast screening that has been proposed as a supplementary modality to mammography for breast cancer detection. While ABUS has better performance in dense breasts, reading ABUS images is exhausting and time-consuming. So, a computer-aided detection system is necessary for interpretation of these images. Mass segmentation plays a vital role in the computer-aided detection systems and it affects the overall performance. Mass segmentation is a challenging task because of the large variety in size, shape, and texture of masses. Moreov…
UC San Diego（加州大学圣地亚哥分校）；General Electric Global Research（通用电气全球研究中心）
Ultrasound scanning is essential in several medical diagnostic and therapeutic applications. It is used to visualize and analyze anatomical features and structures that influence treatment plans. However, it is both labor intensive, and its effectiveness is operator dependent. Real-time accurate and robust automatic detection and tracking of anatomical structures while scanning would significantly impact diagnostic and therapeutic procedures to be consistent and efficient. In this paper, we propose a deep learning framework to automatically detect and track a specific anatomical target structu…
A region and category confidence-based multi-task network for carotid ultrasound image segmentation and classification
Hubei University of Technology（湖北工业大学）；Liyuan Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属梨园医院）；Zhongnan Hospital, Wuhan University（武汉大学中南医院）；Robarts Research Institute, Western University（西安大略大学罗伯茨研究所）
CMU-Net: A Strong ConvMixer-based Medical Ultrasound Image Segmentation Network
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；The Affiliated Hospital of Qingdao University（青岛大学附属医院）；University of Idaho（爱达荷大学）
Deep Neural Networks (DNNs) suffer from the performance degradation when image appearance shift occurs, especially in ultrasound (US) image segmentation. In this paper, we propose a novel and intuitive framework to remove the appearance shift, and hence improve the generalization ability of DNNs. Our work has three highlights. First, we follow the spirit of universal style transfer to remove appearance shifts, which was not explored before for US images. Without sacrificing image structure details, it enables the arbitrary style-content transfer. Second, accelerated with Adaptive Instance Norm…
In action understanding in indoor, we have to recognize human pose and action considering privacy. Although camera images can be used for highly accurate human action recognition, camera images do not preserve privacy. Therefore, we propose a new task for human instance segmentation from invisible information, especially airborne ultrasound, for action recognition. To perform instance segmentation from invisible information, we first convert sound waves to reflected sound directional images (sound images). Although the sound images can roughly identify the location of a person, the detailed sh…
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
Ultrasound imaging is generally employed for real-time investigation of internal anatomy of the human body for disease identification. Delineation of the anatomical boundary of organs and pathological lesions is quite challenging due to the stochastic nature of speckle intensity in the images, which also introduces visual fatigue for the observer. This paper introduces a fully convolutional neural network based method to segment organ and pathologies in ultrasound volume by learning the spatial-relationship between closely related classes in the presence of stochastically varying speckle inten…
College of Electronic and Information Engineering of Hebei University（河北大学电子信息工程学院）
The incidence of thyroid nodule is very high and generally increases with the age. Thyroid nodule may presage the emergence of thyroid cancer. The thyroid nodule can be completely cured if detected early. Fine needle aspiration cytology is a recognized early diagnosis method of thyroid nodule. There are still some limitations in the fine needle aspiration cytology, and the ultrasound diagnosis of thyroid nodule has become the first choice for auxiliary examination of thyroid nodular disease. If we could combine medical imaging technology and fine needle aspiration cytology, the diagnostic rate…
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation
Wuhan University of Technology（武汉理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；MedAI Technology (Wuxi) Co. Ltd.（MedAI科技（无锡）有限公司）；Technical University of Munich（慕尼黑工业大学）
Eindhoven University of Technology（埃因霍温理工大学）；Shandong University of Science and Technology（山东科技大学）；Philips Research（飞利浦研究院）
Accurate and efficient catheter segmentation in 3D ultrasound (US) is essential for cardiac intervention. Currently, the state-of-the-art segmentation algorithms are based on convolutional neural networks (CNNs), which achieved remarkable performances in a standard Cartesian volumetric data. Nevertheless, these approaches suffer the challenges of low efficiency and GPU unfriendly image size. Therefore, such difficulties and expensive hardware requirements become a bottleneck to build accurate and efficient segmentation models for real clinical application. In this paper, we propose a novel Fru…
University of Toulouse（图卢兹大学）；IRIT/INP-ENSEEIHT（图卢兹国立综合理工学院国立高等电子技术、电子学、计算机、水力学院/图卢兹信息学研究所）；IRIT, CNRS UMR 5505（法国国家科学研究中心联合研究单位5505 图卢兹信息学研究所）
This paper proposes a joint segmentation and deconvolution Bayesian method for medical ultrasound (US) images. Contrary to piecewise homogeneous images, US images exhibit heavy characteristic speckle patterns correlated with the tissue structures. The generalized Gaussian distribution (GGD) has been shown to be one of the most relevant distributions for characterizing the speckle in US images. Thus, we propose a GGD-Potts model defined by a label map coupling US image segmentation and deconvolution. The Bayesian estimators of the unknown model parameters, including the US image, the label map …
Uncertainty-Aware Extreme Point Tracing for Weakly Supervised Ultrasound Image Segmentation
College of Computer Science, Inner Mongolia University（内蒙古大学计算机学院）；Baotou Medical College（包头医学院）；Department of Ultrasound Medicine, The Second Affiliated Hospital of Baotou Medical College（包头医学院第二附属医院超声医学科）
FAMSeg: Fetal Femur and Cranial Ultrasound Segmentation Using Feature-Aware Attention and Mamba Enhancement
Guangxi Key Laboratory of Machine Vision and Intelligent Control（广西机器视觉与智能控制重点实验室）；Wuzhou University（梧州大学）；Faculty of Innovation Engineering（创新工程学院）；Macau University of Science and Technology（澳门科技大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Shenzhen Maternity and Child Healthcare Hospital（深圳妇幼保健医院）；Southern Medical University（南方医科大学）；College of Big Data and Software Engineering（大数据与软件工程学院）；College of Electronical and Information Engineering（电子与信息工程学院）
University of Oslo（奥斯陆大学）；GE Vingmed Ultrasound（GE维ingmed超声公司）；GE Research（通用电气研究中心）
Accurate and consistent predictions of echocardiography parameters are important for cardiovascular diagnosis and treatment. In particular, segmentations of the left ventricle can be used to derive ventricular volume, ejection fraction (EF) and other relevant measurements. In this paper we propose a new automated method called EchoGraphs for predicting ejection fraction and segmenting the left ventricle by detecting anatomical keypoints. Models for direct coordinate regression based on Graph Convolutional Networks (GCNs) are used to detect the keypoints. GCNs can learn to represent the cardiac…
An Edge-aware Prompt-enhanced SAM for Ultrasound Image Segmentation
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University（航空信息安全与可信计算重点实验室，教育部，武汉大学计算机科学与工程学院）；National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University（多媒體軟體國家工程研究中心，人工智能研究院，武漢大學計算機科學學院，湖北省多媒體與網絡通信工程重點實驗室）
针对超声图像分割中 SAM 边界描绘不佳问题，提出 EP-SAM，通过图像编码器多块特征提取和边缘感知监督，生成高质量提示引导模型，在多个基准测试中表现优于现有基于 SAM 的方法。
Department of Radiology and Diagnostic Imaging, University of Alberta（放射科与诊断影像学系，阿尔伯塔大学）
Elbow and wrist fractures are the most common fractures in pediatric populations. Automatic segmentation of musculoskeletal structures in ultrasound (US) can improve diagnostic accuracy and treatment planning. Fractures appear as cortical defects but require expert interpretation. Deep learning (DL) can provide real-time feedback and highlight key structures, helping lightly trained users perform exams more confidently. However, pixel-wise expert annotations for training remain time-consuming and costly. To address this challenge, we propose FlexICL, a novel and flexible in-context learning (I…
The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；The Hong Kong Polytechnic University（香港理工大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Boundary incompleteness raises great challenges to automatic prostate segmentation in ultrasound images. Shape prior can provide strong guidance in estimating the missing boundary, but traditional shape models often suffer from hand-crafted descriptors and local information loss in the fitting procedure. In this paper, we attempt to address those issues with a novel framework. The proposed framework can seamlessly integrate feature extraction and shape prior exploring, and estimate the complete boundary with a sequential manner. Our framework is composed of three key modules. Firstly, we seria…
