Ultrasound (US) is one of the most commonly used imaging modalities in both diagnosis and surgical interventions due to its low-cost, safety, and non-invasive characteristic. US image segmentation is currently a unique challenge because of the presence of speckle noise. As manual segmentation requires considerable efforts and time, the development of automatic segmentation algorithms has attracted researchers attention. Although recent methodologies based on convolutional neural networks have shown promising performances, their success relies on the availability of a large number of training data, which is prohibitively difficult for many applications. Therefore, in this study we propose the use of simulated US images and natural images as auxiliary datasets in order to pre-train our segmentation network, and then to fine-tune with limited in vivo data. We show that with as little as 19 in vivo images, fine-tuning the pre-trained network improves the dice score by 21% compared to training from scratch. We also demonstrate that if the same number of natural and simulation US images is available, pre-training on simulation data is preferable.
The Chinese University of Hong Kong（香港中文大学）；University of Cambridge（剑桥大学）；Inception Institute of Artificial Intelligence（启迪人工智能研究院）；The Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Southern University of Science and Technology（南方科技大学）；Chinese Academy of Sciences（中国科学院）
Automatic breast lesion segmentation in ultrasound helps to diagnose breast cancer, which is one of the dreadful diseases that affect women globally. Segmenting breast regions accurately from ultrasound image is a challenging task due to the inherent speckle artifacts, blurry breast lesion boundaries, and inhomogeneous intensity distributions inside the breast lesion regions. Recently, convolutional neural networks (CNNs) have demonstrated remarkable results in medical image segmentation tasks. However, the convolutional operations in a CNN often focus on local regions, which suffer from limit…
Nankai University（南开大学）；Civil Aviation General Hospital（民航总医院）；Chinese PLA General Hospital（中国人民解放军总医院）
Breast lesions segmentation is an important step of computer-aided diagnosis system, and it has attracted much attention. However, accurate segmentation of malignant breast lesions is a challenging task due to the effects of heterogeneous structure and similar intensity distributions. In this paper, a novel bidirectional aware guidance network (BAGNet) is proposed to segment the malignant lesion from breast ultrasound images. Specifically, the bidirectional aware guidance network is used to capture the context between global (low-level) and local (high-level) features from the input coarse sal…
Breast tumor segmentation provides accurate tumor boundary, and serves as a key step toward further cancer quantification. Although deep learning-based approaches have been proposed and achieved promising results, existing approaches have difficulty in detecting small breast tumors. The capacity to detecting small tumors is particularly important in finding early stage cancers using computer-aided diagnosis (CAD) systems. In this paper, we propose a novel deep learning architecture called Small Tumor-Aware Network (STAN), to improve the performance of segmenting tumors with different size. The…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Breast tumor segmentation is a critical task in computer-aided diagnosis (CAD) systems for breast cancer detection because accurate tumor size, shape and location are important for further tumor quantification and classification. However, segmenting small tumors in ultrasound images is challenging, due to the speckle noise, varying tumor shapes and sizes among patients, and the existence of tumor-like image regions. Recently, deep learning-based approaches have achieved great success for biomedical image analysis, but current state-of-the-art approaches achieve poor performance for segmenting …
Classification of malignancy for breast cancer and other cancer types is usually tackled as an object detection problem: Individual lesions are first localized and then classified with respect to malignancy. However, the drawback of this approach is that abstract features incorporating several lesions and areas that are not labelled as a lesion but contain global medically relevant information are thus disregarded: especially for dynamic contrast-enhanced breast MRI, criteria such as background parenchymal enhancement and location within the breast are important for diagnosis and cannot be cap…
We present Spectral GUI, a multiplatform breast MR image analysis tool designed to facilitate the segmentation of fibro glandular tissues and lesions in T1 weighted breast MR images via a graphical user interface (GUI). Spectral GUIR uses spectrum loft method [1] for breast MR image segmentation. Not only is it interactive, but robust and expeditious at the same time. Being devoid of any machine learning algorithm, it shows exceptionally high execution speed with minimal overheads. The accuracy of the results has been simultaneously measured using performance metrics and expert entailment. The…
Korea University（高丽大学）；Hankik University of Foreign Studies（韩国外国语大学）
We propose a method for effectively utilizing weakly annotated image data in an object detection tasks of breast ultrasound images. Given the problem setting where a small, strongly annotated dataset and a large, weakly annotated dataset with no bounding box information are available, training an object detection model becomes a non-trivial problem. We suggest a controlled weight for handling the effect of weakly annotated images in a two stage object detection model. We~also present a subsequent active learning scheme for safely assigning weakly annotated images a strong annotation using the …
University of Idaho（爱达荷大学）；Utah State University（犹他州立大学）；Harbin Institute of Technology（哈尔滨工业大学）
Breast cancer is one of the leading causes of cancer death among women worldwide. In clinical routine, automatic breast ultrasound (BUS) image segmentation is very challenging and essential for cancer diagnosis and treatment planning. Many BUS segmentation approaches have been studied in the last two decades, and have been proved to be effective on private datasets. Currently, the advancement of BUS image segmentation seems to meet its bottleneck. The improvement of the performance is increasingly challenging, and only few new approaches were published in the last several years. It is the time…
East Delta University（东三角洲大学）；University of Chittagong（吉大港大学）
Breast cancer has become a symbol of tremendous concern in the modern world, as it is one of the major causes of cancer mortality worldwide. In this regard, breast ultrasonography images are frequently utilized by doctors to diagnose breast cancer at an early stage. However, the complex artifacts and heavily noised breast ultrasonography images make diagnosis a great challenge. Furthermore, the ever-increasing number of patients being screened for breast cancer necessitates the use of automated end-to-end technology for highly accurate diagnosis at a low cost and in a short time. In this conce…
Utah State University（犹他州立大学）；Harbin Institute of Technology（哈尔滨工业大学）；the First Affiliated Hospital of Harbin Medical University（哈尔滨医科大学附属第一医院）
Breast cancer is one of the most serious disease affecting women's health. Due to low cost, portable, no radiation, and high efficiency, breast ultrasound (BUS) imaging is the most popular approach for diagnosing early breast cancer. However, ultrasound images are low resolution and poor quality. Thus, developing accurate detection system is a challenging task. In this paper, we propose a fully automatic segmentation algorithm consisting of two parts: fuzzy fully convolutional network and accurately fine-tuning post-processing based on breast anatomy constraints. In the first part, the image i…
Harbin Institute of Technology（哈尔滨工业大学）；University of Idaho（爱达荷大学）；Utah State University（犹他州立大学）；Affiliated Hospital of Medical College Qingdao University（青岛大学医学院附属医院）；Second Hospital of Hebei Medical University（河北医科大学第二医院）
Breast ultrasound (BUS) image segmentation is challenging and critical for BUS Comput-er-Aided Diagnosis (CAD) systems. Many BUS segmentation approaches have been studied in the last two decades, but the performances of most approaches have been assessed using relatively small private datasets with different quantitative metrics, which results in a discrepancy in performance comparison. Therefore, there is a pressing need for building a benchmark to compare existing methods using a public dataset objectively, to determine the performance of the best breast tumor segmentation algorithm availabl…
University of Delaware（特拉华大学）；University of Guilan（吉兰大学）；Eindhoven University of Technology（埃因霍温理工大学）；Iran University of Science and Technology（伊朗科学技术大学）
Automated 3-D breast ultrasound (ABUS) is a newfound system for breast screening that has been proposed as a supplementary modality to mammography for breast cancer detection. While ABUS has better performance in dense breasts, reading ABUS images is exhausting and time-consuming. So, a computer-aided detection system is necessary for interpretation of these images. Mass segmentation plays a vital role in the computer-aided detection systems and it affects the overall performance. Mass segmentation is a challenging task because of the large variety in size, shape, and texture of masses. Moreov…
School of Biomedical Engineering, Southern Medical University（南方医科大学生物医学工程学院）
Breast ultrasound (BUS) image segmentation plays a crucial role in a computer-aided diagnosis system, which is regarded as a useful tool to help increase the accuracy of breast cancer diagnosis. Recently, many deep learning methods have been developed for segmentation of BUS image and show some advantages compared with conventional region-, model-, and traditional learning-based methods. However, previous deep learning methods typically use skip-connection to concatenate the encoder and decoder, which might not make full fusion of coarse-to-fine features from encoder and decoder. Since the str…
Georgia Institute of Technology（佐治亚理工学院）；Telecom Paris（巴黎电信学院）；Medicine School of Tunis（突尼斯医学院）
Breast cancer is one of the factors that cause the increase of mortality of women. The most widely used method for diagnosing this geological disease i.e. breast cancer is the ultrasound scan. Several key features such as the smoothness and the texture of the tumor captured through ultrasound scans encode the abnormality of the breast tumors (malignant from benign). However, ultrasound scans are often noisy and include irrelevant parts of the breast that may bias the segmentation of eventual tumors. In this paper, we are going to extract the region of interest ( i.e, bounding boxes of the tumo…
In this work we study the impact of noise on the training of object detection networks for the medical domain, and how it can be mitigated by improving the training procedure. Annotating large medical datasets for training data-hungry deep learning models is expensive and time consuming. Leveraging information that is already collected in clinical practice, in the form of text reports, bookmarks or lesion measurements would substantially reduce this cost. Obtaining precise lesion bounding boxes through automatic mining procedures, however, is difficult. We provide here a quantitative evaluatio…
Iran University of Science and Technology（伊朗科技大学）；Radboud University Medical Center（拉德堡德大学医学中心）；Netherlands Cancer Institute（荷兰癌症研究所）
Purpose: We propose a deep learning-based computer-aided detection (CADe) method to detect breast lesions in ultrafast DCE-MRI sequences. This method uses both the three-dimensional spatial information and temporal information obtained from the early-phase of the dynamic acquisition. Methods: The proposed CADe method, based on a modified 3D RetinaNet model, operates on ultrafast T1 weighted sequences, which are preprocessed for motion compensation, temporal normalization, and are cropped before passing into the model. The model is optimized to enable the detection of relatively small breast le…
School of Information Technology and Electrical Engineering, University of Queensland（昆士兰大学信息技术与电气工程学院）；Science and Engineering Faculty, Queensland University of Technology（昆士兰科技大学科学与工程学院）
Previous studies on computer aided detection/diagnosis (CAD) in 4D breast magnetic resonance imaging (MRI) regard lesion detection, segmentation and characterization as separate tasks, and typically require users to manually select 2D MRI slices or regions of interest as the input. In this work, we present a breast MRI CAD system that can handle 4D multimodal breast MRI data, and integrate lesion detection, segmentation and characterization with no user intervention. The proposed CAD system consists of three major stages: region candidate generation, feature extraction and region candidate cla…
School of Computing, University of Buckingham（白金汉大学计算学院）
Deep Convolutional Neural Networks (CNN) provides an "end-to-end" solution for image pattern recognition with impressive performance in many areas of application including medical imaging. Most CNN models of high performance use hand-crafted network architectures that require expertise in CNNs to utilise their potentials. In this paper, we applied the Efficient Neural Architecture Search (ENAS) method to find optimal CNN architectures for classifying breast lesions from ultrasound (US) images. Our empirical study with a dataset of 524 US images shows that the optimal models generated by using …
University of Minnesota（明尼苏达大学）；University of Cincinnati（辛辛那提大学）
Image classification is widely used to build predictive models for breast cancer diagnosis. Most existing approaches overwhelmingly rely on deep convolutional networks to build such diagnosis pipelines. These model architectures, although remarkable in performance, are black-box systems that provide minimal insight into the inner logic behind their predictions. This is a major drawback as the explainability of prediction is vital for applications such as cancer diagnosis. In this paper, we address this issue by proposing an explainable machine learning pipeline for breast cancer diagnosis base…
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究院）；The Hong Kong Polytechnic University（香港理工大学）
Breast lesion detection in ultrasound is critical for breast cancer diagnosis. Existing methods mainly rely on individual 2D ultrasound images or combine unlabeled video and labeled 2D images to train models for breast lesion detection. In this paper, we first collect and annotate an ultrasound video dataset (188 videos) for breast lesion detection. Moreover, we propose a clip-level and video-level feature aggregated network (CVA-Net) for addressing breast lesion detection in ultrasound videos by aggregating video-level lesion classification features and clip-level temporal features. The clip-…
University of Houston（休斯顿大学）；Houston Methodist（休斯顿卫理公会医院）
Accurate breast lesion risk estimation can significantly reduce unnecessary biopsies and help doctors decide optimal treatment plans. Most existing computer-aided systems rely solely on mammogram features to classify breast lesions. While this approach is convenient, it does not fully exploit useful information in clinical reports to achieve the optimal performance. Would clinical features significantly improve breast lesion classification compared to using mammograms alone? How to handle missing clinical information caused by variation in medical practice? What is the best way to combine mamm…
Breast Ultrasound Report Generation using LangChain
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Chung-Ang University Hospital（中央大学医院）；Chung-Ang University College of Medicine（中央大学医学院）
University of Malaya（马来亚大学）；University of Tehran（德黑兰大学）
Reliable classification of benign and malignant lesions in breast ultrasound images can provide an effective and relatively low cost method for early diagnosis of breast cancer. The accuracy of the diagnosis is however highly dependent on the quality of the ultrasound systems and the experience of the users (radiologists). The leverage in deep convolutional neural network approaches provided solutions in efficient analysis of breast ultrasound images. In this study, we proposed a new framework for classification of breast cancer lesions by use of an attention module in modified VGG16 architect…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Incorporating human domain knowledge for breast tumor diagnosis is challenging, since shape, boundary, curvature, intensity, or other common medical priors vary significantly across patients and cannot be employed. This work proposes a new approach for integrating visual saliency into a deep learning model for breast tumor segmentation in ultrasound images. Visual saliency refers to image maps containing regions that are more likely to attract radiologists visual attention. The proposed approach introduces attention blocks into a U-Net architecture, and learns feature representations that prio…
Shanghai Jiao Tong University（上海交通大学）；Shanghai General Hospital（上海市第一人民医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）
Previous deep learning based Computer Aided Diagnosis (CAD) system treats multiple views of the same lesion as independent images. Since an ultrasound image only describes a partial 2D projection of a 3D lesion, such paradigm ignores the semantic relationship between different views of a lesion, which is inconsistent with the traditional diagnosis where sonographers analyze a lesion from at least two views. In this paper, we propose a multi-task framework that complements Benign/Malignant classification task with lesion recognition (LR) which helps leveraging relationship among multiple views …
Graduate School at Shenzhen, Tsinghua University（清华大学深圳研究生院）；Shenzhen City Baoan District Women’s and Children’s Hospital（深圳市宝安区妇幼保健院）；Shenzhen Hospital of Peking University（北京大学深圳医院）；Beijing Tsinghua Changgung Hospital, Tsinghua University（清华大学北京清华长庚医院）；School of Biomedical Informatics, University of Texas Health Sciences Center at Houston（德克萨斯大学休斯顿健康科学中心生物医学信息学院）
Background: Ultrasound is one of the preferred choices for early screening of dense breast cancer. Clinically, doctors have to manually write the screening report which is time-consuming and laborious, and it is easy to miss and miswrite. Aim: We proposed a new pipeline to automatically generate AI breast ultrasound screening reports based on ultrasound images, aiming to assist doctors in improving the efficiency of clinical screening and reducing repetitive report writing. Methods: AI was used to efficiently generate personalized breast ultrasound screening preliminary reports, especially for…
Institute of Fundamental Technological Research, Polish Academy of Sciences（波兰科学院基础技术研究所）；University of California, San Diego（加州大学圣迭戈分校）
Morphological features play an important role in breast mass classification in sonography. While benign breast masses tend to have a well-defined ellipsoidal contour, shape of malignant breast masses is commonly ill-defined and highly variable. Various handcrafted morphological features have been developed over the years to assess this phenomenon and help the radiologists differentiate benign and malignant masses. In this paper we propose an automatic approach to morphology analysis, we express shapes of breast masses as points on the Kendall's shape manifold. Next, we use the full Procrustes …
DSU-Net: An Attention-Enhanced Dense Skip U-Net for Breast Lesion Segmentation in Mammographic Images
Department of Biomedical Engineering, University of Wisconsin-Milwaukee（威斯康星大学密尔沃基分校生物医学工程系）；Department of Mechanical Engineering, Northern Arizona University（北亚利桑那大学机械工程系）
Northeastern University（东北大学）；Sino-Dutch Biomedical and Information Engineering School（中荷生物医学与信息工程学院）
Ultrasound image diagnosis of breast tumors has been widely used in recent years. However, there are some problems of it, for instance, poor quality, intense noise and uneven echo distribution, which has created a huge obstacle to diagnosis. To overcome these problems, we propose a novel method, a breast cancer classification with ultrasound images based on SLIC (BCCUI). We first utilize the Region of Interest (ROI) extraction based on Simple Linear Iterative Clustering (SLIC) algorithm and region growing algorithm to extract the ROI at the super-pixel level. Next, the features of ROI are extr…
UGGNet: Bridging U-Net and VGG for Advanced Breast Cancer Diagnosis
School of Engineering, Tokyo Institute of Technology（东京技术大学工学院）；Faculty of Information Technology Nguyen Tat Thanh University（Nguyen Tat Thanh大学信息科技学院）
Shifting More Attention to Breast Lesion Segmentation in Ultrasound Videos
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Shenzhen University（深圳大学）；Guangdong Province Traditional Chinese Medical Hospital（广东省中医院）
Automated breast ultrasound (ABUS) is a new and promising imaging modality for breast cancer detection and diagnosis, which could provide intuitive 3D information and coronal plane information with great diagnostic value. However, manually screening and diagnosing tumors from ABUS images is very time-consuming and overlooks of abnormalities may happen. In this study, we propose a novel two-stage 3D detection network for locating suspected lesion areas and further classifying lesions as benign or malignant tumors. Specifically, we propose a 3D detection network rather than frequently-used segme…
Breast cancer is the most common invasive cancer in women. Besides the primary B-mode ultrasound screening, sonographers have explored the inclusion of Doppler, strain and shear-wave elasticity imaging to advance the diagnosis. However, recognizing useful patterns in all types of images and weighing up the significance of each modality can elude less-experienced clinicians. In this paper, we explore, for the first time, an automatic way to combine the four types of ultrasonography to discriminate between benign and malignant breast nodules. A novel multimodal network is proposed, along with pr…
Illinois Institute of Technology（伊利诺伊理工学院）；Rutgers University（罗格斯大学）
Automatically segmenting lesion area in breast ultrasound (BUS) images is a challenging one due to its noise, speckle and artifacts. Edge-map of BUS images also does not help because in most cases the edge-map gives no information whatsoever. Almost all segmentation technique takes the edge-map of the image as its first step, though there are a few algorithms that try to avoid edge-maps as well. Improving the edge-map of breast ultrasound images theoretically improves the chances of automatic segmentation to be more precise. In this paper, we propose a semi-automatic technique of histogram spl…
South China University of Technology（华南理工大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）
Ultrasonography is an important routine examination for breast cancer diagnosis, due to its non-invasive, radiation-free and low-cost properties. However, the diagnostic accuracy of breast cancer is still limited due to its inherent limitations. It would be a tremendous success if we can precisely diagnose breast cancer by breast ultrasound images (BUS). Many learning-based computer-aided diagnostic methods have been proposed to achieve breast cancer diagnosis/lesion classification. However, most of them require a pre-define ROI and then classify the lesion inside the ROI. Conventional classif…
The advancements in deep learning technologies have produced immense contributions to biomedical image analysis applications. With breast cancer being the common deadliest disease among women, early detection is the key means to improve survivability. Medical imaging like ultrasound presents an excellent visual representation of the functioning of the organs; however, for any radiologist analysing such scans is challenging and time consuming which delays the diagnosis process. Although various deep learning based approaches are proposed that achieved promising results, the present article intr…
University of Delaware（特拉华大学）；Amazon Web Services（亚马逊云科技）；University of Rochester（罗切斯特大学）
Objective and Impact Statement. We adopt a deep learning model for bone osteolysis prediction on computed tomography (CT) images of murine breast cancer bone metastases. Given the bone CT scans at previous time steps, the model incorporates the bone-cancer interactions learned from the sequential images and generates future CT images. Its ability of predicting the development of bone lesions in cancer-invading bones can assist in assessing the risk of impending fractures and choosing proper treatments in breast cancer bone metastasis. Introduction. Breast cancer often metastasizes to bone, cau…
Is Two-shot All You Need? A Label-efficient Approach for Video Segmentation in Breast Ultrasound
Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging（广东省生物医学测量与超声成像重点实验室）；School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University（深圳大学医学部生物医学工程学院）；Shenzhen University（深圳大学）
College of Information Science and Technology, Northwest University（西北大学信息科学与技术学院）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Breast ultrasound (US) is an effective imaging modality for breast cancer detec-tion and diagnosis. The structural characteristics of breast lesion play an im-portant role in Computer-Aided Diagnosis (CAD). In this paper, a novel struc-ture-aware triplet path networks (SATPN) was designed to integrate classifica-tion and two image reconstruction tasks to achieve accurate diagnosis on US im-ages with small training dataset. Specifically, we enhance clinically-approved breast lesion structure characteristics though converting original breast US imag-es to BIRADS-oriented feature maps (BFMs) with…
