Image classification is widely used to build predictive models for breast cancer diagnosis. Most existing approaches overwhelmingly rely on deep convolutional networks to build such diagnosis pipelines. These model architectures, although remarkable in performance, are black-box systems that provide minimal insight into the inner logic behind their predictions. This is a major drawback as the explainability of prediction is vital for applications such as cancer diagnosis. In this paper, we address this issue by proposing an explainable machine learning pipeline for breast cancer diagnosis based on ultrasound images. We extract first- and second-order texture features of the ultrasound images and use them to build a probabilistic ensemble of decision tree classifiers. Each decision tree learns to classify the input ultrasound image by learning a set of robust decision thresholds for texture features of the image. The decision path of the model predictions can then be interpreted by decomposing the learned decision trees. Our results show that our proposed framework achieves high predictive performance while being explainable.
The Chinese University of Hong Kong（香港中文大学）；University of Cambridge（剑桥大学）；Inception Institute of Artificial Intelligence（启迪人工智能研究院）；The Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Southern University of Science and Technology（南方科技大学）；Chinese Academy of Sciences（中国科学院）
Automatic breast lesion segmentation in ultrasound helps to diagnose breast cancer, which is one of the dreadful diseases that affect women globally. Segmenting breast regions accurately from ultrasound image is a challenging task due to the inherent speckle artifacts, blurry breast lesion boundaries, and inhomogeneous intensity distributions inside the breast lesion regions. Recently, convolutional neural networks (CNNs) have demonstrated remarkable results in medical image segmentation tasks. However, the convolutional operations in a CNN often focus on local regions, which suffer from limit…
Breast tumor segmentation provides accurate tumor boundary, and serves as a key step toward further cancer quantification. Although deep learning-based approaches have been proposed and achieved promising results, existing approaches have difficulty in detecting small breast tumors. The capacity to detecting small tumors is particularly important in finding early stage cancers using computer-aided diagnosis (CAD) systems. In this paper, we propose a novel deep learning architecture called Small Tumor-Aware Network (STAN), to improve the performance of segmenting tumors with different size. The…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Breast tumor segmentation is a critical task in computer-aided diagnosis (CAD) systems for breast cancer detection because accurate tumor size, shape and location are important for further tumor quantification and classification. However, segmenting small tumors in ultrasound images is challenging, due to the speckle noise, varying tumor shapes and sizes among patients, and the existence of tumor-like image regions. Recently, deep learning-based approaches have achieved great success for biomedical image analysis, but current state-of-the-art approaches achieve poor performance for segmenting …
School of Computing, University of Buckingham（白金汉大学计算学院）
Deep Convolutional Neural Networks (CNN) provides an "end-to-end" solution for image pattern recognition with impressive performance in many areas of application including medical imaging. Most CNN models of high performance use hand-crafted network architectures that require expertise in CNNs to utilise their potentials. In this paper, we applied the Efficient Neural Architecture Search (ENAS) method to find optimal CNN architectures for classifying breast lesions from ultrasound (US) images. Our empirical study with a dataset of 524 US images shows that the optimal models generated by using …
Breast Ultrasound Report Generation using LangChain
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Chung-Ang University Hospital（中央大学医院）；Chung-Ang University College of Medicine（中央大学医学院）
University of Malaya（马来亚大学）；University of Tehran（德黑兰大学）
Reliable classification of benign and malignant lesions in breast ultrasound images can provide an effective and relatively low cost method for early diagnosis of breast cancer. The accuracy of the diagnosis is however highly dependent on the quality of the ultrasound systems and the experience of the users (radiologists). The leverage in deep convolutional neural network approaches provided solutions in efficient analysis of breast ultrasound images. In this study, we proposed a new framework for classification of breast cancer lesions by use of an attention module in modified VGG16 architect…
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究院）；The Hong Kong Polytechnic University（香港理工大学）
Breast lesion detection in ultrasound is critical for breast cancer diagnosis. Existing methods mainly rely on individual 2D ultrasound images or combine unlabeled video and labeled 2D images to train models for breast lesion detection. In this paper, we first collect and annotate an ultrasound video dataset (188 videos) for breast lesion detection. Moreover, we propose a clip-level and video-level feature aggregated network (CVA-Net) for addressing breast lesion detection in ultrasound videos by aggregating video-level lesion classification features and clip-level temporal features. The clip-…
Graduate School at Shenzhen, Tsinghua University（清华大学深圳研究生院）；Shenzhen City Baoan District Women’s and Children’s Hospital（深圳市宝安区妇幼保健院）；Shenzhen Hospital of Peking University（北京大学深圳医院）；Beijing Tsinghua Changgung Hospital, Tsinghua University（清华大学北京清华长庚医院）；School of Biomedical Informatics, University of Texas Health Sciences Center at Houston（德克萨斯大学休斯顿健康科学中心生物医学信息学院）
Background: Ultrasound is one of the preferred choices for early screening of dense breast cancer. Clinically, doctors have to manually write the screening report which is time-consuming and laborious, and it is easy to miss and miswrite. Aim: We proposed a new pipeline to automatically generate AI breast ultrasound screening reports based on ultrasound images, aiming to assist doctors in improving the efficiency of clinical screening and reducing repetitive report writing. Methods: AI was used to efficiently generate personalized breast ultrasound screening preliminary reports, especially for…
UGGNet: Bridging U-Net and VGG for Advanced Breast Cancer Diagnosis
School of Engineering, Tokyo Institute of Technology（东京技术大学工学院）；Faculty of Information Technology Nguyen Tat Thanh University（Nguyen Tat Thanh大学信息科技学院）
Korea University（高丽大学）；Hankik University of Foreign Studies（韩国外国语大学）
We propose a method for effectively utilizing weakly annotated image data in an object detection tasks of breast ultrasound images. Given the problem setting where a small, strongly annotated dataset and a large, weakly annotated dataset with no bounding box information are available, training an object detection model becomes a non-trivial problem. We suggest a controlled weight for handling the effect of weakly annotated images in a two stage object detection model. We~also present a subsequent active learning scheme for safely assigning weakly annotated images a strong annotation using the …
Institute of Fundamental Technological Research, Polish Academy of Sciences（波兰科学院基础技术研究所）；University of California, San Diego（加州大学圣迭戈分校）
Morphological features play an important role in breast mass classification in sonography. While benign breast masses tend to have a well-defined ellipsoidal contour, shape of malignant breast masses is commonly ill-defined and highly variable. Various handcrafted morphological features have been developed over the years to assess this phenomenon and help the radiologists differentiate benign and malignant masses. In this paper we propose an automatic approach to morphology analysis, we express shapes of breast masses as points on the Kendall's shape manifold. Next, we use the full Procrustes …
Shenzhen University（深圳大学）；Guangdong Province Traditional Chinese Medical Hospital（广东省中医院）
Automated breast ultrasound (ABUS) is a new and promising imaging modality for breast cancer detection and diagnosis, which could provide intuitive 3D information and coronal plane information with great diagnostic value. However, manually screening and diagnosing tumors from ABUS images is very time-consuming and overlooks of abnormalities may happen. In this study, we propose a novel two-stage 3D detection network for locating suspected lesion areas and further classifying lesions as benign or malignant tumors. Specifically, we propose a 3D detection network rather than frequently-used segme…
Nankai University（南开大学）；Civil Aviation General Hospital（民航总医院）；Chinese PLA General Hospital（中国人民解放军总医院）
Breast lesions segmentation is an important step of computer-aided diagnosis system, and it has attracted much attention. However, accurate segmentation of malignant breast lesions is a challenging task due to the effects of heterogeneous structure and similar intensity distributions. In this paper, a novel bidirectional aware guidance network (BAGNet) is proposed to segment the malignant lesion from breast ultrasound images. Specifically, the bidirectional aware guidance network is used to capture the context between global (low-level) and local (high-level) features from the input coarse sal…
Northeastern University（东北大学）；Sino-Dutch Biomedical and Information Engineering School（中荷生物医学与信息工程学院）
Ultrasound image diagnosis of breast tumors has been widely used in recent years. However, there are some problems of it, for instance, poor quality, intense noise and uneven echo distribution, which has created a huge obstacle to diagnosis. To overcome these problems, we propose a novel method, a breast cancer classification with ultrasound images based on SLIC (BCCUI). We first utilize the Region of Interest (ROI) extraction based on Simple Linear Iterative Clustering (SLIC) algorithm and region growing algorithm to extract the ROI at the super-pixel level. Next, the features of ROI are extr…
Breast cancer is the most common invasive cancer in women. Besides the primary B-mode ultrasound screening, sonographers have explored the inclusion of Doppler, strain and shear-wave elasticity imaging to advance the diagnosis. However, recognizing useful patterns in all types of images and weighing up the significance of each modality can elude less-experienced clinicians. In this paper, we explore, for the first time, an automatic way to combine the four types of ultrasonography to discriminate between benign and malignant breast nodules. A novel multimodal network is proposed, along with pr…
East Delta University（东三角洲大学）；University of Chittagong（吉大港大学）
Breast cancer has become a symbol of tremendous concern in the modern world, as it is one of the major causes of cancer mortality worldwide. In this regard, breast ultrasonography images are frequently utilized by doctors to diagnose breast cancer at an early stage. However, the complex artifacts and heavily noised breast ultrasonography images make diagnosis a great challenge. Furthermore, the ever-increasing number of patients being screened for breast cancer necessitates the use of automated end-to-end technology for highly accurate diagnosis at a low cost and in a short time. In this conce…
Illinois Institute of Technology（伊利诺伊理工学院）；Rutgers University（罗格斯大学）
Automatically segmenting lesion area in breast ultrasound (BUS) images is a challenging one due to its noise, speckle and artifacts. Edge-map of BUS images also does not help because in most cases the edge-map gives no information whatsoever. Almost all segmentation technique takes the edge-map of the image as its first step, though there are a few algorithms that try to avoid edge-maps as well. Improving the edge-map of breast ultrasound images theoretically improves the chances of automatic segmentation to be more precise. In this paper, we propose a semi-automatic technique of histogram spl…
South China University of Technology（华南理工大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）
Ultrasonography is an important routine examination for breast cancer diagnosis, due to its non-invasive, radiation-free and low-cost properties. However, the diagnostic accuracy of breast cancer is still limited due to its inherent limitations. It would be a tremendous success if we can precisely diagnose breast cancer by breast ultrasound images (BUS). Many learning-based computer-aided diagnostic methods have been proposed to achieve breast cancer diagnosis/lesion classification. However, most of them require a pre-define ROI and then classify the lesion inside the ROI. Conventional classif…
The advancements in deep learning technologies have produced immense contributions to biomedical image analysis applications. With breast cancer being the common deadliest disease among women, early detection is the key means to improve survivability. Medical imaging like ultrasound presents an excellent visual representation of the functioning of the organs; however, for any radiologist analysing such scans is challenging and time consuming which delays the diagnosis process. Although various deep learning based approaches are proposed that achieved promising results, the present article intr…
Is Two-shot All You Need? A Label-efficient Approach for Video Segmentation in Breast Ultrasound
Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging（广东省生物医学测量与超声成像重点实验室）；School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University（深圳大学医学部生物医学工程学院）；Shenzhen University（深圳大学）
College of Information Science and Technology, Northwest University（西北大学信息科学与技术学院）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Breast ultrasound (US) is an effective imaging modality for breast cancer detec-tion and diagnosis. The structural characteristics of breast lesion play an im-portant role in Computer-Aided Diagnosis (CAD). In this paper, a novel struc-ture-aware triplet path networks (SATPN) was designed to integrate classifica-tion and two image reconstruction tasks to achieve accurate diagnosis on US im-ages with small training dataset. Specifically, we enhance clinically-approved breast lesion structure characteristics though converting original breast US imag-es to BIRADS-oriented feature maps (BFMs) with…
School of Electrical Engineering, Tel-Aviv University（特拉维夫大学电气工程学院）；The Manuel and Raquel Klachky Chair of Image Processing, School of Electrical Engineering, Tel-Aviv University（特拉维夫大学电气工程学院 Manuel and Raquel Klachky 图像处理教授职位）；Diagnostic Imaging, Sheba Medical Center, affiliated to the Sackler School of Medicine, Tel-Aviv University（舍巴医疗中心诊断影像科，隶属于特拉维夫大学萨克勒医学院）
Mammography and ultrasound are extensively used by radiologists as complementary modalities to achieve better performance in breast cancer diagnosis. However, existing computer-aided diagnosis (CAD) systems for the breast are generally based on a single modality. In this work, we propose a deep-learning based method for classifying breast cancer lesions from their respective mammography and ultrasound images. We present various approaches and show a consistent improvement in performance when utilizing both modalities. The proposed approach is based on a GoogleNet architecture, fine-tuned for o…
Charles Stark Draper Laboratory（查尔斯·斯塔克·德拉普尔实验室）；Stanford Medicine（斯坦福医学）；Bracco Diagnostics Corporation（布拉科诊断公司）
Molecularly targeted contrast enhanced ultrasound (mCEUS) is a clinically promising approach for early cancer detection through targeted imaging of VEGFR2 (KDR) receptors. We have developed computational enhancement techniques for mCEUS tailored to address the unique challenges of imaging contrast accumulation in humans. These techniques utilize dynamic analysis to distinguish molecularly bound contrast agent from other contrast-mode signal sources, enabling analysis of contrast agent accumulation to be performed during contrast bolus arrival when the signal due to molecular binding is stronge…
Computer-aided detection (CAD) of benign and malignant breast lesions becomes increasingly essential in breast ultrasound (US) imaging. The CAD systems rely on imaging features identified by the medical experts for their performance, whereas deep learning (DL) methods automatically extract features from the data. The challenge of the DL is the insufficiency of breast US images available to train the DL models. Here, we present an ensemble transfer learning model to classify benign and malignant breast tumors using B-mode breast US (B-US) and strain elastography breast US (SE-US) images. This m…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mayo Clinic（梅奥诊所）；Washington University in St. Louis（圣路易斯华盛顿大学）；Beckman Institute, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校贝克曼研究所）
Automatic breast lesion detection and classification is an important task in computer-aided diagnosis, in which breast ultrasound (BUS) imaging is a common and frequently used screening tool. Recently, a number of deep learning-based methods have been proposed for joint localization and classification of breast lesions using BUS images. In these methods, features extracted by a shared network trunk are appended by two independent network branches to achieve classification and localization. Improper information sharing might cause conflicts in feature optimization in the two branches and leads …
Hybrid Attention Network for Accurate Breast Tumor Segmentation in Ultrasound Images
School of Information Engineering, Xi’an Eurasia University（信息工程学院，西安欧亚大学）；Department of Computer Science, University of Engineering and Technology Lahore（计算机科学系，拉合尔工程科技大学）
University of Rochester（罗切斯特大学）；University of Rochester Medical Center（罗切斯特大学医学中心）
The improved diagnostic accuracy of ultrasound breast examinations remains an important goal. In this study, we propose a biophysical feature based machine learning method for breast cancer detection to improve the performance beyond a benchmark deep learning algorithm and to furthermore provide a color overlay visual map of the probability of malignancy within a lesion. This overall framework is termed disease specific imaging. Previously, 150 breast lesions were segmented and classified utilizing a modified fully convolutional network and a modified GoogLeNet, respectively. In this study mul…
Universitat Rovira i Virgili（罗维拉-威尔吉利大学）；Bioinformatics Institute（生物信息学研究所）
This paper proposes an efficient solution for tumor segmentation and classification in breast ultrasound (BUS) images. We propose to add an atrous convolution layer to the conditional generative adversarial network (cGAN) segmentation model to learn tumor features at different resolutions of BUS images. To automatically re-balance the relative impact of each of the highest level encoded features, we also propose to add a channel-wise weighting block in the network. In addition, the SSIM and L1-norm loss with the typical adversarial loss are used as a loss function to train the model. Our model…
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；National University of Singapore（新加坡国立大学）；Shenzhen University（深圳大学）；School of Biomedical Engineering and Informatics, Nanjing Medical University（南京医科大学生物医学工程与信息学院）；Pohang University of Science and Technology (POSTECH)（浦项科技大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Heidelberg University（海德堡大学）；Helmholtz Imaging（亥姆霍兹成像机构）
Breast cancer is one of the most common causes of death among women worldwide. Early detection helps in reducing the number of deaths. Automated 3D Breast Ultrasound (ABUS) is a newer approach for breast screening, which has many advantages over handheld mammography such as safety, speed, and higher detection rate of breast cancer. Tumor detection, segmentation, and classification are key components in the analysis of medical images, especially challenging in the context of 3D ABUS due to the significant variability in tumor size and shape, unclear tumor boundaries, and a low signal-to-noise r…
Politecnico di Milano（米兰理工大学）；Istituto Italiano di Tecnologia（意大利技术研究院）；Università Politecnica delle Marche（马尔凯理工大学）
Breast cancer is one of the deadliest cancer worldwide. Timely detection could reduce mortality rates. In the clinical routine, classifying benign and malignant tumors from ultrasound (US) imaging is a crucial but challenging task. An automated method, which can deal with the variability of data is therefore needed. In this paper, we compared different Convolutional Neural Networks (CNNs) and transfer learning methods for the task of automated breast tumor classification. The architectures investigated in this study were VGG-16 and Inception V3. Two different training strategies were investiga…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Incorporating human domain knowledge for breast tumor diagnosis is challenging, since shape, boundary, curvature, intensity, or other common medical priors vary significantly across patients and cannot be employed. This work proposes a new approach for integrating visual saliency into a deep learning model for breast tumor segmentation in ultrasound images. Visual saliency refers to image maps containing regions that are more likely to attract radiologists visual attention. The proposed approach introduces attention blocks into a U-Net architecture, and learns feature representations that prio…
Beyond Traditional Approaches: Multi-Task Network for Breast Ultrasound Diagnosis
University of Technology and Education, Vietnam（越南技术教育大学）；National Cheng Kung University, Taiwan（国立成功大学）；FPT University Ha Noi, Vietnam（FPT大学河内校区）；Ho Chi Minh City Medicine and Pharmacy University, Vietnam（胡志明市医药大学）；Taipei Medical University, Taiwan（台北医学大学）；Vietnamese-German University, Vietnam（越南德国大学）
Shifting More Attention to Breast Lesion Segmentation in Ultrasound Videos
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
University of Delaware（特拉华大学）；University of Guilan（吉兰大学）；Eindhoven University of Technology（埃因霍温理工大学）；Iran University of Science and Technology（伊朗科学技术大学）
Automated 3-D breast ultrasound (ABUS) is a newfound system for breast screening that has been proposed as a supplementary modality to mammography for breast cancer detection. While ABUS has better performance in dense breasts, reading ABUS images is exhausting and time-consuming. So, a computer-aided detection system is necessary for interpretation of these images. Mass segmentation plays a vital role in the computer-aided detection systems and it affects the overall performance. Mass segmentation is a challenging task because of the large variety in size, shape, and texture of masses. Moreov…
Ultrasound (US) is one of the most commonly used imaging modalities in both diagnosis and surgical interventions due to its low-cost, safety, and non-invasive characteristic. US image segmentation is currently a unique challenge because of the presence of speckle noise. As manual segmentation requires considerable efforts and time, the development of automatic segmentation algorithms has attracted researchers attention. Although recent methodologies based on convolutional neural networks have shown promising performances, their success relies on the availability of a large number of training d…
Institute of Fundamental Technological Research, Polish Academy of Sciences（波兰科学院基础技术研究所）
Standard classification methods based on handcrafted morphological and texture features have achieved good performance in breast mass differentiation in ultrasound (US). In comparison to deep neural networks, commonly perceived as "black-box" models, classical techniques are based on features that have well-understood medical and physical interpretation. However, classifiers based on morphological features commonly underperform in the presence of the shadowing artifact and ill-defined mass borders, while texture based classifiers may fail when the US image is too noisy. Therefore, in practice …
Shanghai Jiao Tong University（上海交通大学）；Shanghai General Hospital（上海市第一人民医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）
Previous deep learning based Computer Aided Diagnosis (CAD) system treats multiple views of the same lesion as independent images. Since an ultrasound image only describes a partial 2D projection of a 3D lesion, such paradigm ignores the semantic relationship between different views of a lesion, which is inconsistent with the traditional diagnosis where sonographers analyze a lesion from at least two views. In this paper, we propose a multi-task framework that complements Benign/Malignant classification task with lesion recognition (LR) which helps leveraging relationship among multiple views …
Shanghai Jiao Tong University（上海交通大学）；Shanghai General Hospital（上海市第一人民医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）；AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院）
Much progress has been made in the deep neural network (DNN) based diagnosis of mass lesions breast ultrasound (BUS) images. However, the non-mass lesion is less investigated because of the limited data. Based on the insight that mass data is sufficient and shares the same knowledge structure with non-mass data of identifying the malignancy of a lesion based on the ultrasound image, we propose a novel transfer learning framework to enhance the generalizability of the DNN model for non-mass BUS with the help of mass BUS. Specifically, we train a shared DNN with combined non-mass and mass data. …
