Tumor Saliency Estimation for Breast Ultrasound Images via Breast Anatomy Modeling
Utah State University（犹他州立大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Idaho（爱达荷大学）；The Affiliated Hospital of Qingdao University（青岛大学附属医院）；The Second Hospital of Hebei Medical University（河北医科大学第二医院）
Fei Xu, Yingtao Zhang, Min Xian, H. D. Cheng, Boyu Zhang, Jianrui Ding, Chunping Ning, Ying Wang
Tumor saliency estimation aims to localize tumors by modeling the visual stimuli in medical images. However, it is a challenging task for breast ultrasound due to the complicated anatomic structure of the breast and poor image quality; and existing saliency estimation approaches only model generic visual stimuli, e.g., local and global contrast, location, and feature correlation, and achieve poor performance for tumor saliency estimation. In this paper, we propose a novel optimization model to estimate tumor saliency by utilizing breast anatomy. First, we model breast anatomy and decompose breast ultrasound image into layers using Neutro-Connectedness; then utilize the layers to generate the foreground and background maps; and finally propose a novel objective function to estimate the tumor saliency by integrating the foreground map, background map, adaptive center bias, and region-based correlation cues. The extensive experiments demonstrate that the proposed approach obtains more accurate foreground and background maps with the assistance of the breast anatomy; especially, for the images having large or small tumors; meanwhile, the new objective function can handle the images without tumors. The newly proposed method achieves state-of-the-art performance when compared to eight tumor saliency estimation approaches using two breast ultrasound datasets.
The University of Adelaide（阿德莱德大学）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）；Delft University of Technology（代尔夫特理工大学）；Queensland University of Technology（昆士兰科技大学）
There is a heated debate on how to interpret the decisions provided by deep learning models (DLM), where the main approaches rely on the visualization of salient regions to interpret the DLM classification process. However, these approaches generally fail to satisfy three conditions for the problem of lesion detection from medical images: 1) for images with lesions, all salient regions should represent lesions, 2) for images containing no lesions, no salient region should be produced,and 3) lesions are generally small with relatively smooth borders. We propose a new model-agnostic paradigm to …
Utah State University（犹他州立大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Idaho（爱达荷大学）
Breast cancer investigation is of great significance, and developing tumor detection methodologies is a critical need. However, it is a challenging task for breast ultrasound due to the complicated breast structure and poor quality of the images. In this paper, we propose a novel tumor saliency estimation model guided by enriched breast anatomy knowledge to localize the tumor. Firstly, the breast anatomy layers are generated by a deep neural network. Then we refine the layers by integrating a non-semantic breast anatomy model to solve the problems of incomplete mammary layers. Meanwhile, a new…
Utah State University（犹他州立大学）；Harbin Institute of Technology（哈尔滨工业大学）；the First Affiliated Hospital of Harbin Medical University（哈尔滨医科大学附属第一医院）
Breast cancer is one of the most serious disease affecting women's health. Due to low cost, portable, no radiation, and high efficiency, breast ultrasound (BUS) imaging is the most popular approach for diagnosing early breast cancer. However, ultrasound images are low resolution and poor quality. Thus, developing accurate detection system is a challenging task. In this paper, we propose a fully automatic segmentation algorithm consisting of two parts: fuzzy fully convolutional network and accurately fine-tuning post-processing based on breast anatomy constraints. In the first part, the image i…
Cancer is one of the most common and fatal diseases in the world. Breast cancer affects one in every eight women and one in every eight hundred men. Hence, our prime target should be early detection of cancer because the early detection of cancer can be helpful to cure cancer effectively. Therefore, we propose a saliency detection system with the help of advanced deep learning techniques, such that the machine will be taught to emulate actions of pathologists for localization of diagnostically pertinent regions. We study identification of five diagnostic categories of breast cancer by training…
LoGo-MR: Screening Breast MRI for Cancer Risk Prediction by Efficient Omni-Slice Modeling
Netherlands Cancer Institute（荷兰癌症研究所）；Radboud University Medical Center（拉德堡德大学医学中心）；University Medical Center Utrecht（乌得勒支大学医学中心）；Macao Polytechnic University（澳门理工大学）
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Efficient, reliable and reproducible target volume delineation is a key step in the effective planning of breast radiotherapy. However, post-operative breast target delineation is challenging as the contrast between the tumor bed volume (TBV) and normal breast tissue is relatively low in CT images. In this study, we propose to mimic the marker-guidance procedure in manual target delineation. We developed a saliency-based deep learning segmentation (SDL-Seg) algorithm for accurate TBV segmentation in post-operative breast irradiation. The SDL-Seg algorithm incorporates saliency information in t…
Evaluating Post-hoc Interpretability with Intrinsic Interpretability
University of Coimbra（科英布拉大学）；IPO-Porto Research Centre（波尔图肿瘤学研究所研究中心）；University of Applied Sciences Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州高等专业学院））
Microsoft Research（微软研究院）；TuSimple（图森未来）；Baidu Research（百度研究院）；Tencent Medical AI Labs（腾讯医疗AI实验室）；PingAn Tech（平安科技）
Medical imaging is an essential tool in many areas of medical applications, used for both diagnosis and treatment. However, reading medical images and making diagnosis or treatment recommendations require specially trained medical specialists. The current practice of reading medical images is labor-intensive, time-consuming, costly, and error-prone. It would be more desirable to have a computer-aided system that can automatically make diagnosis and treatment recommendations. Recent advances in deep learning enable us to rethink the ways of clinician diagnosis based on medical images. In this t…
Utah State University（犹他州立大学）；University of Idaho（爱达荷大学）；Harbin Institute of Technology（哈尔滨工业大学）；The Affiliated Hospital of Qingdao University（青岛大学附属医院）；The Second Hospital of Hebei Medical University（河北医科大学第二医院）
Automatic tumor segmentation of breast ultrasound (BUS) image is quite challenging due to the complicated anatomic structure of breast and poor image quality. Most tumor segmentation approaches achieve good performance on BUS images collected in controlled settings; however, the performance degrades greatly with BUS images from different sources. Tumor saliency estimation (TSE) has attracted increasing attention to solving the problem by modeling radiologists' attention mechanism. In this paper, we propose a novel hybrid framework for TSE, which integrates both high-level domain-knowledge and …
South China University of Technology（华南理工大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）
Ultrasonography is an important routine examination for breast cancer diagnosis, due to its non-invasive, radiation-free and low-cost properties. However, the diagnostic accuracy of breast cancer is still limited due to its inherent limitations. It would be a tremendous success if we can precisely diagnose breast cancer by breast ultrasound images (BUS). Many learning-based computer-aided diagnostic methods have been proposed to achieve breast cancer diagnosis/lesion classification. However, most of them require a pre-define ROI and then classify the lesion inside the ROI. Conventional classif…
Center for Data Science, New York University（纽约大学数据科学中心）；Department of Computer Science, Courant Institute, New York University（纽约大学库朗研究所计算机科学系）；Department of Radiology, NYU School of Medicine（纽约大学医学院放射科）；Perlmutter Cancer Center, NYU Langone Health（纽约大学朗格尼健康系统珀尔马特癌症中心）；Center for Advanced Imaging Innovation and Research, NYU Langone Health（纽约大学朗格尼健康系统先进成像创新与研究中心）；CIFAR Associate Fellow（CIFAR研究员（注：CIFAR是加拿大英属哥伦比亚大学的一个研究机构，但根据提取规则，此处仅提取“CIFAR”相关部分，严格按英文保留））
Medical images differ from natural images in significantly higher resolutions and smaller regions of interest. Because of these differences, neural network architectures that work well for natural images might not be applicable to medical image analysis. In this work, we extend the globally-aware multiple instance classifier, a framework we proposed to address these unique properties of medical images. This model first uses a low-capacity, yet memory-efficient, network on the whole image to identify the most informative regions. It then applies another higher-capacity network to collect detail…
Are Vision Foundation Models Ready for Out-of-the-Box Medical Image Registration?
Department of Electrical and Computer Engineering, Duke University（电子工程与计算机科学系，杜克大学）；Department of Biostatistics and Bioinformatics, Duke University（生物统计学与生物信息学系，杜克大学）；Departments of Biostatistics and Bioinformatics, Radiology, Electrical and Computer Engineering, and Computer Science, Duke University（生物统计学与生物信息学系、放射学、电子工程与计算机科学系，杜克大学）
Medical Slice Transformer: Improved Diagnosis and Explainability on 3D Medical Images with DINOv2
University Hospital Aachen（亚琛大学医院）；Technical University Dresden（德累斯顿工业大学）；University Hospital Dresden（德累斯顿大学医院）；University Hospital Heidelberg（海德堡大学医院）
Clinical decision support using deep neural networks has become a topic of steadily growing interest. While recent work has repeatedly demonstrated that deep learning offers major advantages for medical image classification over traditional methods, clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. In recent years, this has been addressed by a variety of approaches that have successfully contributed to providing deeper insight. Most notably, additive feature attribution methods are…
Recent advances in deep learning have achieved promising performance for medical image analysis, while in most cases ground-truth annotations from human experts are necessary to train the deep model. In practice, such annotations are expensive to collect and can be scarce for medical imaging applications. Therefore, there is significant interest in learning representations from unlabelled raw data. In this paper, we propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video without any type of human annotation. We assume that in …
Cross-attention-based saliency inference for predicting cancer metastasis on whole slide images
Center for Biomedical Informatics, Wake Forest University School of Medicine（维克森林大学医学院生物医学信息学中心）；Department of Pathology, Wake Forest University School of Medicine（维克森林大学医学院病理学系）
Nankai University（南开大学）；Civil Aviation General Hospital（民航总医院）；Chinese PLA General Hospital（中国人民解放军总医院）
Breast lesions segmentation is an important step of computer-aided diagnosis system, and it has attracted much attention. However, accurate segmentation of malignant breast lesions is a challenging task due to the effects of heterogeneous structure and similar intensity distributions. In this paper, a novel bidirectional aware guidance network (BAGNet) is proposed to segment the malignant lesion from breast ultrasound images. Specifically, the bidirectional aware guidance network is used to capture the context between global (low-level) and local (high-level) features from the input coarse sal…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Breast tumor segmentation is a critical task in computer-aided diagnosis (CAD) systems for breast cancer detection because accurate tumor size, shape and location are important for further tumor quantification and classification. However, segmenting small tumors in ultrasound images is challenging, due to the speckle noise, varying tumor shapes and sizes among patients, and the existence of tumor-like image regions. Recently, deep learning-based approaches have achieved great success for biomedical image analysis, but current state-of-the-art approaches achieve poor performance for segmenting …
NYU Center for Data Science（纽约大学数据科学中心）；Courant Institute of Mathematical Sciences, NYU（纽约大学库朗数学科学研究所）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Jagiellonian University（雅盖隆大学）
In the last few years, deep learning classifiers have shown promising results in image-based medical diagnosis. However, interpreting the outputs of these models remains a challenge. In cancer diagnosis, interpretability can be achieved by localizing the region of the input image responsible for the output, i.e. the location of a lesion. Alternatively, segmentation or detection models can be trained with pixel-wise annotations indicating the locations of malignant lesions. Unfortunately, acquiring such labels is labor-intensive and requires medical expertise. To overcome this difficulty, weakl…
NYU Langone Health（纽约大学朗格尼医学中心）；Jagiellonian University（雅盖隆大学）；New York University（纽约大学）
3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alternative, a neural network that enables efficient classification of full-resolution 3D medical images. Compared to off-the-shelf convolutional neural networks, our network, 3D Globally-Aware Multiple …
Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI
Cambridge University Hospitals（剑桥大学医院）；Mitera Hospital（米特拉医院）；Radboud University Medical Center（拉德堡德大学医学中心）；University Hospital Aachen（亚琛大学医院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Ribera Hospital（里贝拉医院）；Duke Breast Cancer MRI（杜克乳腺癌磁共振成像项目）
Ultrasound is the second most used modality in medical imaging. It is cost effective, hazardless, portable and implemented routinely in numerous clinical procedures. Nonetheless, image quality is characterized by granulated appearance, poor SNR and speckle noise. Specific for malignant tumors, the margins are blurred and indistinct. Thus, there is a great need for improving ultrasound image quality. We hypothesize that this can be achieved, using neural networks, by translation into a more realistic display which mimics an anatomical cut through the tissue. In order to achieve this goal, the p…
Hierarchical Perfusion Graphs for Tumor Heterogeneity Modeling in Glioma Molecular Subtyping
Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, South Korea（生物工程跨学科项目，首尔国立大学，首尔，韩国）；Interdisciplinary Program in Cancer Biology, Seoul National University College of Medicine, Seoul, South Korea（癌症生物学跨学科项目，首尔国立大学医学院，首尔，韩国）；Dept. of Biomedical Sciences, Seoul National University, Seoul, South Korea（生物医学科学系，首尔国立大学，首尔，韩国）；Dept. of Electronic Engineering, Soongsil University, Seoul, South Korea（电子工程系，顺世大学，首尔，韩国）；Dept. of Radiology, Seoul National University Hospital, Seoul, South Korea（放射科，首尔国立大学医院，首尔，韩国）；Dept. of Radiology, Seoul National University College of Medicine, Seoul, South Korea（放射科，首尔国立大学医学院，首尔，韩国）；Healthcare AI Research Institute, Seoul National University Hospital, Seoul, South Korea（医疗人工智能研究院，首尔国立大学医院，首尔，韩国）
